Linked e-resources
Details
Table of Contents
Intro
Foreword
Preface
Who This Book Is For
How This Book Is Organized
What You Need for This Book
Typographical Conventions
Acknowledgments
Contents
About the Authors
Part I Heron Fundamentals
1 Stream Processing
1.1 Big Data Processing
1.1.1 Lambda Architecture
1.1.1.1 Batch Processing Layer
1.1.1.2 Stream Processing Layer
1.1.1.3 Serving Layer
1.1.2 Kappa Architecture
1.2 Big Data Stream Processing
1.3 From Apache Storm to Apache Heron (Incubating)
1.3.1 Motivation for Heron
1.3.2 Heron Design Goals
1.3.3 Join the Apache Heron (Incubating) Community
1.4 Stream Processing Tools
1.5 Summary
References
2 Heron Basics
2.1 Topology Data Model
2.1.1 Topology
2.1.2 Spout
2.1.3 Bolt
2.1.4 Grouping
2.2 Heron Architecture and Components
2.2.1 Cluster-Level Components (Six Components)
2.2.1.1 Scheduler
2.2.1.2 State Manager
2.2.1.3 Uploader
2.2.1.4 Heron CLI
2.2.1.5 Heron Tracker
2.2.1.6 Heron UI
2.2.2 Topology-Level Components (Four Components)
2.2.2.1 Heron Instance
2.2.2.2 Stream Manager
2.2.2.3 Topology Master
2.2.2.4 Metrics Manager
2.3 Submission Process and Failure Handling
2.4 Submit the First Topology
2.4.1 Preparation
2.4.2 Install the Heron Client
2.4.3 Heron Example Topologies
2.4.4 Submit the Topology JAR File
2.4.5 Observe the Running Topology
2.5 Summary
References
3 Study Heron Code
3.1 Code Languages
3.2 Requirements for Compiling
3.3 Prepare the Compiling Environment
3.4 Source Organization
3.4.1 Directory Organization
3.4.2 Bazel Perspective
3.5 Compile Heron
3.6 Examine Compiling Results
3.6.1 Examine the API
3.6.2 Examine Packages
3.7 Run Tests
3.7.1 Unit Test
3.7.2 Integration Test
3.8 Summary
References
Part II Write Heron Topologies
4 Migrate Storm Topology to Heron
4.1 Prepare the Storm Topology Code
4.1.1 Examine the Storm Topology Code
4.1.2 Examine the Storm Flux Code
4.2 Migrate the Storm Topology Code to a HeronTopology Project
4.2.1 Adjust the Topology Java Code
4.2.2 Adjust the Project File pom.xml
4.2.2.1 Add Dependency
4.2.2.2 Build with Dependencies
4.2.3 Compile the Topology JAR File
4.3 Migrate Storm Flux to Heron ECO
4.4 Summary
References
5 Write Topology Code
5.1 Before Writing Code
5.1.1 Design Topology
5.1.2 Choose a Heron API
5.2 Write Topology in Java
5.2.1 Code the Topology
5.2.1.1 Code Main
5.2.1.2 Code Spout
5.2.1.3 Code Bolt
5.2.2 Understand Tuple Flow
5.2.2.1 How Tuple Is Constructed
5.2.2.2 How Tuple Is Routed
5.3 Write Topology in Python
5.3.1 Code Main
5.3.2 Code Spout
5.3.3 Code Bolt
5.3.4 Compile and Run
5.4 Summary
Reference
6 Heron Topology Features
6.1 Delivery Semantics
6.1.1 At-Least-Once
6.1.2 Effectively-Once
6.1.2.1 Requirements for Effectively-Once
6.1.2.2 Exactly-Once Versus Effectively-Once.
Foreword
Preface
Who This Book Is For
How This Book Is Organized
What You Need for This Book
Typographical Conventions
Acknowledgments
Contents
About the Authors
Part I Heron Fundamentals
1 Stream Processing
1.1 Big Data Processing
1.1.1 Lambda Architecture
1.1.1.1 Batch Processing Layer
1.1.1.2 Stream Processing Layer
1.1.1.3 Serving Layer
1.1.2 Kappa Architecture
1.2 Big Data Stream Processing
1.3 From Apache Storm to Apache Heron (Incubating)
1.3.1 Motivation for Heron
1.3.2 Heron Design Goals
1.3.3 Join the Apache Heron (Incubating) Community
1.4 Stream Processing Tools
1.5 Summary
References
2 Heron Basics
2.1 Topology Data Model
2.1.1 Topology
2.1.2 Spout
2.1.3 Bolt
2.1.4 Grouping
2.2 Heron Architecture and Components
2.2.1 Cluster-Level Components (Six Components)
2.2.1.1 Scheduler
2.2.1.2 State Manager
2.2.1.3 Uploader
2.2.1.4 Heron CLI
2.2.1.5 Heron Tracker
2.2.1.6 Heron UI
2.2.2 Topology-Level Components (Four Components)
2.2.2.1 Heron Instance
2.2.2.2 Stream Manager
2.2.2.3 Topology Master
2.2.2.4 Metrics Manager
2.3 Submission Process and Failure Handling
2.4 Submit the First Topology
2.4.1 Preparation
2.4.2 Install the Heron Client
2.4.3 Heron Example Topologies
2.4.4 Submit the Topology JAR File
2.4.5 Observe the Running Topology
2.5 Summary
References
3 Study Heron Code
3.1 Code Languages
3.2 Requirements for Compiling
3.3 Prepare the Compiling Environment
3.4 Source Organization
3.4.1 Directory Organization
3.4.2 Bazel Perspective
3.5 Compile Heron
3.6 Examine Compiling Results
3.6.1 Examine the API
3.6.2 Examine Packages
3.7 Run Tests
3.7.1 Unit Test
3.7.2 Integration Test
3.8 Summary
References
Part II Write Heron Topologies
4 Migrate Storm Topology to Heron
4.1 Prepare the Storm Topology Code
4.1.1 Examine the Storm Topology Code
4.1.2 Examine the Storm Flux Code
4.2 Migrate the Storm Topology Code to a HeronTopology Project
4.2.1 Adjust the Topology Java Code
4.2.2 Adjust the Project File pom.xml
4.2.2.1 Add Dependency
4.2.2.2 Build with Dependencies
4.2.3 Compile the Topology JAR File
4.3 Migrate Storm Flux to Heron ECO
4.4 Summary
References
5 Write Topology Code
5.1 Before Writing Code
5.1.1 Design Topology
5.1.2 Choose a Heron API
5.2 Write Topology in Java
5.2.1 Code the Topology
5.2.1.1 Code Main
5.2.1.2 Code Spout
5.2.1.3 Code Bolt
5.2.2 Understand Tuple Flow
5.2.2.1 How Tuple Is Constructed
5.2.2.2 How Tuple Is Routed
5.3 Write Topology in Python
5.3.1 Code Main
5.3.2 Code Spout
5.3.3 Code Bolt
5.3.4 Compile and Run
5.4 Summary
Reference
6 Heron Topology Features
6.1 Delivery Semantics
6.1.1 At-Least-Once
6.1.2 Effectively-Once
6.1.2.1 Requirements for Effectively-Once
6.1.2.2 Exactly-Once Versus Effectively-Once.