Kafka

Description

Apache Kafka  distributed system is made up of a cluster of one or more servers (called Kafka brokers) that can span multiple datacenters (or cloud instances) and clients, allowing you to create applications that communicate with Kafka brokers to read, write, and process streams of events, Apache storm

Course Content 

1: Kafka Overview

• High-Level Architecture
• Common Use Cases
• Cloudera’s Distribution of Apache Kafka

2: Deploying Apache Kafka

• System Requirements and Dependencies
• Service Roles
• Planning Your Deployment Deploying Kafka Services
• Exercise: Preparing the Exercise Environment
• Exercise: Installing the  Service with Cloudera Manager
• Exercise (optional): Create Metrics Dashboards
• Exercise (optional): Using the CM API

3: Kafka Command Line Basics

• Create and Manage Topics
• Running Producers and Consumers

4: Using Streams Messaging Manager (SMM)

• Streams Messaging Manager Overview
• Producers, Topics, and Consumers
• Data Explorer
• Brokers
• Topic Management
• Exercise: Managing Topics using the CLI
• Exercise: Connecting Producers and Consumers from the Command Line

5: Kafka Java API Basics

• Overview of Kafka’s APIs
• Topic Management from the Java API
• Exercise (optional): Managing Kafka Topics Using the Java API
• Using Producers and Consumers from the Java API
• Exercise: Developing Producers and Consumers with the Java API

6: Improving Availability through Replication

• Replication
• Exercise: Observing Downtime Due to Broker Failure
• Considerations for the Replication Factor
• Exercise: Adding Replicas to Improve Availability

7: Improving Application Scalability

• Partitioning
• How Messages are partitioned
• Exercise: Observing How Partitioning Affects Performance
• Consumer Groups
• Exercise: Implementing Consumer Groups
• Consumer Re-balancing
• Exercise: Using a Key to Control Partition Assignment

8: Improving Application Reliability

• Delivery Semantics
• Demonstration (optional): ISRs vs. ACKs
• Producer Delivery
• Exercise: Idempotent Producer
• Transactions
• Exercise: Transactional Producers and Consumers
• Handling Consumer Failure
• Offset Management
• Exercise: Detecting and Suppressing Duplicate Messages
• Exercise: Handling Invalid Records
• Handling Producer Failure

9: Analyzing Kafka Clusters with SMM

• End-to-End Latency
• Notifiers
• Alert Policies
• Use Cases

10: Monitoring Kafka

• Monitoring Overview
• Monitoring using Cloudera Manager
• Charts and Reports in CM
• Monitoring Recommendations
• Metrics for Troubleshooting
• Diagnosing Service Failure
• Exercise: Monitoring Kafka

11: Managing Kafka

• Managing Kafka Topic Storage
• Demonstration (optional): Message Retention Period
• Log Cleanup and Collection
• Re-balancing Partitions
• Cruise Control
• Exercise: Installing Cruise Control
• Exercise: Troubleshooting Kafka Topics
• Unclean Leader Election
• Exercise: Unclean Leader Election
• Adding and Removing Brokers
• Exercise: Adding and Removing Brokers
• Best Practices

12: Message Structure, Format, and Versioning

• Message Structure
• Schema Registry
• Defining Schemas
• Schema Evolution and Versioning
• Schema Registry Client
• Exercise: Using an Avro Schema

13: Improving Application Performance

• Message Size
• Batching
• Compression
• Exercise: Observing How Compression Affects Performance

14: Improving Kafka Service Performance

• Performance Tuning Strategies for the Administrator
• Cluster Sizing
• Exercise: Planning Capacity Needed for a Use Case

15: Securing the Kafka Cluster

• Encryption
• Authentication
• Authorization
• Auditing

Please Visit  Kafka Official Site :