TTDS6760: Introduction to Kafka for Developers

Become an EPIC Affiliate

To view the class schedule you need to become an Affiliate

  • Largest “Guaranteed To Run” public technical training schedules available
  • Easy to become an Affiliate – no charge or fee
Become an EPIC Affiliate

already an Affiliate?  Login

About this Course

Apache Kafka is a real-time data pipeline processor. It high-scalability, fault tolerance, execution speed, and fluid integrations are some of the key hallmarks that make it an integral part of many Enterprise Data architectures. In this lab intensive two-day course, students will learn how to use Kafka to build streaming solutions.

Audience Profile

This in an intermediate-level course is geared for experienced Java developers seeking to be proficient in Apache Kafka.

At Course Completion

Throughout the course students will be led through a series of progressively advanced topics, where each topic consists of lecture, group discussion, comprehensive hands-on lab exercises, and lab review.

Working in a hands-on learning environment, students will explore

· Overview of Streaming technologies

· Kafka concepts and architecture

· Programming using Kafka API

· Kafka Streams

· Monitoring Kafka

· Tuning / Troubleshooting Kafka

Outline

1. Introduction to Streaming Systems

· Fast data

· Streaming architecture

· Lambda architecture

· Message queues

· Streaming processors

2. Introduction to Kafka

· Architecture

· Comparing Kafka with other queue systems (JMS / MQ)

· Kaka concepts : Messages, Topics, Partitions, Brokers, Producers, commit logs

· Kafka & Zookeeper

· Producing messages

· Consuming messages (Consumers, Consumer Groups)

· Message retention

· Scaling Kafka

· Labs : Getting Kafka up and running; Using Kafka utilities

3. Programming With Kafka

· Configuration parameters

· Producer API (Sending messages to Kafka)

· Consumer API (consuming messages from Kafka)

· Commits , Offsets, Seeking

· Schema with Avro

· Lab : Writing Kafka clients in Java; Benchmarking Producer APIs

4. Kafka Streams

· Streams overview and architecture

· Streams use cases and comparison with other platforms

· Learning Kafka Streaming concepts (KStream, KTable, KStore)

· KStreaming operations (transformations, filters, joins, aggregations)

· Labs: Kafka Streaming labs

5. Administering Kafka

· Hardware / Software requirements

· Deploying Kafka

· Configuration of brokers / topics / partitions / producers / consumers

· Security: How secure Kafka cluster, and secure client communications (SASL, Kerberos)

· Monitoring : monitoring tools

· Capacity Planning : estimating usage and demand

· Trouble shooting : failure scenarios and recovery

6. Monitoring and Instrumenting Kafka

· Monitoring Kafka

· Instrumenting with Metrics library

· Labs; Monitor Kafka cluster

· Instrument Kafka applications and monitor their performance

Case Study / Workshop (Time-Permitting)

Prerequisites

Attendees should be experienced developers who are comfortable with Java, and have reasonable experience working with databases. Students should also be able to navigate Linux command line, and who have basic knowledge of Linux editors (such as VI / nano) for editing code.

Students should have attended the course(s) below, or should have basic skills in these areas:

· TT2104 Core Java Programming Fundamentals