Backend Engineering (Intermediate)

Currently Playing: Apache Kafka Crash Course

Apache Kafka is a distributed stream processing software developed by LinkedIn and written in Scala and Java. Chapters 0:00 Intro 2:32 Kafka Broker 5:39 Kafka Producer 7:11 Kafka Consumer 8:34 Kafka Partitions 12:50 Queue vs Pub-Sub 16:47 Consumer Group 23:06 ZooKeeper 29:45 Coding Example 1:02:48 Kafka Pros-Cons 1:17:00 Summary Download slides here https://payhip.com/b/Y0Cq (Members get all slides for all my video courses for free) In this video I want to focus on the basics of Kafka, talk about how it works give finally spin up a Kafka cluster and write a producer and a consumer. Kafka Components _ producer _consumer _topic _partition How Kafka works Example ( Ride sharing ) Spin Kafka cluster Write a producer Write a consumer Kafka Applications Pros and Cons Source Code https://github.com/hnasr/javascript_playground/tree/master/kafka Docker commands spin up kafka //if you ever mess up or you want to start fresh just stop and remove the containers and rerun the command docker stop zookeeper kafka docker rm zookeeper kafka docker run --name zookeeper -p 2181:2181 -d zookeeper docker run -p 9092:9092 --name kafka -e KAFKA_ZOOKEEPER_CONNECT=husseinmac:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://husseinmac:9092 -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 -d confluentinc/cp-kafka Get my fundamentals of database engineering course link redirects to udemy with coupon applied (https://database.husseinnasser.com) Resources KIP 500 (Remove Zookeeper) https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum Become a Member on YouTube https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join 🧑‍🏫 Courses I Teach https://husseinnasser.com/courses 🏭 Backend Engineering Videos in Order https://backend.husseinnasser.com 💾 Database Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2 🎙️Listen to the Backend Engineering Podcast https://husseinnasser.com/podcast Gears and tools used on the Channel (affiliates) 🖼️ Slides and Thumbnail Design Canva https://partner.canva.com/c/2766475/647168/10068 🎙️ Mic Gear Shure SM7B Cardioid Dynamic Microphone https://amzn.to/3o1NiBi Cloudlifter https://amzn.to/2RAeyLo XLR cables https://amzn.to/3tvMJRu Focusrite Audio Interface https://amzn.to/3f2vjGY 📷 Camera Gear Canon M50 Mark II https://amzn.to/3o2ed0c Micro HDMI to HDMI  https://amzn.to/3uwCxK3 Video capture card https://amzn.to/3f34pyD AC Wall for constant power https://amzn.to/3eueoxP Stay Awesome, Hussein