Apache Kafka

This talk is good intro. Franz is nice mac Kafka client.

Notes

• Same goes with all the Kafka usage, making development literally Kafkaesque. The data eventually flows through your microservice landscape, being eventually consistent and performance issues become a whole different nightmare. Yea, maybe I just didn't read enough of the documentation but maybe your 1500 users per month web-app doesn't need 7 microservices and Kafka as single source of truth, all hold together by 8 developers.

Links

• Confluent - Complete Event Streaming Platform For Apache Kafka.
• Kafka ecosystem ... but step by step
• Playground for Kafka/Confluent Docker experimentations
• Goka - Compact yet powerful distributed stream processing library for Apache Kafka written in Go.
• Streams and Tables in Apache Kafka: A Primer (2020)
• Kafka in Netflix Studios (2020)
• OPERATR - Observe, explore, and control streaming compute.
• Kaf - Kafka CLI inspired by kubectl & docker.
• What every software engineer should know about Apache Kafka (2020) (HN)
• jocko - Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native).
• Kafka Removing Zookeeper Dependency (2020) (HN)
• Kafka-Pixy (gRPC/REST Proxy for Kafka)
• Mirroring from Apache Kafka to Azure Event Hub (2020)
• CMAK - Tool for managing Apache Kafka clusters.
• Redpanda - Real-time engine for modern apps. 10x Faster Kafka. (Code) (HN) (First Impressions) (Redpanda 21.10.1) (HN) (Docs) (Docs Code)
• rochefort - Poor man's kafka (plus in-place mutations and search).
• Why Can’t I Connect to Kafka?
• Eliminating Task Processing Outages by Replacing RabbitMQ with Apache Kafka (2020) (HN)
• Runtime Discovery of Kafka Streams (2020)
• Strimzi - Apache Kafka running on Kubernetes. (Web)
• Trubka - CLI tool for Kafka written in Go.
• franz-go - High performance, pure Go library for interacting with Kafka.
• Kafka is not a Database (2020) (HN) (Tweet)
• Kafka As A Database? Yes Or No (2021)
• Kafka Kit - Tools for Scaling Kafka. (Article)
• Estimating when a message will be consumed in Kafka (2020)
• Kafka message scheduler - Allows you to send message to a target topic on a specific time with a particular payload.
• librdkafka - Apache Kafka C/C++ client library. (HN)
• A review of Kafka books and tutorials (2021)
• Is Pulsar better than Kafka? (2020)
• The Kafka API is great; now let's make it fast (2021)
• Kafka Rust Client
• Kafka timers (2021)
• Contrasting Kafka with Akka (2021)
• Kafka Lag Exporter - Monitor Kafka Consumer Group Latency with Kafka Lag Exporter.
• Awesome Open-Source Contribs for Apache Kafka
• kafka-serde - Rust's serde implementation for the Kafka protocol.
• Apache Kafka Mocking and Testing (2020)
• Kafka Without ZooKeeper: A Sneak Peek At the Simplest Kafka Yet (2021)
• Best Books to Learn Apache Kafka (2021)
• Kafka-Penguin - Easy-to-use, lightweight KafkaJS library for message processing.
• Understanding Kafka with Factorio (2019) (HN)
• Google Pub/Sub Lite for Kafka Users (2021)
• Teach Yourself Apache Kafka and Python with a Jupyter Notebook
• Confluent Hub - Discover Kafka connectors and more.
• Gently Down the Stream - A gentle introduction to Apache Kafka (HN)
• UI for Apache Kafka - Free open-source web UI for monitoring and management of Apache Kafka clusters. (HN)
• Starting with Kafka (2021) (HN)
• Conduktor - Kafka Desktop Client. (GitHub)
• Event-driven OpenFaaS with Managed Kafka from Aiven (2021)
• End-to-End Encryption through Kafka
• Building a Kafka based key-value store in Rust (2021)
• The basics you need to know about Kafka [Graphic Explanation]
• Winton Kafka Streams - Python implementation of Apache Kafka Streams.
• Kafka Rust Example - Simple command line application to read/write message to kafka topic using protobuf.
• Exactly once delivery from Kafka to Delta Lake with Rust
• Optimizing Kafka Performance (2021) (Reddit)
• What is Apache Kafka, and what are Kafka use cases? (2021)
• pg2kafka - Ship changes in Postgres to Kafka.
• Issues we've encountered while building a Kafka based data processing pipeline (2021) (HN)
• josef - Process Kafka streams with Clojure Transducers.
• Kafka Streams in Scala (2021)
• Kafka Streams 101 (2021)
• Kafka Streams 101 – resources to get started (2021)
• rust-rdkafka - Fully asynchronous, futures-enabled Apache Kafka client library for Rust based on librdkafka.
• Sarama - Go library for Apache Kafka.
• kcctl - CLI for Apache Kafka Connect.
• Pinterest Secor - Service implementing Kafka log persistence.
• Scaling Kafka at Honeycomb (2021) (HN)
• kafkactl - Command Line Tool for managing Apache Kafka.
• Kasper - Lightweight library for processing Kafka topics.
• Postgres, Kafka, and the Mysterious 100 GB
• Create page view analytics system using Kafka, Go, Postgres & GraphQL in 5 steps (2021)
• kaproxy - Restful HTTP Proxy For Kafka.
• Brod - Apache Kafka client library for Erlang/Elixir.
• Castle - Framework for building Kafka and avro based apps in typescript.
• Franz - Kafka client library for Erlang/Elixir based on librdkafka implemented as a safe Rust NIF.
• Disaster Recovery for Multi-Region Kafka at Uber (2020)
• Delivering billions of messages exactly once (2017)
• kPow for Apache Kafka - Monitor. Manage. Learn.
• Building Real-Time ETL Pipelines with Apache Kafka (2022)
• confluent-kafka-python - Confluent's Kafka Python Client.
• Kafdrop - Kafka Web UI.
• Introduction to Apache Kafka by James Ward (2017)
• GraphQL Kafka Subscriptions
• Kafka Consumer Autoscaling with KEDA (2022)
• Confluent's Go Client for Apache Kafka
• Apache Kafka Code
• RSKafka - Minimal Rust client for Apache Kafka.
• node-kafka-streams
• Kowl - Web UI for Apache Kafka.
• CloudHut - Use Apache Kafka more efficiently.
• KMinion - Prometheus Exporter for Apache Kafka.
• Kafka Python client
• upstash-kafka - HTTP based Kafka Client for Serverless and Edge Functions.
• KafkaJS - Modern Apache Kafka client for node.js.
• Kafka for Kubernetes - Kafka cluster as Kubernetes StatefulSet, plain manifests and config.
• Greyhound - Rich Kafka client library in Scala.
• kaskade - Kafka text user interface in style.
• kafkacrypto - End-to-End Message Layer Encryption for Kafka.
• Kafka Docker Playground - Fully automated Apache Kafka and Confluent Docker based examples.
• AWS Serverless Kafka (2022)
• kafkaproxy - Reverse proxy for the wire protocol of Apache Kafka.
• Karapace - Your Kafka essentials in one tool. Serving all your Apache Kafka® HTTP needs. (Web)
• Build a Data Streaming App for Sound & Vision (2022)
• Burrow - Kafka Consumer Lag Checking.
• kcat - Generic non-JVM producer and consumer for Apache Kafka.
• Redpanda Data - YouTube
• KafkaSSE - Kafka Consumer to HTTP SSE/EventSource.
• BroadwayKafka - Broadway connector for Kafka.
• CMAK - Tool for managing Apache Kafka clusters.
• kt - Kafka tool that likes JSON.
• Kafka Proxy - Proxy connections to Kafka cluster. Connect through SOCKS Proxy, HTTP Proxy or to cluster running in Kubernetes.
• System Design: Why is Kafka fast? (2022)
• Simplifying Redpanda Raft implementation (2022)
• StreamOps - Kafka and Red Panda Streaming Operations Platform.
• FranzView - Open source kafka monitoring and management tool built with Javascript developers in mind.
• Build a Reactive Data Streaming App with Python and Apache Kafka (2022)
• Docker images for Apache Kafka
• Postgres: a better message queue than Kafka? (2022) (Lobsters)
• Kafka Worker - Kafka broker implementation on top of Cloudflare Workers. (HN)
• Kafka Streams with Serverless Kafka (2022)
• Zilla - Event-driven API gateway that can extend Apache Kafka to the edge.
• Franz - macOS Client for Apache Kafka. (Article) (HN)
• ZIO Kafka - Kafka client for ZIO and ZIO Streams in Scala.
• KStream - Apache Kafka Streams for Go.
• Kafka in 100 Seconds (2023)
• Apache Kafka Beyond the Basics: Windowing (HN)
• Streaming Events from Redpanda Cloud into Imply Polaris (2023)
• NATS to Kafka Bridging
• Optimizing Apache Kafka® for High Throughput (2023)
• FastKafka - Powerful and easy-to-use Python library for building asynchronous web services that interact with Kafka topics. (HN)