Real-time architectures with Kafka, Spark, and the open source streaming stack.
6 articles in this topic
Featuring the use of Cassandra for OLTP, MySQL for OLAP, Kafka and Spark Streaming to ingest and process transactional data streams in a streaming dashboard that visualizes the total volume of money flow in the stock market
High Level Introduction to Apache Kafka, an Event Streaming Platform
In the first part of the article, we will discuss the overview of an open-source streaming data pipline and its environment setup using Docker Compose.
In the second part of the article, we will walk through the design and implementation of OLTP and OLAP databases using Cassandra and MySQL respectively.
In the third part of this series, we will demonstrate how to ingest and process transactional data using Kafka and Spark Streaming.
In the fourth and final installment of this series, we will perform some analytical queries on the MySQL database and create a dashboard using Streamlit.
The same thinking runs all the way through this 12-chapter guide, written by Samuel Chan and ready to read in an afternoon.
A 5-hour workshop across two evenings on building a personal finance terminal from scratch: AI-powered Telegram wired to IDX and SGX market data through the Sectors API.
September 24, 2026Zoom Conferencing (Online)
This is one slice of the catalogue. Browse every topic on the notes index, or subscribe and we will send the next piece your way.