Subscribe for new writing on data engineering and enteprise AI. Never more than once a month.
We have spent years building analytics platforms and AI systems for banks, mining groups, and public institutions across the region. These notes are where our engineers write down what actually worked. You will find deep dives on data pipelines, agentic systems, financial data, and the unglamorous infrastructure that holds it all together. Every article comes out of real client work rather than theory, which is exactly why we think they are worth your time.
A detailed documentation on building a modern data warehouse in BigQuery featuring storage and performance optimization.
Some ideas are too big for a single post. These are our long-form builds, written to be followed from the first part to the last.
End-to-end machine learning project on sentiment analysis. In this post, we will walk through the data collection process with distant supervision method.
End-to-end deep learning with LSTM on tweets from X.com. In this post, we will walk through the steps of data preprocessing and model building.
End-to-end machine learning project on sentiment analysis. In this post, we will walk through the steps of creating a dashboard and deploying our model as a web app with Streamlit.
In this final installment, we will explore some use cases of Twitter sentiment analysis in the field of business and social science.
In the first part of the article, we will discuss the overview of an open-source streaming data pipline and its environment setup using Docker Compose.
In the second part of the article, we will walk through the design and implementation of OLTP and OLAP databases using Cassandra and MySQL respectively.
In the third part of this series, we will demonstrate how to ingest and process transactional data using Kafka and Spark Streaming.
In the fourth and final installment of this series, we will perform some analytical queries on the MySQL database and create a dashboard using Streamlit.
Read query plans in Postgres and BigQuery, then make queries faster with indexes, partitioning, clustering and fewer scanned columns.
A discussion on full text search and indexing strategies in Postgres and BigQuery (part 2 of 2)
Filter rows on values inside a JSONB column with @>, ->> and ILIKE, and unpack nested arrays with jsonb_array_elements. Examples use real IDX company data.
CNN's Fear and Greed Index combines seven market indicators into a score from 0 to 100. We cover each one, then rebuild the index for Indonesian stocks.
We score eight sentiment indicators for the Indonesian stock market, then weight them with PCA, ICA and factor analysis to build one Fear and Greed Index.
In the third part of this article, we will walk through the data handling process - from sourcing into production-ready data.
Everything to get you started with creating programs using Financial APIs in Python.
Flatten deeply nested JSON from a financial API into a Pandas DataFrame using pd.json_normalize and the explode method.
Building an AI app to analyze YouTube comments using Large Language Model (LLM) and Django. In this part, we will focus on data collection and data processing to retrieve YouTube video statistics and comments.
In part 2, we will develop a website that integrates sentiment analysis techniques and a Large Language Model to provide a comprehensive understanding of YouTube comments, enabling users to extract meaningful information effortlessly.
Our engineers have packed years of technical expertise into publications you can own and workshops you can join from anywhere.
The Enterprise AI Agentic Stack is the book product leaders reach for when they want the whole picture, from logs and vector stores to the patterns that hold a real system together.
A hands-on workshop for building a personal market-monitoring system with Claude: screen IDX and SGX stocks, set up alerts in plain English, and build a live dashboard with an analyst built in. Beginner-friendly.
October 28, 2026Zoom Conferencing (Online)
The full catalogue, grouped the way we think about the work. New writing lands in these sections automatically.
Building agents that reason, call tools, and get real work done in production.
An introduction to Agentic AI, its capabilities, and its applications in various fields.
End-to-end large language model projects, from data collection to a served product.
Building an AI app to analyze YouTube comments using Large Language Model (LLM) and Django. In this part, we will focus on data collection and data processing to retrieve YouTube video statistics and comments.
In part 2, we will develop a website that integrates sentiment analysis techniques and a Large Language Model to provide a comprehensive understanding of YouTube comments, enabling users to extract meaningful information effortlessly.
Getting databases and warehouses to be fast, queryable, and ready for analytics.
Read query plans in Postgres and BigQuery, then make queries faster with indexes, partitioning, clustering and fewer scanned columns.
A general introduction to NoSQL databases and their use cases.
A detailed documentation on building a modern data warehouse in BigQuery featuring storage and performance optimization.
Pipelines, orchestration, and the plumbing that moves data reliably at scale.
Build a simple API using Django Rest Framework, and use Redis for caching.
A practical demonstration on building a streaming analytics pipeline for warehouse inventory management on Google Cloud Platform.
Django REST Framework + testing your custom permissions in an API application with Talend API Tester or Postman
Real-time architectures with Kafka, Spark, and the open source streaming stack.
Featuring the use of Cassandra for OLTP, MySQL for OLAP, Kafka and Spark Streaming to ingest and process transactional data streams in a streaming dashboard that visualizes the total volume of money flow in the stock market
High Level Introduction to Apache Kafka, an Event Streaming Platform
In the first part of the article, we will discuss the overview of an open-source streaming data pipline and its environment setup using Docker Compose.
Taking trained models off the laptop and into resilient serving infrastructure.
Exploring Vertex AI Workbench for ML deployment (detailed tutorial on Vertex AI on GCP).
Package a PyTorch model with torch-model-archiver, write a custom handler, serve it through TorchServe's inference and management APIs, then run it in Docker.
Field reports on applying machine learning to messy, high-stakes real-world problems.
Use case of a classifier model for detecting downy mildew disease in Chinese cabbage field.
Explore how predictive maintenance powered by AI is transforming industries like mining and manufacturing by reducing costs and minimizing downtime.
A high level guide on how AI is used in rail transportation industry meant for executives and decision makers.
Working with market data, financial APIs, and the tooling that tames them in Python.
Everything to get you started with creating programs using Financial APIs in Python.
Flatten deeply nested JSON from a financial API into a Pandas DataFrame using pd.json_normalize and the explode method.
Constructing a Fear and Greed Index for the Indonesian market, from theory to data.
CNN's Fear and Greed Index combines seven market indicators into a score from 0 to 100. We cover each one, then rebuild the index for Indonesian stocks.
We score eight sentiment indicators for the Indonesian stock market, then weight them with PCA, ICA and factor analysis to build one Fear and Greed Index.
In the third part of this article, we will walk through the data handling process - from sourcing into production-ready data.
A deep learning journey through social sentiment, from raw tweets to deployed insight.
End-to-end machine learning project on sentiment analysis. In this post, we will walk through the data collection process with distant supervision method.
End-to-end deep learning with LSTM on tweets from X.com. In this post, we will walk through the steps of data preprocessing and model building.
End-to-end machine learning project on sentiment analysis. In this post, we will walk through the steps of creating a dashboard and deploying our model as a web app with Streamlit.
Practical container workflows for shipping data and AI services with confidence.
Install the Compose v2 plugin from Docker's apt repository or the release binaries, then switch your scripts from docker-compose to docker compose.
A Python developer's guide to get started with Docker, Docker-Compose and Docker Desktop
How we work, what we believe, and the craft behind the products we build.
We have re-engineered Supertype from scratch, with a new architecture, high-performance Search, and a much more thoughtful user experience.
Technical writing is the most overlooked skill by software engineers and analytics professionals. This is a set of pedagogical strategies and practical tips to improve your technical writing.
The writing is one side of what we do. Here is how the rest connects.
Looking for our hands-on Generative AI recipes? They live over on the Sectors documentation.