Subscribe for new writing on data engineering and enteprise AI. Never more than once a month.
We have spent years building analytics platforms and AI systems for banks, mining groups, and public institutions across the region. These notes are where our engineers write down what actually worked. You will find deep dives on data pipelines, agentic systems, financial data, and the unglamorous infrastructure that holds it all together. Every article comes out of real client work rather than theory, which is exactly why we think they are worth your time.
A detailed documentation on building a modern data warehouse in BigQuery featuring storage and performance optimization.
Some ideas are too big for a single post. These are our long-form builds, written to be followed from the first part to the last.
End-to-end machine learning project on sentiment analysis. In this post, we will walk through the data collection process with distant supervision method.
End-to-end deep learning with LSTM on tweets from X.com. In this post, we will walk through the steps of data preprocessing and model building.
End-to-end machine learning project on sentiment analysis. In this post, we will walk through the steps of creating a dashboard and deploying our model as a web app with Streamlit.
In this final installment, we will explore some use cases of Twitter sentiment analysis in the field of business and social science.
In the first part of the article, we will discuss the overview of an open-source streaming data pipline and its environment setup using Docker Compose.
In the second part of the article, we will walk through the design and implementation of OLTP and OLAP databases using Cassandra and MySQL respectively.
In the third part of this series, we will demonstrate how to ingest and process transactional data using Kafka and Spark Streaming.
In the fourth and final installment of this series, we will perform some analytical queries on the MySQL database and create a dashboard using Streamlit.
In the first part of the article, we will explore the methodology behind designing a Fear and Greed Index, from conceptualization and mathematical techniques to data gathering, as we quantify market sentiment in real time.
In the second part of this article, we will walk through the calculations for each individual index and discuss the modeling approach for deriving the final Fear and Greed Index for the Indonesian stock market.
In the third part of this article, we will walk through the data handling process - from sourcing into production-ready data.
A whirlwind tour of query optimization strategies ft. query plans, index scans, and BigQuery-specific optimizations (part 1 of 2)
A discussion on full text search and indexing strategies in Postgres and BigQuery (part 2 of 2)
Combining the flexibility of JSON with the robustness of a relational database in Postgres.
Everything to get you started with creating programs using Financial APIs in Python.
The `json_normalize` function and the `explode` method in Pandas can be used to transform deeply nested JSON data from APIs into a Pandas DataFrame.
Building an AI app to analyze YouTube comments using Large Language Model (LLM) and Django. In this part, we will focus on data collection and data processing to retrieve YouTube video statistics and comments.
In part 2, we will develop a website that integrates sentiment analysis techniques and a Large Language Model to provide a comprehensive understanding of YouTube comments, enabling users to extract meaningful information effortlessly.
Our engineers have packed years of technical expertise into publications you can own and workshops you can join from anywhere.
The Enterprise AI Agentic Stack is the book product leaders reach for when they want the whole picture, from logs and vector stores to the patterns that hold a real system together.
A 4-hour, in-person code-along classroom in Singapore on building robust, enterprise-grade Agentic AI applications with established patterns and practical implementation blueprints.
November 4, 2026Waterloo Centre
The full catalogue, grouped the way we think about the work. New writing lands in these sections automatically.
Building agents that reason, call tools, and get real work done in production.
An introduction to Agentic AI, its capabilities, and its applications in various fields.
End-to-end large language model projects, from data collection to a served product.
Building an AI app to analyze YouTube comments using Large Language Model (LLM) and Django. In this part, we will focus on data collection and data processing to retrieve YouTube video statistics and comments.
In part 2, we will develop a website that integrates sentiment analysis techniques and a Large Language Model to provide a comprehensive understanding of YouTube comments, enabling users to extract meaningful information effortlessly.
Getting databases and warehouses to be fast, queryable, and ready for analytics.
A general introduction to NoSQL databases and their use cases.
A detailed documentation on building a modern data warehouse in BigQuery featuring storage and performance optimization.
A whirlwind tour of query optimization strategies ft. query plans, index scans, and BigQuery-specific optimizations (part 1 of 2)
Pipelines, orchestration, and the plumbing that moves data reliably at scale.
Build a simple API using Django Rest Framework, and use Redis for caching.
A practical demonstration on building a streaming analytics pipeline for warehouse inventory management on Google Cloud Platform.
Django REST Framework + testing your custom permissions in an API application with Talend API Tester or Postman
Real-time architectures with Kafka, Spark, and the open source streaming stack.
Featuring the use of Cassandra for OLTP, MySQL for OLAP, Kafka and Spark Streaming to ingest and process transactional data streams in a streaming dashboard that visualizes the total volume of money flow in the stock market
High Level Introduction to Apache Kafka, an Event Streaming Platform
In the first part of the article, we will discuss the overview of an open-source streaming data pipline and its environment setup using Docker Compose.
Taking trained models off the laptop and into resilient serving infrastructure.
Exploring Vertex AI Workbench for ML deployment (detailed tutorial on Vertex AI on GCP).
How to use TorchServe to serve your PyTorch model (detailed TorchServe tutorial)
Field reports on applying machine learning to messy, high-stakes real-world problems.
Use case of a classifier model for detecting downy mildew disease in Chinese cabbage field.
Explore how predictive maintenance powered by AI is transforming industries like mining and manufacturing by reducing costs and minimizing downtime.
A high level guide on how AI is used in rail transportation industry meant for executives and decision makers.
Working with market data, financial APIs, and the tooling that tames them in Python.
Everything to get you started with creating programs using Financial APIs in Python.
The `json_normalize` function and the `explode` method in Pandas can be used to transform deeply nested JSON data from APIs into a Pandas DataFrame.
Constructing a Fear and Greed Index for the Indonesian market, from theory to data.
In the first part of the article, we will explore the methodology behind designing a Fear and Greed Index, from conceptualization and mathematical techniques to data gathering, as we quantify market sentiment in real time.
In the second part of this article, we will walk through the calculations for each individual index and discuss the modeling approach for deriving the final Fear and Greed Index for the Indonesian stock market.
In the third part of this article, we will walk through the data handling process - from sourcing into production-ready data.
A deep learning journey through social sentiment, from raw tweets to deployed insight.
End-to-end machine learning project on sentiment analysis. In this post, we will walk through the data collection process with distant supervision method.
End-to-end deep learning with LSTM on tweets from X.com. In this post, we will walk through the steps of data preprocessing and model building.
End-to-end machine learning project on sentiment analysis. In this post, we will walk through the steps of creating a dashboard and deploying our model as a web app with Streamlit.
Practical container workflows for shipping data and AI services with confidence.
Upgrading to Docker Compose V2 from legacy v1 without Docker Desktop
A Python developer's guide to get started with Docker, Docker-Compose and Docker Desktop
How we work, what we believe, and the craft behind the products we build.
We have re-engineered Supertype from scratch, with a new architecture, high-performance Search, and a much more thoughtful user experience.
Technical writing is the most overlooked skill by software engineers and analytics professionals. This is a set of pedagogical strategies and practical tips to improve your technical writing.
The writing is one side of what we do. Here is how the rest connects.
Looking for our hands-on Generative AI recipes? They live over on the Sectors documentation.