Cassandra Secondary Indexes – An Overview

One of the primary goals of Apache Cassandra is the storage and retrieval of data as quickly as possible. One of the tools that databases provide is a mechanism called an index, which, like an index at the back of a book, provides quick access to the desired data. Cassandra provides an implementation of secondary […]

Configuration Miscellaneous

The Simplest Local Kubernetes Web Application Setup

So I’ve been taking some time to look into Kubernetes. One of the first things I wanted to try was to write my own simple application that uses Docker and Kubernetes locally. When I attempted to do that, it wasn’t obvious how, and furthermore, I noticed that it didn’t seem like there was a straightforward […]

Cassandra Scalability

Creating Multiple Local Cassandra Datacenters with Docker

The Apache Cassandra database has long included support for multiple datacenters. That is, Cassandra allows an organization to configure a cluster to actively store data across disparate datacenters. There are multiple reasons do this, including: Improve geographic responsiveness Disaster recovery Separation of transactional and analytic workloads In general, when trying to wrap my head around […]


Creating Custom Kafka Partitioners

If you’ve used Kafka, you’ve likely heard about partitions. Kafka allows you to partition the data in a given topic so that the processing work can be divided among multiple nodes. Thus partitioning of the data allows more data to be processed in parallel. Kafka’s default logic will attempt to evenly distribute messages into the […]


What is HTAP?

I’ve begun seeing the database-related acronym HTAP thrown about more and more, so I did a little research to understand its meaning and implications. The acronym HTAP has been established by Gartner (by Wikipedia) as follows: Hybrid transaction/analytical processing (HTAP) is an emerging application architecture that “breaks the wall” between transaction processing and analytics. It […]

Cassandra Scalability

An Example Using Cassandra With Zipkin

Zipkin is a system for tracing, viewing, and troubleshooting distributed systems and microservice-based applications. Here’s the description from the Zipkin website: Zipkin is a distributed tracing system. It helps gather timing data needed to troubleshoot latency problems in microservice architectures. It manages both the collection and lookup of this data. Zipkin’s design is based on the […]

Hadoop Scalability

How Big is Your Elephant?

In order to augment some research we’re doing, we’d like to determine the sizes of Hadoop clusters that organizations are commonly deploying. Why? Well, organizations with 100+ nodes have different needs than those with five, for example.

Configuration Hadoop Hive Scalability

Give Your MySQL Account Access to Hive

One area of the Apache Hive documentation that’s not entirely explicit is in regard to the database privileges needed for its metastore[1]. Developers often become accustomed to creating a database account that has all privileges granted. But in the Real World, end users of Hive must configure it to point to a metastore RDBMS account […]

Miscellaneous Scalability

SSDs in the Data Center — Is $/GB/IOPS the Only Relevant Metric?

Wikia’s Artur Bergman recently gave a talk at Velocity about SSD adoption that has generated a lot of buzz. The video can be viewed here. Warning: the video is rated PG-13 for language and adult situations. The focus of his talk was that the relevant metric for data center storage is $/GB/IOPS. He showed how […]


Pay Off Your Technical Debt

The first thing I do when I get my hands on a client’s code is to figure out the size of the code base. I execute something like this to determine the number of lines of code (LoC) using a fresh checkout from trunk: $ find . -name “*.java” -exec cat {} \; | wc […]