Mustard Grain
One of the primary goals of Apache Cassandra is the storage and retrieval of data as quickly as possible. One of the tools that databases provide is a mechanism called an index, which, like an index at the back of a book, provides quick access to the desired data. Cassandra provides an implementation of secondary […]
The Apache Cassandra database has long included support for multiple datacenters. That is, Cassandra allows an organization to configure a cluster to actively store data across disparate datacenters. There are multiple reasons do this, including: Improve geographic responsiveness Disaster recovery Separation of transactional and analytic workloads In general, when trying to wrap my head around […]
I’ve begun seeing the database-related acronym HTAP thrown about more and more, so I did a little research to understand its meaning and implications. The acronym HTAP has been established by Gartner (by Wikipedia) as follows: Hybrid transaction/analytical processing (HTAP) is an emerging application architecture that “breaks the wall” between transaction processing and analytics. It […]
Zipkin is a system for tracing, viewing, and troubleshooting distributed systems and microservice-based applications. Here’s the description from the Zipkin website: Zipkin is a distributed tracing system. It helps gather timing data needed to troubleshoot latency problems in microservice architectures. It manages both the collection and lookup of this data. Zipkin’s design is based on the […]
In order to augment some research we’re doing, we’d like to determine the sizes of Hadoop clusters that organizations are commonly deploying. Why? Well, organizations with 100+ nodes have different needs than those with five, for example.
One area of the Apache Hive documentation that’s not entirely explicit is in regard to the database privileges needed for its metastore[1]. Developers often become accustomed to creating a database account that has all privileges granted. But in the Real World, end users of Hive must configure it to point to a metastore RDBMS account […]
Wikia’s Artur Bergman recently gave a talk at Velocity about SSD adoption that has generated a lot of buzz. The video can be viewed here. Warning: the video is rated PG-13 for language and adult situations. The focus of his talk was that the relevant metric for data center storage is $/GB/IOPS. He showed how […]
The first thing I do when I get my hands on a client’s code is to figure out the size of the code base. I execute something like this to determine the number of lines of code (LoC) using a fresh checkout from trunk: $ find . -name “*.java” -exec cat {} \; | wc […]
One feature that Hive gets for free by virtue of being layered atop Hadoop is the S3 file system implementation. The upshot being that all the raw, textual data you have stored in S3 is just a few hoops away from being queried using Hive’s SQL-esque language. Imagine you have an S3 bucket un-originally named […]
According to the Hive web site: Hive is a data warehouse infrastructure built on top of Hadoop that provides tools to enable easy data summarization, adhoc querying and analysis of large datasets data stored in Hadoop files. Hive is built on top of various technologies, the most notable being Hadoop and HDFS. As a result, […]