Key-value storage systems are popular due to their simplicity, performance, and scalability. Such systems are gradually evolving and some recent ones now support transactions, which are a convenient means to coordinate access to data shared among many clients. Examples of such systems include Hyperdex Warp, CockroachDB, Walter, and TAPIR. Because these systems are so new, there are not many ways to assess how well they implement transactions. To address this shortcoming, we propose Transtorm, a benchmark suite for such transactional key-value storage systems (TKVS).
TKVS’s are today generally evaluated using ad-hoc workloads, making it hard to understand and compare the performance characteristics of different systems. To our knowledge, the only existing benchmark designed for transactional key-value storage systems is [YCSB+T]. YCSB+T extends the popular non-transactional YCSB benchmark by bundling a number of the benchmark’s operations into transactions. This extension is simple and intended to preserve the workload characteristics of the original YCSB benchmark, but as a result it does not directly gauge how well a TKVS addresses the various challenges of implementing transactions. At the same time, while there are many transaction processing benchmarks for traditional databases (e.g., TPC-C), they do not target the scale-out workloads for which TKVS’s are designed. Our goal is to design a benchmark to specifically exercise and challenge the TKVS’s transactional mechanisms.