Votes

This wiki lists the steps required to run our votes experiment from the paper (figure 10).

Votes Harness

Our votes experiment harness focuses on a common operation from Lobsters: voting on stories/articles.

Our harness uses a simplified schema and queries extracted adapted from in-deployment Lobsters. It begins by creating a configurable number of articles during the priming phase, along with some amount of votes. During the load stage: it performs two types of queries: (1) adding a vote to some article (write), and (2) querying the total number of votes that an article has (read). The distribution of votes with respect to article is zipf in accordance with the observed distribution of Lobsters. Reads and Writes are batched to amortize network overheads. The harness generates a load with 95% reads and 5% writes, as is common in web applications.

The harness has three modes: it can be run against MariaDB, K9db, or MariaDB+memcached. The first is a standard baseline where writes are executed as SQL INSERT statements, and the reads correspond to a query that aggregates votes over their article using a GROUP BY. The second runs similar queries, but K9db uses a view to automatically cache the query behind the scenes. This cache is always up-to-date and thus is always used to respond to every query.

The last setup adds a demand-filled cache on top of MariaDB: when the harness generates a read, it first checks the cache to see if the result is available, and queries MariaDB if it is not, updating the cache with the result. Whenever the harness generates a write to vote for a particular article, it invalidates the cache entry corresponding to that article. The cache is initially empty during priming. We run the harness for a warmup period before starting measurements so the cache is somewhat warm.

The Votes Experiment (Figure 10)

You can run the experiment using our provided script, which on our setup takes about 10 minutes:

cd <K9DB_DIR>
./experiments/scripts/votes.sh

The script runs the harness in all three modes, recording all the measurements in log files, and then plots them using our plotting script. The output plots will be produced at experiments/scripts/outputs/votes/, while the measurements logs are available at experiments/scripts/logs/votes/.

If running the experiment on a setup different from our Google cloud machines, you may need to adjust the target parameter inside the script, in case it is too aggressive for your setup.