Skip to content
This repository has been archived by the owner on Jun 16, 2021. It is now read-only.

Home

Jump to bottom
Rob Jansen edited this page Jun 5, 2020 · 36 revisions

shadow-plugin-tor wiki

This wiki explains how to set up and use the Shadow plug-in that runs Tor. It can be used to run a private Tor anonymity network on a single machine using the Shadow discrete-event network simulator. For more information about Shadow, see https://shadow.github.io and https://github.com/shadow.

This plug-in is most useful in conjunction with the traffic generator plug-in included with Shadow to transfer data across private anonymity networks and measure performance characteristics.

Dependencies

After installing Shadow and its dependencies:

YUM (Fedora):

sudo yum install -y gcc automake autoconf zlib zlib-devel xz-libs xz-devel

APT (Ubuntu):

sudo apt-get -y install gcc automake autoconf zlib1g-dev liblzma5 liblzma-dev

Setup and Install

Get shadow-plugin-tor

git clone https://github.com/shadow/shadow-plugin-tor.git

Quick auto setup

We need to setup openssl and libevent with custom configuration options in order to run Tor in Shadow. The setup script should handle this for you (run ./setup --help for more details).

cd shadow-plugin-tor
./setup dependencies
./setup build
./setup install

Important build options (see ./setup build --help for full list):

  • --tor-prefix to build with your local custom Tor distribution (instead of downloading one from torproject.org).
  • --openssl-prefix and --libevent-prefix to specify custom library locations

Detailed manual setup

The manual setup process is not necessary if the auto setup completed successfully.

WARNING: The following may not work and needs to be updated or removed.

Build openssl and libevent

Download the libraries and run:

cd openssl
./config --prefix=/home/${USER}/.shadow shared threads enable-ec_nistp_64_gcc_128 -fPIC
make
make install_sw
cd ../libevent
./configure --prefix=/home/${USER}/.shadow --enable-shared CFLAGS="-fPIC -I/home/${USER}/.shadow" LDFLAGS="-L/home/${USER}/.shadow"
make
make install

NOTE: if you want to use valgrind and don't want it to report spurious openssl errors, add this to the end of the openssl configure line: -g -pg -DPURIFY -Bsymbolic

Get and configure tor

We need to get the tor source so we can compile it into our Shadow plug-in, and then configure it to obtain a proper orconfig.h file.

cd shadow-plugin-tor
mkdir build
cd build
git clone --branch release-0.3.0 https://git.torproject.org/tor.git
cd tor
./autogen.sh
./configure --disable-asciidoc CFLAGS="-fPIC -fno-inline -fno-strict-aliasing -U_FORTIFY_SOURCE -O" --with-libevent-dir=`readlink -f ~`/.shadow --with-openssl-dir=`readlink -f ~`/.shadow
make
cd ..

Note that --with-openssl-dir and --with-libevent-dir should specify the install path of the custom built OpenSSL and Libevent.

build plugin using cmake
mkdir main
cd main
cmake ../.. -DTOR_VERSION_A=0 -DTOR_VERSION_B=2 -DTOR_VERSION_C=9 -DTOR_VERSION_D=11
make
make install
cmake troubleshooting

First try rebuilding to ensure that the cmake cache is up to date

rm -rf build
mkdir build
cd build

using VERBOSE=1 for more verbose output

cmake ..
VERBOSE=1 make

Configuring Shadow to Run the Plug-in

shadow-plugin-tor allows us to configure a private Tor network on our machine and transfer data through it. The plug-in requires a library to be preloaded with LD_PRELOAD and also requires setting some extra environment variables. These variables will be set in the shadow.config.xml configuration file and Shadow will handle the rest.

Example experiment configurations can be found in the shadow-plugin-tor/resource directory. Please untar the shadowtor-minimal-config.tar.xz example to understand how to configure your nodes to run Tor.

cd resource
tar xaf shadowtor-minimal-config.tar.xz
mv shadowtor-minimal-config shadowtor-minimal

Your shadow.config.xml file must specify that the Tor plug-in and the Tor preload libraries should be loaded, and that it should run an instance of the Tor plugin as a virtual process, e.g.:

  <plugin id="tor" path="~/.shadow/lib/libshadow-plugin-tor.so" />
  <plugin id="tor-preload" path="~/.shadow/lib/libshadow-preload-tor.so" />
  <host id="relay">
    <process plugin="tor" preload="tor-preload" arguments="[...]"  [...] />
  </host>

The arguments attribute of the process XML element specifies application arguments for configuring a host's instance of the plug-in. The arguments will be passed directly to Tor, and so all options in the same format as specified in the Tor config manual are allowed.

Scalability

For large Tor networks, you will run up against open file limits and other system limits. If you see resource-related errors, make sure you have updated your limits as described in this Shadow wiki page.

Running Tor in Shadow

The minimal example can be run as follows:

# the minimal configuration was already extracted above
cd shadowtor-minimal
# run the simulation
shadow shadow.config.xml > shadow.log

If this completes successfully (after about a minute or so), we should find that 40 streams have completed successfully:

for d in shadow.data/hosts/*client*; do grep "stream-success" ${d}/* ; done | tee clients.log | wc -l

The raw transfer message lines also contain more useful transfer statistics.

Notes:

  • Each Tor node may be configured using the *torrc files and in the arguments attribute in the shadow.config.xml file
  • The shadow.data/hosts/ directory contains a private data directory for each host running in the experiment
  • All Tor log output is redirected to a log file in the data directory for the host that generated it
  • All Shadow log output for every node is redirected to the standard output stream

The above minimal example is mostly useful for debugging. A slightly larger toy example is also available in the resource/ directory.

Tor Experimentation and Analysis

Suppose we want to test the performance difference between 2 of Tor's schedulers. We could do the following to setup our experiments:

cd resource
tar xaf shadowtor-toy-config.tar.xz
cp -R shadowtor-toy-config vanilla
cp -R shadowtor-toy-config priority

At this point you should add the string CircuitPriorityHalflife 30 to the end of the *.torrc file located in the priority directory. This will enable on all nodes the scheduler that prioritizes circuits based on exponentially-weighted moving average circuit throughputs.

Now you can run both experiments and plot the results: (NOTE: each experiment may take a few hours to run, so be patient)

cd vanilla
shadow -w 3 shadow.config.xml > shadow.log
cd ../priority
shadow -w 3 shadow.config.xml > shadow.log
cd ../

The following commands can be used to parse and plot those results:

cd vanilla
python ~/shadow/src/tools/parse-shadow.py --prefix vanilla-results shadow.log
python ~/shadow-plugin-tor/tools/parse-torctl.py --prefix vanilla-results shadow.data/hosts
cd ../priority
python ~/shadow/src/tools/parse-shadow.py --prefix priority-results shadow.log
python ~/shadow-plugin-tor/tools/parse-torctl.py --prefix priority-results shadow.data/hosts
cd ..
python ~/shadow/src/tools/plot-shadow.py --prefix "scheduler" --data vanilla-results/ "vanilla" --data priority-results/ "priority"

Visualize some statistics by opening the scheduler-shadow.pdf file. If you want to check the performance results from the traffic generator tool tgen, follow the documentation here.

Generating a new Tor Network

If you don't want to use the pre-generated examples in shadow-plugin-tor/resource or you want to customize the network, you can generate your own.

UPDATE: You probably want to use the new tornetgen tool here instead of using the process in this section.

The following process assumes you have an Internet connection and that the Shadow base directory exists in your home directory at ~/.

Prepare Alexa website data

The following produces alexa-top-1000-ips.csv: a csv list of the top 1000 servers.

wget http://s3.amazonaws.com/alexa-static/top-1m.csv.zip
unzip top-1m.csv.zip
python ~/shadow-plugin-tor/tools/parsealexa.py

Prepare Tor metrics data

This process requires lots of data and metrics from Tor. Be prepared to wait for the following downloads to complete.

NOTE: you probably should modify the URLs below to grab the most up-to-date metrics data.

wget https://collector.torproject.org/archive/relay-descriptors/server-descriptors/server-descriptors-2017-01.tar.xz
tar xaf server-descriptors-2017-01.tar.xz
wget https://collector.torproject.org/archive/relay-descriptors/extra-infos/extra-infos-2017-01.tar.xz
tar xaf extra-infos-2017-01.tar.xz
wget https://collector.torproject.org/archive/relay-descriptors/consensuses/consensuses-2017-01.tar.xz
tar xaf consensuses-2017-01.tar.xz
wget https://metrics.torproject.org/userstats-relay-country.csv

Start generating!

NOTES:

  • You'll need to use one of the consensus files from the consensuses-2017-01 directory in the following steps.
  • Make sure you have already built shadow-plugin-tor (using './setup build') because this process requires the tor and tor-gencert binaries to exist.
export PATH=${PATH}:~/shadow-plugin-tor/build/tor/src/core/or:~/shadow-plugin-tor/build/tor/src/app:~/shadow-plugin-tor/build/tor/src/tools
mkdir mytor
cd mytor
python ~/shadow-plugin-tor/tools/generate.py --help
python ~/shadow-plugin-tor/tools/generate.py --nauths 1 --nrelays 20 --nclients 200 --nservers 20 --fweb 0.90 --fbulk 0.10 --nperf50k 10 --nperf1m 10 --nperf5m 10 ../alexa-top-1000-ips.csv ../2017-01-31-23-00-00-consensus ../server-descriptors-2017-01/ ../extra-infos-2017-01/ ../userstats-relay-country.csv

If everything went smoothly, shadow can be run from inside the 'mytor' directory as usual.

Compare your Shadow-Tor network with Live Tor network performance

Tor uses onionperf to download files through the Tor network, and uses the time to download these files over time as a performance benchmark. Our Shadow-Tor network similarly runs perf50k, perf1m, and perf5m clients that mirror the Tor network benchmarks.

After we run a Shadow-Tor experiment, we can compare the performance of the perf50k, perf1m, and perf5m clients in Shadow with the actual Tor performance from the Tor metrics portal.

Suppose you ran the same 'vanilla' and 'priority' experiments from above in your generated network, and used parse-shadow.py and parse-tgen.py to get the results as explained above. We can then use the following script to get the live Tor network results:

mkdir torperf-results
cd torperf-results
# first update the parameters at the top of the ~/shadow-plugin-tor/tools/parse-torperf.py file
python ~/shadow-plugin-tor/tools/parse-torperf.py

This script will parse the TorPerf archive data and produce json files in the same format as those produced by Shadow. We can then easily plot and compare Tor performance with Shadow performance with the existing plot-shadow.py script distributed with Shadow in the shadow/tools directory.

# suppose your results are in the vanilla, priority, and torperf directories
python ~/shadow/src/tools/plot-shadow.py --prefix "scheduler" --data vanilla-results/ "vanilla" --data priority-results/ "priority" --data torperf-results/ "torperf"
Clone this wiki locally