paradnn

ParaDnn is a tool that generates parameterized deep neural network models. It provides large “end-to-end” models covering current and future applications, and parameterizing the models to explore a much larger design space of DNN model attributes. If you use paradnn for your project, please cite our paper:

Wang, Yu Emma, and Gu-Yeon Wei and David Brooks. 
"A Systematic Methodology for Analysis of Deep Learning Hardware and Software Platforms." 
The 3rd Conference on Machine Learning and Systems (MLSys) (2020).

@inproceedings{wang2020systematic,
  title={A Systematic Methodology for Analysis of Deep Learning Hardware and Software Platforms},
  author={Wang, Yu Emma and Wei, Gu-Yeon and Brooks, David},
  booktitle={The 3rd Conference on Machine Learning and Systems (MLSys)},
  year={2020}
}

See this link for the PDF.

This repository also includes the analysis tools demonstrated in the paper.

Canonical Models

ParaDnn generates three types of multi-layer models: fully-connected, convolutional, and recurrent models, summarized as below.

Requirements

python >= 3.7.6
1.13 >= TensorFlow >= 1.6

Environment

Please refer to TensorFlow installation guide for setting up virtual env. After that, please do

pip3 install --user --upgrade tensorflow==1.13.1
pip3 install --upgrade google-api-python-client
pip3 install --upgrade oauth2client
pip3 install notebook
pip3 install seaborn
pip3 install matplotlib
pip3 install sklearn

Test

python test.py --use_tpu=$USE_TPU

Run on CPUs

For example, to run FC models on CPUs, first modify the hyperparameter ranges in paradnn/run/fc_cpu.sh, and do

cd paradnn/
bash run/fc_cpu.sh

Run on TPUs

To run FC models on TPUs, first modify the hyperparameter ranges in file paradnn/run/fc_tpu.sh, and do

cd paradnn/
bash run/fc_tpu.sh

To collect TPU traces, first modify the hyperparameter ranges and gcp_bucket in file paradnn/run/fc_tpu_trace.sh, and do

cd paradnn/
bash run/fc_tpu_trace.sh

To download the data from the traces

cd ../scripts
bash download_jsonfiles_parallel.py $trace_folder_name $gcp_bucket 

To parse the downloaded data

bash parse_jsonfiles.py $trace_folder_name 

Collect performance data from execution logs

cd ../scripts
python get_perf.py

Run the analysis tools

cd scripts/plotting
jupyter notebook

Yu (Emma) Wang
9/13/2019