Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge #1

Merged
merged 86 commits into from
Oct 4, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
7aaeb5f
Modularization of inference
gheinrich Aug 31, 2016
64a1781
Move digits.__version__ into new file; add tests
lukeyeager Sep 12, 2016
18adbe3
Merge pull request #1058 from lukeyeager/version-file
lukeyeager Sep 12, 2016
6ae5a7d
Add tests for modularization of inference
gheinrich Sep 7, 2016
f577e3c
[ObjDet] Allow non-numeric filenames
lukeyeager Sep 13, 2016
db50481
[ObjDet] Improve docs about case sensitivity
lukeyeager Sep 13, 2016
21b9fd6
[ObjDet] Improve docs about mapping to zero
lukeyeager Sep 13, 2016
d03de36
Merge pull request #1060 from lukeyeager/objdetect-data-fixes
lukeyeager Sep 13, 2016
a258496
[ObjDet] Improve handling of label files
lukeyeager Sep 13, 2016
a687b06
Merge pull request #1061 from lukeyeager/objdet-empty-label-files
lukeyeager Sep 13, 2016
4447eee
Gunicorn fix - add abspath(curdir) to PATH
lukeyeager Sep 14, 2016
149ccdc
Merge pull request #1066 from lukeyeager/gunicorn-abspath
lukeyeager Sep 14, 2016
5edc6ee
[ObjDet] Remove debug print
lukeyeager Sep 15, 2016
380cfc9
Merge pull request #1067 from lukeyeager/objdet-debug-print
lukeyeager Sep 15, 2016
ff0f05d
Merge pull request #1024 from gheinrich/inference-modularization
lukeyeager Sep 15, 2016
ed37d21
Supress numpy VisibleDeprecationWarning
lukeyeager Sep 15, 2016
cb9bb36
cookie.tmp replaced by digits.cookie
jpfairbanks Sep 15, 2016
0fa55cd
Changed a few things so that IE would display the front page correctly
jmancewicz Sep 15, 2016
575d23a
Install a few more deb packages on TravisCI
lukeyeager Sep 15, 2016
fd32cde
Remove changes to socketio logger
lukeyeager Sep 15, 2016
a97f45d
Merge pull request #1072 from lukeyeager/travis-apt-packages
lukeyeager Sep 15, 2016
ac96d77
Merge pull request #1069 from lukeyeager/numpy-warning
lukeyeager Sep 15, 2016
69c149d
Merge pull request #1071 from jmancewicz/internet-explorer-fix
jmancewicz Sep 15, 2016
041c632
Merge pull request #1073 from lukeyeager/socketio-logger
lukeyeager Sep 15, 2016
af88898
Avoid NaN errors and misclassifications in tests
lukeyeager Sep 15, 2016
3b2afbd
Update the ML repo package name
lukeyeager Sep 15, 2016
6b9bff1
Merge pull request #1074 from lukeyeager/test-torch-lenet-fix
lukeyeager Sep 15, 2016
b9136e7
[Tests] More time for data aug model to converge
lukeyeager Sep 15, 2016
a6feac9
Merge pull request #1070 from jpfairbanks/patch-1
gheinrich Sep 16, 2016
8f1e1a6
rearanged group elements and pills on the front page
jmancewicz Sep 7, 2016
96e68ad
Merge pull request #1048 from jmancewicz/front-page-buttons
jmancewicz Sep 19, 2016
7c7a644
Fix path given to utils.filesystem.dir_hash()
lukeyeager Sep 19, 2016
2edd521
Merge pull request #1088 from lukeyeager/dir-hash-path
jmancewicz Sep 19, 2016
b1c2f1c
Use envvars for config instead of a file
lukeyeager Sep 9, 2016
800e352
_HOME -> _ROOT
lukeyeager Sep 20, 2016
e615f4e
Added configuration doc
lukeyeager Sep 20, 2016
722b707
gitignore improvements
lukeyeager Sep 21, 2016
a430587
Merge pull request #1076 from lukeyeager/torch-tests-dataaug
lukeyeager Sep 21, 2016
4a59828
Fix validate_required_if_set validator
gheinrich Sep 21, 2016
23361c7
Merge pull request #1105 from gheinrich/dev/fix-validator
lukeyeager Sep 21, 2016
04e4004
Merge pull request #1099 from lukeyeager/gitignore
lukeyeager Sep 21, 2016
3afc367
Merge pull request #1091 from lukeyeager/envvars
lukeyeager Sep 21, 2016
b34c431
Move gunicorn_config.py into digits/
lukeyeager Sep 20, 2016
2bd5095
Move download_data from tools/ to scripts/
lukeyeager Sep 20, 2016
396ba08
Move tools/ to digits/tools/
lukeyeager Sep 20, 2016
f7b10cc
Add setup.py script
lukeyeager Sep 21, 2016
575b2d5
Small changes to the "About" dropdown
lukeyeager Sep 22, 2016
b9b7499
Merge pull request #1110 from lukeyeager/updates-to-about
jmancewicz Sep 22, 2016
3b9cfde
Merge pull request #1093 from lukeyeager/setup
lukeyeager Sep 22, 2016
552e4bb
Add support for plug-ins
gheinrich Jul 21, 2016
f2b0b0a
Merge pull request #927 from gheinrich/dev/plugin-support
lukeyeager Sep 23, 2016
fa3eb9d
Add --print-data to analyze_db tool
lukeyeager Sep 15, 2016
a173f0f
Merge pull request #1068 from lukeyeager/analyze-db-print-data
lukeyeager Sep 23, 2016
604ce0e
Move the download_data scripts into digits/
lukeyeager Sep 26, 2016
eb4dd98
Move digits-devserver to digits/__main__.py
lukeyeager Sep 26, 2016
c72d3e6
Refactor to allow "--version" without loading jobs
lukeyeager Sep 26, 2016
d2b01fc
Refactor to expose digits.__main__:main
lukeyeager Sep 26, 2016
6765cd2
Move image gradient data extension to plug-in area
gheinrich Aug 2, 2016
21fd8cb
Move image gradient view extension to plug-in area
gheinrich Aug 2, 2016
8fc190e
Remove option to make an extension invisible
gheinrich Sep 26, 2016
f907bb5
Update Travis script to install gradients plug-ins
gheinrich Sep 23, 2016
aab35bd
Update regression tutorial for plug-in install
gheinrich Aug 2, 2016
ba2bbf6
Merge pull request #1120 from lukeyeager/move-download-data
lukeyeager Sep 26, 2016
88666fa
Merge pull request #947 from gheinrich/dev/gradients-plugin
lukeyeager Sep 26, 2016
c28e8d4
Merge pull request #1121 from lukeyeager/move-digits-devserver
lukeyeager Sep 26, 2016
360a16e
digits-devserver: pass through command-line args
lukeyeager Sep 26, 2016
63f7950
Fix some imports
lukeyeager Sep 26, 2016
593f11c
Merge pull request #1122 from lukeyeager/fix-digits-devserver
lukeyeager Sep 26, 2016
9a43e99
Bugfix in digits/__main__
lukeyeager Sep 27, 2016
54ef0d2
Fix bug in gunicorn_config
lukeyeager Sep 27, 2016
f6d146c
Merge pull request #1124 from lukeyeager/fix-main
lukeyeager Sep 27, 2016
df72597
Merge pull request #1125 from lukeyeager/fix-gunicorn-config
lukeyeager Sep 27, 2016
1ff2776
Stop deleting logfiles
lukeyeager Sep 27, 2016
a77706d
Merge pull request #1126 from lukeyeager/fix-logfile-config
lukeyeager Sep 27, 2016
eb044f4
Update docs for v4.1
lukeyeager Sep 26, 2016
d4bc3db
Reproducible deb package builds with docker
lukeyeager Sep 23, 2016
a81c07d
Remove nginx.site in favor of deb/extras/
lukeyeager Sep 23, 2016
918b545
Test deb build on Travis
lukeyeager Sep 27, 2016
ed25fac
Merge pull request #1123 from lukeyeager/4.1-docs
lukeyeager Sep 27, 2016
649035f
Merge pull request #1113 from lukeyeager/deb-packaging
lukeyeager Sep 27, 2016
dc3b66d
Remove deb build from TravisCI
lukeyeager Sep 28, 2016
c74ef69
Merge pull request #1128 from lukeyeager/travis-remove-deb
lukeyeager Sep 29, 2016
cebb66d
Remove gunicorn and digits-server
lukeyeager Sep 30, 2016
a6c74bd
Add packaging support for Systemd (for 16.04)
lukeyeager Sep 28, 2016
be2bc82
Merge pull request #1127 from lukeyeager/systemd
lukeyeager Sep 30, 2016
f8b74e8
Merge pull request #1075 from lukeyeager/repo-pkg-name-change
lukeyeager Oct 3, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 12 additions & 10 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
# Ignore these file types
*.cfg
*.log
*.pyc
# Temporary files
*.swp

# Ignore emacs backup files
*~
TAGS

# Ignore these files
.coverage
# Compiled / optimized files
*.py[cod]

# Tests
/.coverage

# Ignore these directories
# virtualenv
/venv/
/digits/jobs/

# setuptools
/build/
/dist/
/*.egg-info/
9 changes: 7 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,14 +48,19 @@ install:
- pip install -r requirements.txt
- pip install -r requirements_test.txt

# image gradients plug-ins
- pip install -e .
- pip install plugins/data/imageGradients
- pip install plugins/view/imageGradients

before_script:
- export CAFFE_ROOT=~/caffe
- export TORCH_ROOT=~/torch/install
- export TORCH_ROOT=~/torch
# Disable OpenMP multi-threading
- export OMP_NUM_THREADS=1

script:
./digits-test -v --with-coverage --cover-package=digits,tools,scripts
./digits-test -v --with-coverage --cover-package=digits

after_success:
coveralls
Expand Down
3 changes: 3 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
recursive-include digits/templates *
recursive-include digits/static *
recursive-include digits/standard-networks *
23 changes: 20 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,30 @@ DIGITS (the **D**eep Learning **G**PU **T**raining **S**ystem) is a webapp for t

# Installation

If you are using Ubuntu 14.04, you can install pre-built packages for the latest major release.
Visit [docs/UbuntuInstall.md](docs/UbuntuInstall.md) for instructions.
| Installation method | Supported platform[s] | Available versions | Instructions |
| --- | --- | --- | --- |
| Deb packages | Ubuntu 14.04 | [14.04 repo](http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1404/x86_64) | [docs/UbuntuInstall.md](docs/UbuntuInstall.md) |
| Docker | Linux | [DockerHub tags](https://hub.docker.com/r/nvidia/digits/tags/) | [nvidia-docker wiki](https://github.com/NVIDIA/nvidia-docker/wiki/DIGITS) |
| Source | Ubuntu 14.04, 16.04 | [GitHub tags](https://github.com/NVIDIA/DIGITS/releases) | [docs/BuildDigits.md](docs/BuildDigits.md) |

To build DIGITS and its requirements from source, visit [docs/BuildDigits.md](docs/BuildDigits.md) for instructions.
# Usage

Once you have installed DIGITS, visit [docs/GettingStarted.md](docs/GettingStarted.md) for an introductory walkthrough.

Then, take a look at some of the other documentation at [docs/](docs/) and [examples/](examples/):

* [Getting started with Torch](docs/GettingStartedTorch.md)
* [Fine-tune a pretrained model](examples/fine-tuning/README.md)
* [Train an autoencoder network](examples/autoencoder/README.md)
* [Train a regression network](examples/regression/README.md)
* [Train a Siamese network](examples/siamese/README.md)
* [Train a text classification network](examples/text-classification/README.md)
* [Train an object detection network](examples/object-detection/README.md)
* [Learn more about weight initialization](examples/weight-init/README.md)
* [Use Python layers in your Caffe networks](examples/python-layer/README.md)
* [Download a model and use it to classify an image outside of DIGITS](examples/classification/README.md)
* [Overview of the REST API](docs/API.md)

# Get help

### Installation issues
Expand Down
62 changes: 4 additions & 58 deletions digits-devserver
Original file line number Diff line number Diff line change
@@ -1,60 +1,6 @@
#!/usr/bin/env python2
# Copyright (c) 2014-2016, NVIDIA CORPORATION. All rights reserved.
#!/bin/bash
# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.

import argparse
import sys

import digits

if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Run the DIGITS development server')
parser.add_argument('-p', '--port',
type=int,
default=5000,
help='Port to run app on (default 5000)'
)
parser.add_argument('-c', '--config',
action='store_true',
help='Edit the application configuration'
)
parser.add_argument('-d', '--debug',
action='store_true',
help='Run the application in debug mode (reloads when the source changes and gives more detailed error messages)'
)
parser.add_argument('--version',
action='store_true',
help='Print the version number and exit'
)

args = vars(parser.parse_args())

from digits import config

if args['version']:
print digits.__version__
sys.exit()

if args['config']:
config.load_config('normal')
else:
config.load_config('quiet')

from digits.webapp import app, socketio, scheduler

print ' ___ ___ ___ ___ _____ ___'
print ' | \_ _/ __|_ _|_ _/ __|'
print ' | |) | | (_ || | | | \__ \\'
print ' |___/___\___|___| |_| |___/', digits.__version__
print

try:
if not scheduler.start():
print 'ERROR: Scheduler would not start'
else:
app.debug = args['debug']
socketio.run(app, '0.0.0.0', args['port'])
except KeyboardInterrupt:
pass
finally:
scheduler.stop()
set -e

python -m digits $@
26 changes: 0 additions & 26 deletions digits-server

This file was deleted.

3 changes: 3 additions & 0 deletions digits/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
/digits.cfg
/jobs/
/digits.log*
5 changes: 4 additions & 1 deletion digits/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
__version__ = '4.1-dev'
# Copyright (c) 2014-2016, NVIDIA CORPORATION. All rights reserved.
from __future__ import absolute_import

from .version import __version__
66 changes: 66 additions & 0 deletions digits/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Copyright (c) 2014-2016, NVIDIA CORPORATION. All rights reserved.

import argparse
import os.path
import sys


# Update PATH to include the local DIGITS directory
PARENT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
found_parent_dir = False
for p in sys.path:
if os.path.abspath(p) == PARENT_DIR:
found_parent_dir = True
break
if not found_parent_dir:
sys.path.insert(0, PARENT_DIR)


def main():
parser = argparse.ArgumentParser(description='DIGITS server')
parser.add_argument('-p', '--port',
type=int,
default=5000,
help='Port to run app on (default 5000)'
)
parser.add_argument('-d', '--debug',
action='store_true',
help='Run the application in debug mode (reloads when the source changes and gives more detailed error messages)'
)
parser.add_argument('--version',
action='store_true',
help='Print the version number and exit'
)

args = vars(parser.parse_args())

import digits

if args['version']:
print digits.__version__
sys.exit()

print ' ___ ___ ___ ___ _____ ___'
print ' | \_ _/ __|_ _|_ _/ __|'
print ' | |) | | (_ || | | | \__ \\'
print ' |___/___\___|___| |_| |___/', digits.__version__
print

import digits.config
import digits.log
import digits.webapp

try:
if not digits.webapp.scheduler.start():
print 'ERROR: Scheduler would not start'
else:
digits.webapp.app.debug = args['debug']
digits.webapp.socketio.run(digits.webapp.app, '0.0.0.0', args['port'])
except KeyboardInterrupt:
pass
finally:
digits.webapp.scheduler.stop()


if __name__ == '__main__':
main()
22 changes: 14 additions & 8 deletions digits/config/__init__.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,20 @@
# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved.
from __future__ import absolute_import

import os
# Create this object before importing the following imports, since they edit the list
option_list = {}

# These are the only two functions that the rest of DIGITS needs to use
from .current_config import config_value
from .load import load_config
from . import caffe
from . import gpu_list
from . import jobs_dir
from . import log_file
from . import torch
from . import server_name

if 'DIGITS_MODE_TEST' in os.environ:
# load the config automatically during testing
# it's hard to do it manually with nosetests
load_config()

def config_value(option):
"""
Return the current configuration value for the given option
"""
return option_list[option]

Loading