Skip to content

Commit

Permalink
Remove pybuilder, Add poetry (#215)
Browse files Browse the repository at this point in the history 
* Remove pybuilder, use flakeheaven, bandit

* Delete build.py

* Delete requirements.txt

* Delete build-requirements.txt

* Delete setup.py

* Add poetry

* Update pyproject.toml

* Create poetry.lock

* Run unit tests with pytest

Run unit tests with pytest using config in pyproject.toml

* Update ci-build.yml

* Rename CI jobs

* Update mnist.py

* Update README.md

* Update PULL_REQUEST_TEMPLATE.md
  • Loading branch information
@mgkwill
mgkwill committed Mar 8, 2022
1 parent 9622855 commit 46ad705
Show file tree
Hide file tree
Showing 10 changed files with 2,426 additions and 275 deletions.
4 changes: 2 additions & 2 deletions .github/PULL_REQUEST_TEMPLATE.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ Your PR fulfills the following requirements:
- [ ] [Docs](https://github.com/lava-nc/docs) reviewed and added / updated if needed (for bug fixes / features)
- [ ] PR conforms to [Coding Conventions](https://lava-nc.org/developer_guide.html#coding-conventions)
- [ ] [PR applys BSD 3-clause or LGPL2.1+ Licenses](https://lava-nc.org/developer_guide.html#add-a-license) to all code files
- [ ] Lint (`pyb`) passes locally
- [ ] Build tests (`pyb -E unit`) or (`python -m unittest`) passes locally
- [ ] Lint (`flakeheaven lint src/lava tests/`) and (`bandit -r src/lava/.`) pass locally
- [ ] Build tests (`pytest`) passes locally


## Pull request type
Expand Down
99 changes: 81 additions & 18 deletions .github/workflows/ci-build.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ on: [push, pull_request]

jobs:
lint:
name: Lint with flake8
name: Lint Code
runs-on: ubuntu-latest

steps:
Expand All @@ -14,17 +14,40 @@ jobs:
with:
python-version: 3.9

- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true

- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v2
with:
path: ~/.cache
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-01-${{ hashFiles('**/poetry.lock') }}

- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --no-root

- name: Install Lava
run: poetry install --no-interaction

- name: Run flakeheaven (flake8)
if: runner.os == 'Linux' || runner.os == 'macOS'
run: |
python -m pip install --upgrade pip
pip install pybuilder
source $VENV
flakeheaven lint src/lava tests/
- name: Run pybuilder w/o unit tests
- name: Run flakeheaven (flake8)
if: runner.os == 'Windows'
run: |
pyb analyze
.venv\Scripts\activate.ps1
flakeheaven lint src/lava tests/
security-lint:
name: Security Lint with bandit
name: Security Lint Code
runs-on: ubuntu-latest

steps:
Expand All @@ -35,17 +58,35 @@ jobs:
with:
python-version: 3.9

- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true

- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v2
with:
path: ~/.cache
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-01-${{ hashFiles('**/poetry.lock') }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pybuilder
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --no-root

- name: Run pybuilder to lint security w/ bandit
run: |
pyb bandit
- name: Install Lava
run: poetry install --no-interaction

- name: Run bandit
uses: tj-actions/[email protected]
with:
targets: |
src/lava/.
options: "-r --format custom --msg-template '{abspath}:{line}: {test_id}[bandit]: {severity}: {msg}'"

unit-tests:
name: Run unit tests
name: Unit Test Code + Coverage
runs-on: ${{ matrix.operating-system }}
strategy:
matrix:
Expand All @@ -59,12 +100,34 @@ jobs:
with:
python-version: 3.9

- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true

- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v2
with:
path: ~/.cache
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-01-${{ hashFiles('**/poetry.lock') }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install .
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --no-root

- name: Install Lava
run: poetry install --no-interaction

- name: Run unit tests
if: runner.os == 'Linux' || runner.os == 'macOS'
run: |
source $VENV
pytest
- name: Run unit tests
if: runner.os == 'Windows'
run: |
python -m unittest
.venv\Scripts\activate.ps1
pytest
99 changes: 57 additions & 42 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,53 +65,68 @@ All of Lava's core APIs and higher-level components are released, by default, wi

### Cloning Lava and Running from Source

We highly recommend cloning the repository and using pybuilder to setup lava.
You will need to install pybuilder for the same.
We highly recommend cloning the repository and using poetry to setup lava.
You will need to install poetry.

Open a python terminal and run based on the OS you are on:

#### [Linux/MacOS]
```bash
$ cd ~
$ python3 -m venv python3_venv
$ source python3_venv/bin/activate
$ pip install -U pip
$ pip install poetry>=1.1.13
$ git clone [email protected]:lava-nc/lava.git
$ cd lava
$ pip install -r build-requirements.txt
$ pip install -r requirements.txt
$ pip install -e .
$ python -m unittest
$ poetry config virtualenvs.in-project = true
$ poetry install
$ source .venv/bin/activate
$ pytest
```
Note that you should install the core Lava repository (lava) before installing other Lava libraries such as lava-optimization or lava-dl.

#### [Windows]
```cmd
cd %HOMEPATH%
python -m venv python3_venv
python3_venv\Scripts\activate.bat
python -m pip install --upgrade pip
pip install poetry>=1.1.13
git clone [email protected]:lava-nc/lava.git
cd lava
pip install -r build-requirements.txt
pip install -r requirements.txt
pip install -e .
python -m unittest
poetry config virtualenvs.in-project = true
poetry install
.venv\Scripts\activate
pytest
```
Note that you should install the core Lava repository (lava) before installing other Lava libraries such as lava-optimization or lava-dl.

You should expect the following output after running the unit tests:
```
PyBuilder version 0.13.3
Build started at 2021-10-25 13:32:02
------------------------------------------------------------
[INFO] Activated environments: unit
[INFO] Building Lava version 0.1.0
...... PyBuilder Logs ...
[INFO] Running unit tests
[INFO] Executing unit tests from Python modules in /home/user/lava/lava/tests
[INFO] Executed 72 unit tests
[INFO] All unit tests passed.
$ pytest
============================================== test session starts ==============================================
platform linux -- Python 3.8.10, pytest-7.0.1, pluggy-1.0.0
rootdir: /home/user/lava, configfile: pyproject.toml, testpaths: tests
plugins: cov-3.0.0
collected 205 items
tests/lava/magma/compiler/test_channel_builder.py . [ 0%]
tests/lava/magma/compiler/test_compiler.py ........................ [ 12%]
tests/lava/magma/compiler/test_node.py .. [ 13%]
tests/lava/magma/compiler/builder/test_channel_builder.py . [ 13%]
...... pytest output ...
tests/lava/proc/sdn/test_models.py ........ [ 98%]
tests/lava/proc/sdn/test_process.py ... [100%] =============================================== warnings summary ================================================
...... pytest output ...
src/lava/proc/lif/process.py 38 0 100%
src/lava/proc/monitor/models.py 27 0 100%
src/lava/proc/monitor/process.py 79 0 100%
src/lava/proc/sdn/models.py 159 9 94% 199-202, 225-231
src/lava/proc/sdn/process.py 59 0 100%
-----------------------------------------------------------------------------------------------------------------TOTAL 4048 453 89%
Required test coverage of 85.0% reached. Total coverage: 88.81%
============================ 199 passed, 6 skipped, 2 warnings in 118.17s (0:01:58) =============================
```

### [Alternative] Installing Lava from Binaries
Expand All @@ -125,37 +140,37 @@ Open a python terminal and run:

#### [Windows/MacOS/Linux]
```bash
$ python3 -m venv python3_venv
$ python3 -m venv .venv
$ source .venv/bin/activate ## Or Windows: .venv\Scripts\activate
$ pip install -U pip
$ pip install lava-nc-0.3.0.tar.gz
```

### Linting, Testing, Documentation and Packaging

```bash
# Install pybuilder
$ pip install -r build-requirements.txt
# Install poetry
$ pip install poetry
$ poetry config virtualenvs.in-project = true
$ poetry install
$ poetry shell

# Run linting
$ pyb analyze
$ flakeheaven lint src/lava tests

# Run unit tests
$ pyb -E unit
# Alternate unit test run
$ python -m unittest

# Generate documentation
$ pyb sphinx_generate_documentation
# Run unit tests
$ pytest

# Create distribution
$ pyb publish
#### Find dists at target/dist/Lava-<version>/dist/
$ poetry build
#### Find builds at dist/

# Run Secuity Linting
$ pyb bandit
$ bandit -r src/lava/.
#### If security linting fails run bandit directly
#### and format failures
S bandit -r lava/ --format custom --msg-template "{abspath}:{line}: {test_id}[bandit]: {severity}: {msg}"
$ bandit -r src/lava/. --format custom --msg-template '{abspath}:{line}: {test_id}[bandit]: {severity}: {msg}'
```

## Running Lava on Intel Loihi
Expand All @@ -175,8 +190,8 @@ instructions, email: [email protected]

- Login to INRC VM with your credentials
- Follow the instructions to Install or Clone Lava
- cd /nfs/ncl/releases/lava/0.1.0
- pip install lava-nc-0.1.0.tar.gz
- cd /nfs/ncl/releases/lava/0.3.0
- pip install lava-nc-0.3.0.tar.gz

## Coding example
### Building a simple feed-forward network
Expand Down
9 changes: 0 additions & 9 deletions build-requirements.txt
View file

This file was deleted.

Loading

0 comments on commit 46ad705

Please sign in to comment.