-
Python 3.6.9 (or 3.6.x)
-
Any decent text editor (e.g. Notepad++ )
-
PyCharm Community Edition
-
Git Client 2.22.x or better
For CDK pipelines
-
Install aws-cli .See https://cdkworkshop.com/15-prerequisites/100-awscli.html
-
Install nodejs with npm https://nodejs.org/
-
Install aws-cdk toolkit. Make sure it is added in path
npm install -g aws-cdk
- Markdown: For this readme documentation https://www.markdownguide.org/basic-syntax/
- CDK https://cdkworkshop.com/
- PyGreSQL (use DB-API) http://www.pygresql.org/contents/tutorial.html#first-steps-with-the-db-api-2-0-interface
-
Ensure python, git executables are in your PATH variable.
-
Checkout code using git clone into a directory, say: ROOT
git clone
-
Change directory to ROOT
cd $ROOT
-
Create Python Virtual environment. Make sure you are using virtualenv belonging to python 3.6.x
- Linux/macOS : virtualenv venv
-
Open Terminal and activate your virtual environment and install external python modules via below command:
(Windows): venv\Scripts\activate
pip install -r requirements-dev.txt
-
Configure the project in PyCharm IDE
- Open python code root directory via menu
File | Open... | Open File or Folder (choose $ROOT)
-
Once opened, configure the Project Interpreter (Python executable)
Note:- PyCharm may already detected venv automatically. So you may not need to perform this step.
File | Settings | Project: aws-glue-python-kickstart| Project Interpreter
In
Settings
dialog box, click onsettings button
in front ofProject Interpreter
dropdown box
and selectAdd..
InAdd Python Interpreter
dialog box chooseExisting environment
option
and select (windows) $ROOT/venv/Scripts/python path.
ClickOK
to close the dialog box
Now ensure or choose the newly added virtual environment in the drop down field (on top)
ClickApply
to apply change. -
Configure the pytest as default integrated test runner.
Note:- PyCharm may already detected pytest as default runner automatically. So you may not need to perform this step.
File | Settings | Tools | Python Integrated Tools
Chose
py.test
in dropdown fieldDefault test Runner:
and click onApply
button -
Configure default run configuration
Run | Edit Configurations...
> File | Settings | Tools | Python Integrated Tools
Navigate to
Templates
>Python panel
For fieldWorking Directory
choose project root path ($ROOT
)Navigate to
Templates
>Python tests
>pytest
For fieldWorking Directory
choose project root path ($ROOT
)
and click onOK
button to close theRun/Debug Configurations
dialog box. -
For setting any OS ENVIRONMENT variable, you can create the
.env
file at root folder by copying the file.env.template
. This file is meant to host environment variable meaningful for your local environment only. So never commit this. -
Add Source directories
devops
-
Activate the venv, if not already done
cd $ROOT
(windows) venv/Scripts/activate -
Code Build
No compilation required (as of now)
-
Python Code style check (report at target/flake-reports/index.html)
See
setup.cfg#flake8
section for custom PEP-8 rule configurationflake8
Note:- You may use
# flake8: noqa
for disabling inspection on any code line -
Python Unit Test (report at target/pytest-report.html)
You can run all unit test cases from PyCharm too by right clicking on
tests
directory andRun pytests in tests
optionpytest
-
Python Unit Test with code Coverage (target/coverage-reports/index.html)
See
setup.cfg#coverage:report
section for configuration and coverage thresholdpytest --cov=osiris --cov-report html --cov-report term
- Cover code under package
osiris
- Generate html report and
- Generate terminal report
- Cover code under package
-
Generating .whl files for Glue Shell Jobs
python setup.py package
- Generate artifacts under dist/ directory.
-
Run CDK
cdk synth [--toolkit-stack-name stackname ]
cdk deploy
cdk destroy