Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding github action and docs #1

Merged
merged 27 commits into from
May 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
fb13d91
added template of pyproject toml
May 10, 2024
4ab983c
chore: Add python-dependencies-vscode extension to devcontainer.json
May 10, 2024
f323237
chore: Add Dependabot configuration for weekly updates of pip and doc…
May 10, 2024
4b60ff5
chore: Add unit test workflow for running tests and coverage
May 10, 2024
d267a1b
chore: Add Github Actions workflow for running unit tests and generat…
May 10, 2024
33d79af
chore: Update unit test workflow to run on main branch and schedule i…
May 10, 2024
fe36025
Add placeholder unit test
May 10, 2024
88df794
feat: Add placeholder function and unit test
May 10, 2024
e19b149
chore: Update unit test workflow to run on multiple Python versions
May 10, 2024
7a84642
chore: Update Python versions in unit test workflow
May 10, 2024
491e044
chore: Update Dockerfile to include Python 3.12-bullseye base image a…
May 10, 2024
2251314
chore: Update README.md with badges for CI/CD and license information
May 10, 2024
d48ed4a
chore: Add Docker Hub and GitHub Container Registry image publishing …
May 10, 2024
fb79042
chore: Add permissions for reading contents and writing packages in p…
May 10, 2024
e3071b9
chore: Add Docker Hub and GitHub Container Registry image publishing …
May 10, 2024
aff7518
chore: Add conditional steps for Python 3.10 in unit test workflow
May 10, 2024
b9feea1
chore: Update Dockerfile to include Python 3.12-bullseye base image a…
May 10, 2024
b18ea5c
chore: Update Dockerfile.dev to install additional Python packages fo…
May 10, 2024
19c0d12
chore: Update pre-commit hooks and dependencies for code formatting a…
May 10, 2024
6aa513c
chore: Update Dockerfile.dev to install additional Python packages fo…
May 10, 2024
4bd3d38
chore: Update devcontainer.json to install pre-commit hooks and depen…
May 10, 2024
cd0323e
chore: Update devcontainer.json, Dockerfile.dev, and pre-commit hooks…
May 10, 2024
2ffd3e1
Merge branch 'main' into dev
ninja-asa May 10, 2024
1f12925
chore: Update black pre-commit hook to version 24.4.2
May 10, 2024
b0bbc8d
Merge branch 'dev' of https://github.com/ninja-asa/template-python-pr…
May 10, 2024
afd54f8
chore: adding black under project in TOML
May 10, 2024
0dbe17d
chore: Remove black pre-commit hook and update dependencies
May 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .devcontainer/Dockerfile.dev
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,6 @@ RUN apt-get update -y && apt-get install --no-install-recommends -y \

RUN apt install nano -y

# Installing the requirements.txt
# Install common python packages for development
RUN pip install --upgrade pip
RUN pip install black pytest coverage poetry pylint pre-commit
6 changes: 3 additions & 3 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
// README at: https://github.com/devcontainers/templates/tree/main/src/anaconda
{
"name": "Python 3.12",
"build": {

"build": {
"dockerfile": "Dockerfile.dev"
},

Expand All @@ -14,7 +14,7 @@
// "forwardPorts": [],

// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "python --version",
"postCreateCommand": "python --version && pre-commit install",
"customizations": {
"vscode": {
"extensions": [
Expand Down
15 changes: 15 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file

version: 2
updates:
- package-ecosystem: "pip" # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "weekly"
- package-ecosystem: "docker" # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "weekly"
71 changes: 71 additions & 0 deletions .github/workflows/publish-docker-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
name: Template for Docker Image Push to Docker Hub
on:
push:
branches: [ "main", "dev"]
paths:
- 'Dockerfile'
- 'app/**'
- 'tests/**'
- 'pyproject.toml'
- '.github/workflows/publish-docker-image.yml'

workflow_dispatch: # allow user to specify which
inputs:
environment:
description: 'Target environment'
required: true
default: 'dev' # default value if not provided

permissions:
contents: read
packages: write

jobs:
push_to_registry:
name: Push Docker Image to Docker Hub
runs-on: ubuntu-latest
steps:
- name: Checkout the repo
uses: actions/checkout@v4
- name: Build version
id: date
run: echo "{name}=date::$(date +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_OUTPUT

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/[email protected]
with:
images: |
ninjaasa/template-python-project
ghcr.io/${{ github.repository }}
# tag image to be latest if pushing to main branch, dev if pushing to dev branch.
# also tag combined with run_id to ensure unique tag
tags: |
type=raw,value=${{ github.ref == 'refs/heads/main' && 'latest' || 'dev' }}_${{ github.run_id }}
type=raw,value=${{ github.ref == 'refs/heads/main' && 'latest' || 'dev' }}
labels: |
org.label-schema.build-date=${{ steps.date.outputs.date}}
org.opencontainers.image.created=${{ steps.date.outputs.date}}

- name: Build and push Docker image
uses: docker/[email protected]
with:
context: .
file: Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

78 changes: 78 additions & 0 deletions .github/workflows/unit-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
name: Run Tests

on:
pull_request:
# Run when PR is submitted to main branch
branches: [ main ]
push:
branches:
# Run when code is pushed to main branch
- main
schedule:
# Run every weekday at 2am
- cron: "0 2 * * 1-5"
workflow_dispatch:
# Run when manually triggered

permissions:
actions: read
contents: read
pull-requests: write

jobs:
test:
name: Unit Tests and Coverage
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12"]
steps:
- uses: actions/[email protected]
- name: Set up Python
uses: actions/[email protected]
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install poetry
poetry install
- name: Run tests
run: poetry run pytest
- name: Check Coverage
if: matrix.python-version == '3.10'
run: |
poetry run coverage run -m pytest
poetry run coverage xml
echo "COVERAGE_PERCENT=$(poetry run coverage report --format=total | awk '{print $NF}' | tr -d '%')" >> $GITHUB_ENV

- name: Python Coverage
uses: orgoro/[email protected]
# only if pull request or push
if: github.event_name == 'pull_request' && matrix.python-version == '3.10'
with:
# local path to a coverage xml file like the output of pytest --cov
coverageFile: "coverage.xml"
# github token
token: ${{ secrets.GITHUB_TOKEN }}
# the coverage threshold for average over all files [0,1]
thresholdAll: 0.7

- name: Dynamic Badges
if: matrix.python-version == '3.10'
uses: Schneegans/[email protected]
with:
# Your secret with the gist scope
auth: ${{ secrets.GIST_SECRET }}
# The ID of the gist to use
gistID: 8e54c78cf86c9b23df72f9f987282266
# The *.json or *.svg filename of the badge data
filename: template-python-project-coverage.json
# The left text of the badge
label: Coverage
# The right text of the badge
message: ${{ env.COVERAGE_PERCENT }} %
valColorRange: ${{ env.COVERAGE_PERCENT }}
maxColorRange: 100
minColorRange: 0
forceUpdate: true
45 changes: 45 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
repos:
- repo: https://github.com/PyCQA/isort
rev: 5.13.2
hooks:
- id: isort
name: isort
args: ["--profile=black"]
- id: isort
name: isort (cython)
types: [cython]
args: ["--profile=black"]

- repo: https://github.com/psf/black-pre-commit-mirror
rev: 24.4.2
hooks:
- id: black
# It is recommended to specify the latest version of Python
# supported by your project here, or alternatively use
# pre-commit's default_language_version, see
# https://pre-commit.com/#top_level-default_language_version
language_version: python3.12

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
hooks:
- id: check-added-large-files
- id: check-ast
- id: check-builtin-literals
- id: check-case-conflict
- id: check-docstring-first
- id: check-shebang-scripts-are-executable
- id: check-merge-conflict
- id: debug-statements
- id: destroyed-symlinks
- id: detect-private-key
- id: end-of-file-fixer
exclude: ^LICENSE|\.(html|csv|txt|svg|py)$
- id: name-tests-test
args: [--pytest-test-first]
- id: no-commit-to-branch
args: [--branch, main]
- id: requirements-txt-fixer
- id: trailing-whitespace
args: [--markdown-linebreak-ext=md]
exclude: \.(html|svg)$
37 changes: 37 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Specifying the base image
FROM python:3.12-bullseye

# Install system dependencies and pip
RUN apt-get update -y && apt-get install --no-install-recommends -y \
python3-pip

RUN apt install nano -y

# Installing the requirements.txt
RUN pip install --upgrade pip

# Install local package and its requirements
COPY . /workspace
WORKDIR /workspace
RUN pip install .

# Set labels
LABEL org.opencontainers.image.source=https://github.com/ninja-asa/template-python-project/
LABEL org.label-schema.vcs-url=https://github.com/ninja-asa/template-python-project/

LABEL org.opencontainers.image.description="Template Python Project."
LABEL org.label-schema.description="Template Python Project"

LABEL org.opencontainers.image.licenses="MIT"

LABEL org.label-schema.schema-version="1.0"

LABEL org.label-schema.docker.cmd="docker run"

LABEL org.opencontainers.image.authors="ninja-asa"

LABEL org.opencontainers.image.title="Template Python Project"
LABEL org.label-schema.title="Template Python Project"

# Start the application
CMD ["python", "app"]
41 changes: 41 additions & 0 deletions Github.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Github Actions

Templates for Github Actions workflows I have used for Continuous Testing and Integration can be found in the `.github/workflows` directory.

## Run Unit Tests and Code Coverage

This workflow has the following jobs:
- run unit tests using `pytest`
- run code coverage using `coverage` and generating a report published in the PR
- update the code coverage badge in the `README.md` file

If using this template, you will need to perform the following steps:
- In [github.com](https://github.com):
- add a secret `GIST_SECRET` with read and write access to gists in your repository settings
- add a secret to dependabot `GIST_SECRET` with read and write access to gists in your repository settings
- create a public gist with the code coverage badge
- add `Read and Write` permissions under the repository settings, `Actions` section, to the `GITHUB_TOKEN` secret (allows publishing the coverage report in the PR)
- In the [workflow file](.github/workflows/run_tests.yml):
- update the `GIST_ID` and `filename` to match the gist you created
- In the [README.md](README.md):
- update the code coverage badge URL to contain the url to the gist you created

Useful links:
- [Github Action - Dynamic Badges](https://github.com/marketplace/actions/dynamic-badges)

## Publish Docker Image

This workflow has the following jobs:
- build the docker image
- publish the docker image to the Github Container Registry and Docker Hub

If using this template, you will need to perform the following steps:
- In [dockerhub.com](https://hub.docker.com):
- create a token with read and write access to your repositories
- In [github.com](https://github.com):
- add a secret `DOCKERHUB_USERNAME` with your Docker Hub username
- add a secret `DOCKERHUB_TOKEN` with your Docker Hub token
- In the [workflow file](.github/workflows/publish_docker_image.yml):
- update the `image_name` to match the name of your image
- update the `dockerfile` to match the name of your Dockerfile
- validate the tags being used
41 changes: 32 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,46 @@
# template-python-project
This is intended to be a repository to serve as a template to my own, and others projects using Python.
# template-python-project :page_facing_up:

[![Run Tests](https://github.com/ninja-asa/template-python-project/actions/workflows/unit-tests.yml/badge.svg)](https://github.com/ninja-asa/template-python-project/actions/workflows/unit-tests.yml)
![Coverage Badge](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/ninja-asa/8e54c78cf86c9b23df72f9f987282266/raw/7f5d2722c29497fa777f925552778219a137756d/template-python-project-coverage.json)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

This is intended to be a repository to serve as a template to my own, and others projects using Python.

## Contents

- `.github/workflows/`: contains the GitHub Actions workflows.
- `.vscode/`: contains the settings for Visual Studio Code.
- `.devcontainer/`: contains the settings for the development container and the development Dockerfile.
- `app/`: contains the source code of the project.
- `tests/`: contains the tests of the project.

## Getting Started

This template relies on using Docker for development and using Visual Studio Code as the IDE.

### Pre-requisites
- [Docker](https://www.docker.com/)
- [Visual Studio Code](https://code.visualstudio.com/)
- [Remote - Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)

### Step by Step
To start developing in any repository using this template:
- Clone the repository
- Open the repository in VSCode
- Setup intended Python version in the `Dockerfile.dev`
- Press `F1` and type `Remote-Containers: Reopen in Container`
- Start developing
- Start developing :rocket:

> During this setup, suggested extensions will be installed in the container, and the Python environment will be created, with `pytest`, `coverage`, `poetry` and `black` installed.

## Features
### Docker-based development
This is tuned for VSCode and to support container based development.
### Configuration

You will find the `.vscode` directory with the files needed to make it work. However, before starting to develop, check the `python` version in the `Dockerfile.dev` - ensure you are using a version that suits your needs.
- [Github Action](Github.md): details needed configuration for the GitHub Actions workflows.

## Useful links:
- support status of `python` in the [Python Developer's Guide](https://devguide.python.org/versions/#versions).
- vulnerabilities in the [Mailing List by Python Software Foundation CVE Numbering Authority and Python Security Response Team](https://mail.python.org/archives/list/[email protected]/latest).
- vulnerabilities in the
- Microsoft Package Template for Python [here](https://github.com/microsoft/python-package-template/blob/main/pyproject.toml).

## Common Issues
### Dev Container Cannot Start - Issue with communicating with Docker Enginer
Expand Down Expand Up @@ -47,4 +70,4 @@ Issue:
- Dev container cannot start building

Solution:
- Have no internet connectivity to get docker image from remote registry
- Have no internet connectivity to get docker image from remote registry
Empty file added app/__init__.py
Empty file.
7 changes: 7 additions & 0 deletions app/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
from placeholder import placeholder

def main():
print(placeholder())

if __name__ == '__main__':
main()
2 changes: 2 additions & 0 deletions app/placeholder.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
def placeholder():
return True
Loading
Loading