Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor ojd_daps_skills #223

Merged
merged 49 commits into from
Aug 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
ca03403
initial refactor commit
May 7, 2024
6b80592
refactor v2
May 8, 2024
dc88ae1
isort file
May 8, 2024
b9fe09e
update pytest; isort files
May 8, 2024
fcab149
use different poetry action
May 8, 2024
5dcbcac
update poetry
May 8, 2024
08153df
update pytest file
May 8, 2024
902db16
update workflow
May 8, 2024
7be49c7
update workflow
May 8, 2024
2ea4d85
update pytest
May 8, 2024
ecc50d5
remove 3.10
May 8, 2024
33e30c9
only ubuntu;3.9
May 8, 2024
13d95cc
multiple python versions and systems:
May 8, 2024
b64ef1e
test for 3.9, 3.10
May 8, 2024
ddb643e
multiple system compatability
May 8, 2024
1f5c074
change poetry version
May 8, 2024
8c6e899
change poetry version
May 8, 2024
1692e74
fix error
May 8, 2024
61346e0
remove windows
May 8, 2024
9766c1b
update workflow
May 8, 2024
f2181ec
update workflow
May 8, 2024
8bb59d0
install spacy model
May 8, 2024
1372b8e
test with just windows
May 8, 2024
582d741
add poetry path
May 8, 2024
d770827
test just windows
May 8, 2024
4948f10
remove windows
May 8, 2024
b55432e
add lint github actions
May 8, 2024
321f09a
update readme
May 8, 2024
7f47cb3
fix formatting
May 8, 2024
b5e9ccd
enforce black style
May 8, 2024
6fb49c3
black and isort are circular
May 8, 2024
1542deb
remove isort as its in conflict with black
May 8, 2024
620ea0b
add lock file
May 8, 2024
d7bda99
update extract_skills_config
May 8, 2024
4839866
add poetry.lock
May 8, 2024
0c87545
debug
May 10, 2024
ec208b9
debug
May 10, 2024
c25a3c5
fix multiskill appending, skills mapper test and use refactor s3 fold…
lizgzil Jun 26, 2024
2d3ea01
Use latest MS model version and correct some issues in the MS process…
lizgzil Jun 27, 2024
0f9e570
fix black formatting
Jack-Vines Jul 9, 2024
8b64b91
update linting
Jack-Vines Jul 9, 2024
b27c25d
fix linting
Jack-Vines Jul 9, 2024
682f25d
Update tests
Jack-Vines Aug 4, 2024
3f7c569
testing
Jack-Vines Aug 4, 2024
795bed0
ruff improvements
Jack-Vines Aug 4, 2024
3804c7d
Black improvements
Jack-Vines Aug 4, 2024
a8e3864
Update README
Jack-Vines Aug 4, 2024
377d795
add docs
Jack-Vines Aug 13, 2024
028dda5
Merge branch 'dev' into ojoner
Jack-Vines Aug 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .cookiecutter/config

This file was deleted.

8 changes: 0 additions & 8 deletions .envrc

This file was deleted.

33 changes: 33 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Deploy docs to gh-pages

on:
workflow_dispatch:
push:
branches:
- dev
- main

jobs:
build:
runs-on: ubuntu-latest
name: Deploy docs to gh-pages
steps:
- name: Checkout
uses: actions/checkout@v2

- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.8

# - name: Install dependencies
# run: |
# $CONDA/bin/python -m pip install --upgrade pip
# $CONDA/bin/python -m pip install mkdocs-material

- name: Deploy
uses: mhausenblas/mkdocs-deploy-gh-pages@master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CONFIG_FILE: mkdocs.yml
REQUIREMENTS: requirements.txt
33 changes: 33 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Linting

on:
push:

jobs:
linting-python:
name: Lint Python
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v3

- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: "3.9"

- name: Install Dependencies
shell: bash
run: |
pip install ruff black isort

- name: Run ruff
if: always()
shell: bash
run: python -m ruff check ojd_daps_skills tests

- name: Run black
if: always()
shell: bash
run: black --check ojd_daps_skills tests
49 changes: 0 additions & 49 deletions .github/workflows/pytest.yml

This file was deleted.

58 changes: 58 additions & 0 deletions .github/workflows/pytest.yml.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: Unit Tests

on: [push]

jobs:
test:
runs-on: ${{ matrix.os }}

strategy:
matrix:
os: ["ubuntu-latest", "macos-latest"]
python-version: ["3.9", "3.10"]
steps:
#----------------------------------------------
# check-out repo and set-up python
#----------------------------------------------
- name: Check out repository
uses: actions/checkout@v4
- name: Set up python ${{ matrix.python-version }}
id: setup-python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
#----------------------------------------------
# ----- install & configure poetry -----
#----------------------------------------------
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
#----------------------------------------------
# load cached venv if cache exists
#----------------------------------------------
- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v3
with:
path: .venv
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ hashFiles('**/poetry.lock') }}
#----------------------------------------------
# install dependencies if cache does not exist
#----------------------------------------------
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --no-root
#----------------------------------------------
# install your root project, if required
#----------------------------------------------
- name: Install additional dependencies
run: |
poetry install --no-interaction
poetry run python -m spacy download en_core_web_sm
#----------------------------------------------
# add matrix specifics and run test suite
#----------------------------------------------
- name: Run tests
run: poetry run pytest tests/ --verbose
25 changes: 0 additions & 25 deletions .github/workflows/release.yml

This file was deleted.

5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@

*.lock

#ignore data files
__MACOSX/
ojd_daps_skills_data/
ojd_daps_skills_models/

# Proposed convention
scratch*

Expand Down
49 changes: 1 addition & 48 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,54 +7,7 @@ repos:
language: system
types: [python]
require_serial: true
- id: check-added-large-files
name: Check for added large files
entry: check-added-large-files
language: system
- id: check-merge-conflict
name: Check for files with merge conflict strings
entry: check-merge-conflict
language: system
- id: end-of-file-fixer
name: Fix End of Files
entry: end-of-file-fixer
language: system
types: [text]
stages: [commit, push, manual]
- id: trailing-whitespace
name: Trim Trailing Whitespace
entry: trailing-whitespace-fixer
language: system
types: [text]
stages: [commit, push, manual]
- id: check-toml
name: Check Toml
entry: check-toml
language: system
types: [toml]
- id: check-yaml
name: Check Yaml
entry: check-yaml
language: system
types: [yaml]
# - id: reorder-python-imports
# name: Reorder python imports
# entry: reorder-python-imports
# language: system
# types: [python]
# - id: flake8
# name: flake8
# entry: flake8
# language: system
# types: [python]
# args: ["--config=.flake8"]
# require_serial: true
- id: no-commit-to-branch
name: Prevent commits to dev / master
entry: no-commit-to-branch
language: python
args: ["-b", dev, "-b", master, "-b", main]
pass_filenames: false

- repo: https://github.com/prettier/pre-commit
rev: v2.1.2
hooks:
Expand Down
10 changes: 0 additions & 10 deletions LICENSE

This file was deleted.

Loading
Loading