Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

201 update dependencies #202

Merged
merged 43 commits into from
Oct 9, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
a15e381
fix(cython bugs): remove awscli
sqr00t Sep 25, 2023
1141278
fix(cython bugs): update numpy==1.24.4, scipy==1.10.1, scikit-learn==…
sqr00t Sep 25, 2023
0af4119
fix(remove awscli): removes dependency on awscli for retrieving publi…
sqr00t Sep 27, 2023
e3a6d29
fix(app): remove numpy spec, remove awscli
sqr00t Sep 27, 2023
af6157e
fix(transformers): bump transformers version to 4.33.3 dependabot alert
sqr00t Sep 27, 2023
b42df79
chore(data download): remove duplicate code
sqr00t Sep 28, 2023
7a3693a
test-feat(windows actions): experiment adding platform dimensions to …
sqr00t Sep 28, 2023
a945e4e
test-fix(windows actions): try pip as pymodule
sqr00t Sep 28, 2023
beb5ca3
test-fix(windows actions): add runner.os conditional evals
sqr00t Sep 28, 2023
964cea0
test-fix(windows actions): rearrange config
sqr00t Sep 28, 2023
08ef449
test-fix(windows actions): attempt specifying bash shell
sqr00t Sep 28, 2023
81a0eaa
test-fix(windows actions): add encoding env var
sqr00t Sep 28, 2023
2917c3f
test-fix(windows actions): move encoding utf8 env var
sqr00t Sep 28, 2023
3d9d671
test-fix(windows actions): use SETX
sqr00t Sep 28, 2023
f87d175
test-fix(windows actions): change env setting syntax
sqr00t Sep 28, 2023
5f65ee1
test-fix(windows actions): move encoding utf8 env var
sqr00t Sep 28, 2023
4ee2408
test-fix(windows actions): try pip as py3 module with utf8 io env var
sqr00t Sep 28, 2023
03a08c6
test-fix(windows actions): try another env var
sqr00t Sep 28, 2023
ab33aad
test-fix(windows actions): remove editable mode install
sqr00t Sep 28, 2023
9475d9e
test-fix(windows actions): add back editable install
sqr00t Sep 28, 2023
f29626a
fix(windows actions): remove windows actions due to complexities
sqr00t Sep 28, 2023
66c5666
test-feat(gh actions cache): add caching pip dependencies
sqr00t Sep 28, 2023
77fa26b
revert
Aiden-RC Sep 28, 2023
b576760
Merge branch 'nestauk:dev' into dev
Aiden-RC Sep 28, 2023
ff5f996
Merge remote-tracking branch 'upstream/201_update_dependencies' into dev
Aiden-RC Sep 28, 2023
d5703d3
Merge branch 'dev' of https://github.com/Aiden-RC/ojd_daps_skills int…
Aiden-RC Sep 28, 2023
24bd7c3
Update pytest.yml
Aiden-RC Sep 28, 2023
fcaea55
Update pytest.yml
Aiden-RC Sep 28, 2023
8b514c0
Update setup.py
Aiden-RC Sep 28, 2023
c03a948
try except potential fix for tag cmd
Aiden-RC Sep 28, 2023
961063d
Update setup.py
Aiden-RC Sep 28, 2023
2c55d0a
Update setup.py
Aiden-RC Sep 28, 2023
9d2dade
Update setup.py
Aiden-RC Sep 28, 2023
5fbe787
remove probably un-used code
Aiden-RC Sep 28, 2023
f971eae
remove aws cli from requirements
Aiden-RC Sep 28, 2023
ad825e1
Merge pull request #204 from Aiden-RC/dev
india-kerle Oct 6, 2023
31e58f7
fix(version tag): add pyproject.toml to use setuptools_scm with relea…
sqr00t Oct 6, 2023
ccd6c36
fix(merge conflict): remove workflow action on pull request
sqr00t Oct 6, 2023
505db88
fix(merge conflict): remove AWSCLI section of docs
sqr00t Oct 6, 2023
0f5470d
chore(setup): use Path object instead of os.path, remove unused imports
sqr00t Oct 6, 2023
efebf85
feat(gh actions cache): enable caching unchanged pip dependencies
sqr00t Oct 6, 2023
d2fdbae
chore!(file download): remove manual download of base model, remove c…
sqr00t Oct 9, 2023
958fd6e
chore(docs): update Sphinx autodocs
sqr00t Oct 9, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 26 additions & 3 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,46 @@ on: [push]

jobs:
build:
runs-on: ubuntu-latest
runs-on: ${{ matrix.os.host }}
strategy:
matrix:
python-version: ["3.8", "3.9"]
os:
- name: ubuntu
host: ubuntu-latest
- name: windows
host: windows-latest

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
cache: "pip"

- name: (ubuntu) Install dependencies
if: runner.os != 'windows'
run: |
python -m pip install --upgrade pip
pip install -r requirements_dev.txt
python -m spacy download en_core_web_sm
pip install -e ."[test]"
- name: Test with pytest
- name: (ubuntu) Test with pytest
if: runner.os != 'windows'
run: |
pytest --verbose

- name: (windows) Install dependencies
if: runner.os == 'windows'
shell: bash
run: |
python -m pip install --upgrade pip
pip install -r requirements_dev.txt
python -m spacy download en_core_web_sm
pip install -e .
- name: (windows) Test with pytest
if: runner.os == 'windows'
shell: bash
run: |
pytest --verbose
15 changes: 2 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,16 +29,9 @@ You can use pip to install the library:
pip install ojd-daps-skills
```

You will also need to download [spaCy's](https://spacy.io/models/en) `en_core_web_sm` model:
Note that this package was developed on MacOS and tested on Ubuntu. Changes have been made to be compatible on a Windows system but are not tested and cannot be guaranteed.

```
python -m spacy download en_core_web_sm
```

Note that this package was developed on MacOS and tested on Ubuntu. Changes have been made to be compatible on a Windows system but are not tested and cannot be guaranteed.
### AWS CLI
sqr00t marked this conversation as resolved.
Show resolved Hide resolved

When the package is first used it will automatically download a folder of neccessary data and models. This file is ~ 1GB. Although you don't need to have AWS credentials for this to work, you will need to download the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html).
When the package is first used it will automatically download a folder of neccessary data and models. (~1GB)

## TL;DR: Using Nesta's Skills Extractor library <a name="usage"></a>

Expand Down Expand Up @@ -152,10 +145,6 @@ git clone [email protected]:nestauk/ojd_daps_skills.git
- `mkdir .cookiecutter/state`
- `touch .cookiecutter/state/conda-create.log`
- Run `make install` to configure the development environment
- Download spacy model:
- `python -m spacy download en_core_web_sm`

If you don't have the AWS CLI installed - you can download a zipped folder of the data [by clicking here](https://open-jobs-indicators.s3.eu-west-1.amazonaws.com/escoe_extension/ojd_daps_skills_data.zip). After downloading and unzipping, it is important that this folder is moved to the project's parent folder - i.e. `ojd_daps_skills/`.

### Project structure

Expand Down
Binary file modified docs/build/doctrees/about.doctree
Binary file not shown.
Binary file modified docs/build/doctrees/custom_usage.doctree
Binary file not shown.
Binary file modified docs/build/doctrees/environment.pickle
Binary file not shown.
Binary file modified docs/build/doctrees/extract_skills.doctree
Binary file not shown.
Binary file modified docs/build/doctrees/index.doctree
Binary file not shown.
Binary file modified docs/build/doctrees/labelling.doctree
Binary file not shown.
Binary file modified docs/build/doctrees/license.doctree
Binary file not shown.
Binary file modified docs/build/doctrees/model_card.doctree
Binary file not shown.
Binary file modified docs/build/doctrees/pipeline_summary.doctree
Binary file not shown.
Loading