Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New Release 0.14.2 #1323

Open
wants to merge 49 commits into
base: main
Choose a base branch
from
Open

New Release 0.14.2 #1323

wants to merge 49 commits into from

Conversation

LennartPurucker
Copy link
Contributor

No description provided.

pre-commit-ci bot and others added 30 commits August 1, 2023 02:24
@pre-commit-ci
[pre-commit.ci] pre-commit autoupdate
9c50420 
updates:
- [github.com/psf/black: 23.3.0 → 23.7.0](psf/black@23.3.0...23.7.0)
- [github.com/pycqa/flake8: 6.0.0 → 6.1.0](PyCQA/flake8@6.0.0...6.1.0)
@PGijsbers
Raise correct TypeError and improve type check
2623152
@PGijsbers
Type check with isinstance instead of type() ==
4b0ec45
@PGijsbers
Merge pull request #1271 from openml/pre-commit-ci-update-config
1d7fbda 
[pre-commit.ci] pre-commit autoupdate
@PGijsbers
Docker enhancement #1277 (#1278)
7e69d04 
* Add multiple build platforms

* Automatically update Dockerhub description

* Launch Python instead of Bash by default

* Change `omlp` directory name to less cryptic `openml`

* Change directory to `openml` for running purpose of running script

For mounted scripts, instructions say to mount them to `/openml`,
so we have to `cd` before invoking `python`.

* Update readme to reflect updates (python by default, rename dirs)

* Add branch/code for doc and test examples as they are required

* Ship docker images with readme

* Only update readme on release, also try build docker on PR

* Update the toc descriptions
@varshneydevansh @pre-commit-ci
@PGijsbers @LennartPurucker
fix: carefully replaced minio_url with parquet_url (#1280)
e1ecfe9 
* carefully replaced minio with parquet

* fix: corrected some mistakes

* fix: restored the instances of minio

* fix: updated the documentation

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add #1280

I used a `next` header instead of a specific version since we don't know if it will be 0.15.0 or 0.14.2. We can change it before the next release.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Pieter Gijsbers <[email protected]>
Co-authored-by: Lennart Purucker <[email protected]>
@PGijsbers
Pytest/utils (#1269)
5f5424a 
* Extract mocked_perform_api_call because its independent of object

* Remove _multiprocess_can_split_ as it is a nose directive

and we use pytest

* Convert test list all

* Add markers and refactor test_list_all_for_tasks for pytest

* Add cache marker

* Converted remainder of tests to pytest
@v-parmar @LennartPurucker @pre-commit-ci
Documented remaining Attributes of classes and functions (#1283)
d45cf37 
Add documentation and type hints for the remaining attributes of classes and functions.

---------

Co-authored-by: Lennart Purucker <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@pre-commit-ci
[pre-commit.ci] pre-commit autoupdate (#1281)
7d6a5f4 
updates:
- [github.com/psf/black: 23.7.0 → 23.11.0](psf/black@23.7.0...23.11.0)
- [github.com/pre-commit/mirrors-mypy: v1.4.1 → v1.7.0](pre-commit/mirrors-mypy@v1.4.1...v1.7.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@pre-commit-ci
[pre-commit.ci] pre-commit autoupdate (#1291)
3b5ba6a 
updates:
- [github.com/psf/black: 23.11.0 → 23.12.1](psf/black@23.11.0...23.12.1)
- [github.com/pre-commit/mirrors-mypy: v1.7.0 → v1.8.0](pre-commit/mirrors-mypy@v1.7.0...v1.8.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@dependabot
Bump actions/checkout from 3 to 4 (#1284)
3c39d75 
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
Bump docker/login-action from 2 to 3 (#1285)
540bd63 
Bumps [docker/login-action](https://github.com/docker/login-action) from 2 to 3.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](docker/login-action@v2...v3)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
Bump docker/metadata-action from 4 to 5 (#1286)
884d998 
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 4 to 5.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Upgrade guide](https://github.com/docker/metadata-action/blob/master/UPGRADE.md)
- [Commits](docker/metadata-action@v4...v5)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
Bump docker/setup-qemu-action from 2 to 3 (#1287)
8e1673d 
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 2 to 3.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](docker/setup-qemu-action@v2...v3)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
Bump docker/build-push-action from 4 to 5 (#1288)
d6283e8 
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 4 to 5.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](docker/build-push-action@v4...v5)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@mfeurer @LennartPurucker
Add more type annotations (#1261)
a7bcf07 
* Add more type annotations

* add to progress rst

---------

Co-authored-by: Lennart Purucker <[email protected]>
@dependabot
Bump docker/setup-buildx-action from 2 to 3 (#1292)
35a0fc9 
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 2 to 3.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](docker/setup-buildx-action@v2...v3)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
Bump actions/setup-python from 4 to 5 (#1293)
0a74d9e 
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](actions/setup-python@v4...v5)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@LennartPurucker
Rework Tagging Tests for New Server Specification (#1294)
56895c2 
* rework tagging test adjusted for new server specification

* update progress.rst
@eddiebergman @LennartPurucker
ci: Update tooling (#1298)
783f7cd 
* ci: Migrate everything to pyproject.toml

* style: Apply ruff fixes

* style: Apply (more) ruff fixes

* style(ruff): Add some file specific ignores

* ci: Fix build with setuptools

* ci(ruff): Allow prints in cli.py

* fix: Circular import

* test: Use raises(..., match=)

* fix(cli): re-add missing print statements

* fix: Fix some over ruff-ized code

* add make check to documentation

* ci: Change to `build` for sdist

* ci: Add ruff to `"test"` deps

---------

Co-authored-by: Lennart Purucker <[email protected]>
@eddiebergman
ci: Disable 3.6 tests (#1302)
e435706
@eddiebergman @LennartPurucker @pre-commit-ci
fix: Chipping away at ruff lints (#1303)
43c66aa 
* fix: Chipping away at ruff lints

* fix: return lockfile path

* Update openml/config.py

* Update openml/runs/functions.py

* Update openml/tasks/functions.py

* Update openml/tasks/split.py

* Update openml/utils.py

* Update openml/utils.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update openml/config.py

* Update openml/testing.py

* Update openml/utils.py

* Update openml/config.py

* Update openml/utils.py

* Update openml/utils.py

* add concurrency to workflow calls

* adjust docstring

* adjust docstring

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Lennart Purucker <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lennart Purucker <[email protected]>
@janvanrijn
Tagging constraints (#1305)
97ec49e 
* update tagging constraints

* openml python tests

* small fix
@pre-commit-ci
[pre-commit.ci] pre-commit autoupdate (#1306)
1c660fb 
updates:
- https://github.com/charliermarsh/ruff-pre-commit → https://github.com/astral-sh/ruff-pre-commit
- [github.com/astral-sh/ruff-pre-commit: v0.1.5 → v0.1.11](astral-sh/ruff-pre-commit@v0.1.5...v0.1.11)
- [github.com/python-jsonschema/check-jsonschema: 0.27.1 → 0.27.3](python-jsonschema/check-jsonschema@0.27.1...0.27.3)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@LennartPurucker @eddiebergman
@pre-commit-ci @Bilgecelik
Linting Everything - Fix All mypy and ruff Errors (#1307)
6433d5b 
* style: Fix linting split.py

* typing: Fix mypy errors split.py

* typing: data_feature

* typing: trace

* more linting fixes

* typing: finish up trace

* typing: config.py

* typing: More fixes on config.py

* typing: setup.py

* finalize runs linting

* typing: evaluation.py

* typing: setup

* ruff fixes across different files and mypy fixes for run files

* typing: _api_calls

* adjust setup files' linting and minor ruff changes

* typing: utils

* late night push

* typing: utils.py

* typing: tip tap tippity

* typing: mypy 78, ruff ~200

* refactor output format name and minor linting stuff

* other: midway merge

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* typing: I'm runnign out of good messages

* typing: datasets

* leinting for flows and some ruff changes

* no more mypy errors

* ruff runs and setups

* typing: Finish off mypy and ruff errors

Co-authored-by: Bilgecelik <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* style: File wide ignores of PLR0913

This is because the automated pre-commit.ci bot which made automatic
commits and pushes would think the `noqa` on the individualy overloaded
functions was not needed. After removing the `noqa`, the linter then
raised the issue

---------

Co-authored-by: eddiebergman <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Bilgecelik <[email protected]>
@eddiebergman
ci: Remove Python 3.6/7 (#1308)
836d56b
@LennartPurucker
ci: remove 3.7 patch (#1309)
326bf0b
@LennartPurucker @eddiebergman
Make Test Work Again After Ruff and Linter Changes (#1310)
b06ecee 
* mark production tests

* make production test run

* fix test bug -1/N

* add retry raise again after refactor

* fix str dict representation

* test: Fix non-writable home mocks

* testing: not not a change

* testing: trigger CI

* typing: Update typing

* ci: Update testing matrix

* testing: Fixup run flow error check

* ci: Manual dispatch, disable double testing

* ci: Prevent further ci duplication

* ci: Add concurrency checks to all

* ci: Remove the max-parallel on test ci

There are a lot less now and they cancel previous
puhes in the same pr now so it shouldn't be a problem anymore

* testing: Fix windows path generation

* add pytest for server state

* add assert cache state

* some formatting

* fix with cache fixture

* finally remove th finally

* doc: Fix link

* update test matrix

* doc: Update to just point to contributing

* add linkcheck ignore for test server

---------

Co-authored-by: eddiebergman <[email protected]>
@LennartPurucker @janvanrijn
Add Feature Descriptions Rebase Clean (#1316)
8665b34 
Co-authored-by: Jan van Rijn <[email protected]>
@LennartPurucker @eddiebergman
Make Class Label Retrieval More Lenient (#1315)
b22a5e3 
* mark production tests

* make production test run

* fix test bug -1/N

* add retry raise again after refactor

* fix str dict representation

* test: Fix non-writable home mocks

* testing: not not a change

* testing: trigger CI

* typing: Update typing

* ci: Update testing matrix

* testing: Fixup run flow error check

* ci: Manual dispatch, disable double testing

* ci: Prevent further ci duplication

* ci: Add concurrency checks to all

* ci: Remove the max-parallel on test ci

There are a lot less now and they cancel previous
puhes in the same pr now so it shouldn't be a problem anymore

* testing: Fix windows path generation

* add pytest for server state

* add assert cache state

* some formatting

* fix with cache fixture

* finally remove th finally

* doc: Fix link

* update test matrix

* doc: Update to just point to contributing

* add linkcheck ignore for test server

* add special case for class labels that are dtype string

* fix bug and add test

* formatting

---------

Co-authored-by: eddiebergman <[email protected]>
pre-commit-ci bot and others added 5 commits January 15, 2024 20:10
@pre-commit-ci
[pre-commit.ci] pre-commit autoupdate (#1318)
decc7a8 
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.1.11 → v0.1.13](astral-sh/ruff-pre-commit@v0.1.11...v0.1.13)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@eddiebergman @PGijsbers
Fix: update fetching a bucket from MinIO (#1314)
a1cb66b 
* Update fetching a bucket from MinIO

Previously, each dataset had their own bucket:
  https://openml1.win.tue.nl/datasets61/dataset_61.pq

But we were advised to reduce the amount of buckets and
favor hosting many objects in hierarchical structure, so
we now have instead some prefixes to divide up the
dataset objects into separate subdirectories:

  https://openml1.win.tue.nl/datasets/0000/0061/dataset_61.pq

This commit has bypassed pre-commit. Tests should be
updated too.

* ci: Trigger ci

* ci: Add some files to .gitignore

---------

Co-authored-by: PGijsbers <[email protected]>
@LennartPurucker
Update progress.rst for minor release (#1319)
51d1135 
* Update progress.rst for minor release

* update version number

* fix typo
@LennartPurucker @PGijsbers
Prepare Develop for Merge with Main (#1321)
c35a5d5 
Co-authored-by: Pieter Gijsbers <[email protected]>
@eddiebergman @PGijsbers
@pre-commit-ci @varshneydevansh @LennartPurucker @v-parmar @dependabot @mfeurer @janvanrijn @Bilgecelik
build: Rebase dev onto main (#1322)
449f2cb 
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/psf/black: 23.3.0 → 23.7.0](psf/black@23.3.0...23.7.0)
- [github.com/pycqa/flake8: 6.0.0 → 6.1.0](PyCQA/flake8@6.0.0...6.1.0)

* Raise correct TypeError and improve type check

* Type check with isinstance instead of type() ==

* Docker enhancement #1277 (#1278)

* Add multiple build platforms

* Automatically update Dockerhub description

* Launch Python instead of Bash by default

* Change `omlp` directory name to less cryptic `openml`

* Change directory to `openml` for running purpose of running script

For mounted scripts, instructions say to mount them to `/openml`,
so we have to `cd` before invoking `python`.

* Update readme to reflect updates (python by default, rename dirs)

* Add branch/code for doc and test examples as they are required

* Ship docker images with readme

* Only update readme on release, also try build docker on PR

* Update the toc descriptions

* fix: carefully replaced minio_url with parquet_url (#1280)

* carefully replaced minio with parquet

* fix: corrected some mistakes

* fix: restored the instances of minio

* fix: updated the documentation

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add #1280

I used a `next` header instead of a specific version since we don't know if it will be 0.15.0 or 0.14.2. We can change it before the next release.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Pieter Gijsbers <[email protected]>
Co-authored-by: Lennart Purucker <[email protected]>

* Pytest/utils (#1269)

* Extract mocked_perform_api_call because its independent of object

* Remove _multiprocess_can_split_ as it is a nose directive

and we use pytest

* Convert test list all

* Add markers and refactor test_list_all_for_tasks for pytest

* Add cache marker

* Converted remainder of tests to pytest

* Documented remaining Attributes of classes and functions (#1283)

Add documentation and type hints for the remaining attributes of classes and functions.

---------

Co-authored-by: Lennart Purucker <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* [pre-commit.ci] pre-commit autoupdate (#1281)

updates:
- [github.com/psf/black: 23.7.0 → 23.11.0](psf/black@23.7.0...23.11.0)
- [github.com/pre-commit/mirrors-mypy: v1.4.1 → v1.7.0](pre-commit/mirrors-mypy@v1.4.1...v1.7.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* [pre-commit.ci] pre-commit autoupdate (#1291)

updates:
- [github.com/psf/black: 23.11.0 → 23.12.1](psf/black@23.11.0...23.12.1)
- [github.com/pre-commit/mirrors-mypy: v1.7.0 → v1.8.0](pre-commit/mirrors-mypy@v1.7.0...v1.8.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Bump actions/checkout from 3 to 4 (#1284)

Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump docker/login-action from 2 to 3 (#1285)

Bumps [docker/login-action](https://github.com/docker/login-action) from 2 to 3.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](docker/login-action@v2...v3)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump docker/metadata-action from 4 to 5 (#1286)

Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 4 to 5.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Upgrade guide](https://github.com/docker/metadata-action/blob/master/UPGRADE.md)
- [Commits](docker/metadata-action@v4...v5)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump docker/setup-qemu-action from 2 to 3 (#1287)

Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 2 to 3.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](docker/setup-qemu-action@v2...v3)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump docker/build-push-action from 4 to 5 (#1288)

Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 4 to 5.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](docker/build-push-action@v4...v5)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add more type annotations (#1261)

* Add more type annotations

* add to progress rst

---------

Co-authored-by: Lennart Purucker <[email protected]>

* Bump docker/setup-buildx-action from 2 to 3 (#1292)

Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 2 to 3.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](docker/setup-buildx-action@v2...v3)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/setup-python from 4 to 5 (#1293)

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](actions/setup-python@v4...v5)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Rework Tagging Tests for New Server Specification  (#1294)

* rework tagging test adjusted for new server specification

* update progress.rst

* ci: Update tooling (#1298)

* ci: Migrate everything to pyproject.toml

* style: Apply ruff fixes

* style: Apply (more) ruff fixes

* style(ruff): Add some file specific ignores

* ci: Fix build with setuptools

* ci(ruff): Allow prints in cli.py

* fix: Circular import

* test: Use raises(..., match=)

* fix(cli): re-add missing print statements

* fix: Fix some over ruff-ized code

* add make check to documentation

* ci: Change to `build` for sdist

* ci: Add ruff to `"test"` deps

---------

Co-authored-by: Lennart Purucker <[email protected]>

* ci: Disable 3.6 tests (#1302)

* fix: Chipping away at ruff lints (#1303)

* fix: Chipping away at ruff lints

* fix: return lockfile path

* Update openml/config.py

* Update openml/runs/functions.py

* Update openml/tasks/functions.py

* Update openml/tasks/split.py

* Update openml/utils.py

* Update openml/utils.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update openml/config.py

* Update openml/testing.py

* Update openml/utils.py

* Update openml/config.py

* Update openml/utils.py

* Update openml/utils.py

* add concurrency to workflow calls

* adjust docstring

* adjust docstring

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Lennart Purucker <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lennart Purucker <[email protected]>

* Tagging constraints (#1305)

* update tagging constraints

* openml python tests

* small fix

* [pre-commit.ci] pre-commit autoupdate (#1306)

updates:
- https://github.com/charliermarsh/ruff-pre-commit → https://github.com/astral-sh/ruff-pre-commit
- [github.com/astral-sh/ruff-pre-commit: v0.1.5 → v0.1.11](astral-sh/ruff-pre-commit@v0.1.5...v0.1.11)
- [github.com/python-jsonschema/check-jsonschema: 0.27.1 → 0.27.3](python-jsonschema/check-jsonschema@0.27.1...0.27.3)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Linting Everything - Fix All mypy and ruff Errors (#1307)

* style: Fix linting split.py

* typing: Fix mypy errors split.py

* typing: data_feature

* typing: trace

* more linting fixes

* typing: finish up trace

* typing: config.py

* typing: More fixes on config.py

* typing: setup.py

* finalize runs linting

* typing: evaluation.py

* typing: setup

* ruff fixes across different files and mypy fixes for run files

* typing: _api_calls

* adjust setup files' linting and minor ruff changes

* typing: utils

* late night push

* typing: utils.py

* typing: tip tap tippity

* typing: mypy 78, ruff ~200

* refactor output format name and minor linting stuff

* other: midway merge

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* typing: I'm runnign out of good messages

* typing: datasets

* leinting for flows and some ruff changes

* no more mypy errors

* ruff runs and setups

* typing: Finish off mypy and ruff errors

Co-authored-by: Bilgecelik <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* style: File wide ignores of PLR0913

This is because the automated pre-commit.ci bot which made automatic
commits and pushes would think the `noqa` on the individualy overloaded
functions was not needed. After removing the `noqa`, the linter then
raised the issue

---------

Co-authored-by: eddiebergman <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Bilgecelik <[email protected]>

* ci: Remove Python 3.6/7 (#1308)

* ci: remove 3.7 patch (#1309)

* Make Test Work Again After Ruff and Linter Changes (#1310)

* mark production tests

* make production test run

* fix test bug -1/N

* add retry raise again after refactor

* fix str dict representation

* test: Fix non-writable home mocks

* testing: not not a change

* testing: trigger CI

* typing: Update typing

* ci: Update testing matrix

* testing: Fixup run flow error check

* ci: Manual dispatch, disable double testing

* ci: Prevent further ci duplication

* ci: Add concurrency checks to all

* ci: Remove the max-parallel on test ci

There are a lot less now and they cancel previous
puhes in the same pr now so it shouldn't be a problem anymore

* testing: Fix windows path generation

* add pytest for server state

* add assert cache state

* some formatting

* fix with cache fixture

* finally remove th finally

* doc: Fix link

* update test matrix

* doc: Update to just point to contributing

* add linkcheck ignore for test server

---------

Co-authored-by: eddiebergman <[email protected]>

* Add Feature Descriptions Rebase Clean (#1316)

Co-authored-by: Jan van Rijn <[email protected]>

* Make Class Label Retrieval More Lenient (#1315)

* mark production tests

* make production test run

* fix test bug -1/N

* add retry raise again after refactor

* fix str dict representation

* test: Fix non-writable home mocks

* testing: not not a change

* testing: trigger CI

* typing: Update typing

* ci: Update testing matrix

* testing: Fixup run flow error check

* ci: Manual dispatch, disable double testing

* ci: Prevent further ci duplication

* ci: Add concurrency checks to all

* ci: Remove the max-parallel on test ci

There are a lot less now and they cancel previous
puhes in the same pr now so it shouldn't be a problem anymore

* testing: Fix windows path generation

* add pytest for server state

* add assert cache state

* some formatting

* fix with cache fixture

* finally remove th finally

* doc: Fix link

* update test matrix

* doc: Update to just point to contributing

* add linkcheck ignore for test server

* add special case for class labels that are dtype string

* fix bug and add test

* formatting

---------

Co-authored-by: eddiebergman <[email protected]>

* [pre-commit.ci] pre-commit autoupdate (#1318)

* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.1.11 → v0.1.13](astral-sh/ruff-pre-commit@v0.1.11...v0.1.13)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Fix: update fetching a bucket from MinIO (#1314)

* Update fetching a bucket from MinIO

Previously, each dataset had their own bucket:
  https://openml1.win.tue.nl/datasets61/dataset_61.pq

But we were advised to reduce the amount of buckets and
favor hosting many objects in hierarchical structure, so
we now have instead some prefixes to divide up the
dataset objects into separate subdirectories:

  https://openml1.win.tue.nl/datasets/0000/0061/dataset_61.pq

This commit has bypassed pre-commit. Tests should be
updated too.

* ci: Trigger ci

* ci: Add some files to .gitignore

---------

Co-authored-by: PGijsbers <[email protected]>

* Update progress.rst for minor release (#1319)

* Update progress.rst for minor release

* update version number

* fix typo

* Prepare Develop for Merge with Main (#1321)

Co-authored-by: Pieter Gijsbers <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Pieter Gijsbers <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Devansh Varshney (देवांश वार्ष्णेय) <[email protected]>
Co-authored-by: Lennart Purucker <[email protected]>
Co-authored-by: Vishal Parmar <[email protected]>
Co-authored-by: Lennart Purucker <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matthias Feurer <[email protected]>
Co-authored-by: Lennart Purucker <[email protected]>
Co-authored-by: janvanrijn <[email protected]>
Co-authored-by: Lennart Purucker <[email protected]>
Co-authored-by: Bilgecelik <[email protected]>
@PGijsbers
Copy link
Collaborator

There are some problems with the history between main and develop. We can clean this up (for example, by branching off main again), but Lennart proposed we simply abandon the two branch system in favor of using tags for releases.
I personally also would prefer this option, as it's less error prone. We could keep the main branch around for archival reasons. The documentation pages would need an update.

@LennartPurucker LennartPurucker added this to the 0.15 milestone Jan 18, 2024
@codecov-commenter
Copy link

codecov-commenter commented Jan 18, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 84.18246% with 267 lines in your changes missing coverage. Please review.

Project coverage is 83.68%. Comparing base (d26a911) to head (dea8724).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
openml/utils.py 75.23% 26 Missing ⚠️
openml/datasets/dataset.py 86.70% 25 Missing ⚠️
openml/_api_calls.py 73.91% 24 Missing ⚠️
openml/cli.py 0.00% 18 Missing ⚠️
openml/datasets/functions.py 87.14% 18 Missing ⚠️
openml/tasks/task.py 78.20% 17 Missing ⚠️
openml/study/functions.py 72.88% 16 Missing ⚠️
openml/testing.py 76.19% 15 Missing ⚠️
openml/config.py 83.72% 14 Missing ⚠️
openml/extensions/sklearn/extension.py 91.13% 14 Missing ⚠️
... and 16 more
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1323      +/-   ##
==========================================
- Coverage   85.19%   83.68%   -1.51%     
==========================================
  Files          38       38              
  Lines        5107     5297     +190     
==========================================
+ Hits         4351     4433      +82     
- Misses        756      864     +108

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

pre-commit-ci bot and others added 14 commits January 22, 2024 19:43
@pre-commit-ci
[pre-commit.ci] pre-commit autoupdate (#1325)
90380d4 
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.1.13 → v0.1.14](astral-sh/ruff-pre-commit@v0.1.13...v0.1.14)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@BrunoBelucci @PGijsbers
read file in read mode (#1338)
923b49d 
* read file in read mode

* cast parameters to expected types

* Following PGijsbers proposal to ensure that avoid_duplicate_runs is a boolean after reading it from config_file

* Add a test, move parsing of avoid_duplicate_runs

---------

Co-authored-by: PGijsbers <[email protected]>
@PGijsbers
Fix/sklearn test compatibility (#1340)
532be7b 
* Update 'sparse' parameter for OHE for sklearn >= 1.4

* Add compatability or skips for sklearn >= 1.4

* Change 'auto' to 'sqrt' for sklearn>1.3 as 'auto' is deprecated

* Skip flaky test

It is unclear how a condition where the test is supposed to pass
is created. Even after running the test suite 2-3 times, it does
not yet seem to pass.

* Fix typo

* Ignore description comparison for newer scikit-learn

There are some minor changes to the docstrings. I do not know
that it is useful to keep testing it this way, so for now I will
disable the test on newer versions.

* Adjust for scikit-learn 1.3

The loss has been renamed.
The performance of the model also seems to have changed slightly
for the same seed. So I decided to compare with the lower fidelity
that was already used on Windows systems.

* Remove timeout and reruns to better investigate CI failures

* Fix typo in parametername

* Add jobs for more recent scikit-learns

* Expand the matrix with all scikit-learn 1.x versions

* Fix for numpy2.0 compatibility (#1341)

Numpy2.0 cleaned up their namespace.

* Rewrite matrix and update numpy compatibility

* Move comment in-line

* Stringify name of new step to see if that prevented the action

* Fix unspecified os for included jobs

* Fix typo in version pinning for numpy

* Fix version specification for sklearn skips

* Output final list of installed packages for debugging purposes

* Cap scipy version for older versions of scikit-learn

There is a breaking change to the way 'mode' works, that
breaks scikit-learn internals.

* Update parameter base_estimator to estimator for sklearn>=1.4

* Account for changes to sklearn interface in 1.4 and 1.5

* Non-strict reinstantiation requires different scikit-learn version

* Parameters were already changed in 1.4

* Fix race condition (I think)

It seems to me that run.evaluations is set only when the run is
fetched. Whether it has evaluations depends on server state.
So if the server has resolved the traces between the initial
fetch and the trace-check, you could be checking
len(run.evaluations) where evaluations is None.

* Use latest patch version of each minor release

* Convert numpy types back to builtin types

Scikit-learn or numpy changed the typing of the parameters
(seen in a masked array, not sure if also outside of that).
Convert these values back to Python builtins.

* Specify versions with * instead to allow for specific patches

* Flow_exists does not return None but False is the flow does not exist

* Update new version definitions also installation step

* Fix bug introduced in refactoring for np.generic support

We don't want to serialize as the value np.nan, we want to
include the nan directly. It is an indication that the
parameter was left unset.

* Add back the single-test timeout of 600s

* [skip ci] Add note to changelog

* Check that evaluations are present with None-check instead

The default behavior if no evaluation is present is for it
to be None. So it makes sense to check for that instead.
As far as I can tell, run.evaluations should always contain
some items if it is not None. But I added an assert just in
case.

* Remove timeouts again

I suspect they "crash" workers. This of course introduces the risk of hanging processes... But I cannot reproduce the issue locally.
@PGijsbers
Add HTTP headers to all requests (#1342)
e4e6f50 
* Add HTTP headers to all requests

This allows us to better understand the traffic we see to our API.
It is not identifiable to a person.

* Update unit test to pass even with user-agent in header
@PGijsbers
Superseded by pre-commit.ci (#1343)
de983ac
@PGijsbers
Fix/1349 (#1350)
fa7e9db 
* Add packaging dependency

* Change use of distutils to packaging

* Update missed usage of distutils to packaging

* Inline comparison to clear up confusion
@PGijsbers
Lazy arff (#1346)
b4d038f 
* Prefer parquet over arff, do not load arff if not needed

* Only download arff if needed

* Test arff file is not set when downloading parquet from prod
@PGijsbers
Feat/progress (#1335)
1d707e6 
* Add progress bar to downloading minio files

* Do not redownload cached files

There is now a way to force a cache clear, so always redownloading
is not useful anymore.

* Set typed values on dictionary to avoid TypeError from Config

* Add regression test for parsing booleans
@PGijsbers
Add/1034 (#1352) dataset lazy loading default
07e9b9c 
* Towards lazy-by-default for dataset loading

* Isolate lazy behavior to pytest function outside of class

* Solve concurrency issue where test would use same cache

* Ensure metadata is downloaded to verify dataset is processed

* Clean up to reflect new defaults and tests

* Fix oversight from 1335

* Download data as was 0.14 behavior

* Restore test

* Formatting

* Test obsolete, replaced by test_get_dataset_lazy_behavior
@PGijsbers
Make test insensitive to OrderedDict stringification (#1353)
7764ddb 
Sometime between 3.9 and 3.12 the stringification of
ordered dicts changed from using a list of tuples to
a dictionary.
@PGijsbers
Remove archive after it is extracted to save disk space (#1351)
a3e57bb 
* Remove archive after it is extracted to save disk space

* Leave a marker after removing archive to avoid redownload

* Automatic refresh if expected marker is absent

* Be consistent about syntax use for path construction
@knyazer @PGijsbers
Pass kwargs through task to get_dataset (#1345)
d37542b 
* Pass kwargs through task to ```get_dataset```

Allows to follow the directions in the warning ```Starting from Version 0.15 `download_data`, `download_qualities`, and `download_features_meta_data` will all be ``False`` instead of ``True`` by default to enable lazy loading.```

* docs: explain that ```task.get_dataset``` passes kwargs

* Update openml/tasks/task.py

Remove Py3.8+ feature for backwards compatibility

---------

Co-authored-by: Pieter Gijsbers <[email protected]>
@PGijsbers @pre-commit-ci
Change defaults for get_task to be lazy (#1354)
a55a3fc 
* Change defaults for `get_task`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix linting errors

* Add missing type annotation

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@PGijsbers
Release/0.15.0 (#1355)
dea8724 
* Expand 0.15.0 changelog with other PRs not yet added

* Bump version number

* Add newer Python versions since we are compatible

* Revert "Add newer Python versions since we are compatible"

This reverts commit 5088c80.

* Add newer compatible versions of Python
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Documentation enhancement priority
Projects
None yet
Milestone
0.15
Development

Successfully merging this pull request may close these issues.
10 participants
@LennartPurucker @PGijsbers @codecov-commenter @varshneydevansh @v-parmar @mfeurer @eddiebergman @janvanrijn @BrunoBelucci @knyazer