Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial implementation of python modules with poetry #343

Merged
merged 25 commits into from
Mar 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
3748711
Initial implementation of python modules with poetry
layus Mar 8, 2023
96f62d5
Merge python-poetry tests into python tests
layus Mar 10, 2023
ec25c60
Improve nixpkgs_python_repository documentation
layus Mar 10, 2023
5e5ee28
Move large chunks of text to their own file.
layus Mar 10, 2023
5324f76
Explain and improve package_set_to_json.nix
layus Mar 10, 2023
bc18feb
Document and update python_package.bzl
layus Mar 10, 2023
c46ac5f
Generate documentation for nixpkgs_python_repository
layus Mar 10, 2023
5a2cbf6
Fixes for bzlmod
layus Mar 10, 2023
1956554
Use an older numpy for macos
layus Mar 10, 2023
317359f
Fix poetry macos tests
layus Mar 12, 2023
4f23a38
Add a test for python dependencies
layus Mar 12, 2023
323341a
Add mach-nix example for python repository
layus Mar 12, 2023
1fbe84d
Fix mach-nix repository fetching and improve usage
layus Mar 12, 2023
cd837fb
Work around some builtins.fetchGit bug
layus Mar 12, 2023
08d3678
Try to make mach-nix work on ci
layus Mar 12, 2023
f659bf6
Use a simpler python package for import tests
layus Mar 13, 2023
b82342f
Revert "Try to make mach-nix work on ci"
layus Mar 13, 2023
f2af843
Another attempt at mach-nix
layus Mar 13, 2023
a29651b
WIP: mach-nix-debug
layus Mar 13, 2023
562695a
WIP: hopefully fix mach-nix for macos
layus Mar 13, 2023
5ebc68d
Add vanilla nixpkgs python package set
layus Mar 13, 2023
8609a4d
Remove broken mach-nix example
layus Mar 13, 2023
e816bfd
Remove leftover debug data
layus Mar 13, 2023
6e10291
Improve docs as per review comments
layus Mar 14, 2023
02a66d2
Merge branch 'master' into poetry-python-modules
mergify[bot] Mar 14, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .bazelrc.common
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ build --host_platform=@rules_nixpkgs_core//platforms:host
build:bzlmod --enable_bzlmod
build:bzlmod --registry=https://bcr.bazel.build

test --test_output=errors

# User Configuration
# ------------------
try-import %workspace%/.bazelrc.local
Expand Down
146 changes: 146 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ See [examples](/examples/toolchains) for how to use `rules_nixpkgs` with differe
* [nixpkgs_cc_configure_deprecated](#nixpkgs_cc_configure_deprecated)
* [nixpkgs_java_configure](#nixpkgs_java_configure)
* [nixpkgs_python_configure](#nixpkgs_python_configure)
* [nixpkgs_python_repository](#nixpkgs_python_repository)
* [nixpkgs_go_configure](toolchains/go/README.md#nixpkgs_go_configure)
* [nixpkgs_rust_configure](#nixpkgs_rust_configure)
* [nixpkgs_sh_posix_configure](#nixpkgs_sh_posix_configure)
Expand Down Expand Up @@ -1836,6 +1837,151 @@ default is <code>True</code>
</table>


<a id="#nixpkgs_python_repository"></a>

### nixpkgs_python_repository

<pre>
nixpkgs_python_repository(<a href="#nixpkgs_python_repository-name">name</a>, <a href="#nixpkgs_python_repository-repository">repository</a>, <a href="#nixpkgs_python_repository-repositories">repositories</a>, <a href="#nixpkgs_python_repository-nix_file">nix_file</a>, <a href="#nixpkgs_python_repository-nix_file_deps">nix_file_deps</a>, <a href="#nixpkgs_python_repository-quiet">quiet</a>)
</pre>

Define a collection of python packages based on a nix file.

The only entry point is a [`nix_file`](#nixpkgs_python_repository-nix_file)
which should expose a `pkgs` and a `python` attributes. `python` is the
python interpreter, and `pkgs` a set of python packages that will be made
available to bazel.

:warning: All the packages in `pkgs` are built by this rule. It is
therefore not a good idea to expose something as big as `pkgs.python3` as
provided by nixpkgs.

This rule is instead intended to expose an ad-hoc set of packages for your
project, as can be built by poetry2nix, mach-nix, dream2nix or by manually
picking the python packages you need from nixpkgs.

The format is generic to support the many ways to generate such packages
sets with nixpkgs. See our python [`tests`](/testing/toolchains/python) and
[examples](`/examples/toolchains/python`) to get started.

This rule is intended to mimic as closely as possible the [rules_python
API](https://github.com/bazelbuild/rules_python#using-the-package-installation-rules).
`nixpkgs_python_repository` should be a drop-in replacement of `pip_parse`.
As such, it also provides a `requirement` function.

:warning: Using the `requirement` fucntion inherits the same advantages and
limitations as the one in rules_python. All the function does is create a
label of the form `@{nixpkgs_python_repository_name}//:{package_name}`.
While depending on such a label directly will work, the layout may change
in the future. To be on the safe side, define and import your own
`requirement` function if you need to play with these labels.

:warning: Just as with rules_python, nothing is done to enforce consistency
between the version of python used to generate this repository and the one
configured in your toolchain, even if you use nixpkgs_python_toolchain. You
should ensure they both use the same python from the same nixpkgs version.

:warning: packages names exposed by this rule are determined by the `pname`
attribute of the corresponding nix package. These may vary slightly from
names used by rules_python. Should this be a problem, you can provide you
own `requirement` function, for example one that lowercases its argument.


#### Parameters

<table class="params-table">
<colgroup>
<col class="col-param" />
<col class="col-description" />
</colgroup>
<tbody>
<tr id="nixpkgs_python_repository-name">
<td><code>name</code></td>
<td>

required.

<p>

The name for the created package set.

</p>
</td>
</tr>
<tr id="nixpkgs_python_repository-repository">
<td><code>repository</code></td>
<td>

optional.
default is <code>None</code>

<p>

See [`nixpkgs_package`](#nixpkgs_package-repository).

</p>
</td>
</tr>
<tr id="nixpkgs_python_repository-repositories">
<td><code>repositories</code></td>
<td>

optional.
default is <code>{}</code>

<p>

See [`nixpkgs_package`](#nixpkgs_package-repositories).

</p>
</td>
</tr>
<tr id="nixpkgs_python_repository-nix_file">
<td><code>nix_file</code></td>
<td>

optional.
default is <code>None</code>

<p>

See [`nixpkgs_package`](#nixpkgs_package-nix_file).

</p>
</td>
</tr>
<tr id="nixpkgs_python_repository-nix_file_deps">
<td><code>nix_file_deps</code></td>
<td>

optional.
default is <code>[]</code>

<p>

See [`nixpkgs_package`](#nixpkgs_package-nix_file_deps).

</p>
</td>
</tr>
<tr id="nixpkgs_python_repository-quiet">
<td><code>quiet</code></td>
<td>

optional.
default is <code>False</code>

<p>

See [`nixpkgs_package`](#nixpkgs_package-quiet).

</p>
</td>
</tr>
</tbody>
</table>


<a id="#nixpkgs_rust_configure"></a>

### nixpkgs_rust_configure
Expand Down
1 change: 1 addition & 0 deletions docs/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ generate_documentation(
"nixpkgs_go_configure",
"nixpkgs_java_configure",
"nixpkgs_python_configure",
"nixpkgs_python_repository",
"nixpkgs_rust_configure",
"nixpkgs_sh_posix_configure",
"nixpkgs_nodejs_configure",
Expand Down
3 changes: 3 additions & 0 deletions nixpkgs/nixpkgs.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ See [examples](/examples/toolchains) for how to use `rules_nixpkgs` with differe
* [nixpkgs_cc_configure_deprecated](#nixpkgs_cc_configure_deprecated)
* [nixpkgs_java_configure](#nixpkgs_java_configure)
* [nixpkgs_python_configure](#nixpkgs_python_configure)
* [nixpkgs_python_repository](#nixpkgs_python_repository)
* [nixpkgs_go_configure](toolchains/go/README.md#nixpkgs_go_configure)
* [nixpkgs_rust_configure](#nixpkgs_rust_configure)
* [nixpkgs_sh_posix_configure](#nixpkgs_sh_posix_configure)
Expand Down Expand Up @@ -130,6 +131,7 @@ load(
load(
"@rules_nixpkgs_python//:python.bzl",
_nixpkgs_python_configure = "nixpkgs_python_configure",
_nixpkgs_python_repository = "nixpkgs_python_repository",
)
load(
"@rules_nixpkgs_java//:java.bzl",
Expand Down Expand Up @@ -158,6 +160,7 @@ nixpkgs_git_repository = _nixpkgs_git_repository
nixpkgs_local_repository = _nixpkgs_local_repository
nixpkgs_package = _nixpkgs_package
nixpkgs_python_configure = _nixpkgs_python_configure
nixpkgs_python_repository = _nixpkgs_python_repository
nixpkgs_java_configure = _nixpkgs_java_configure
nixpkgs_cc_configure = _nixpkgs_cc_configure
nixpkgs_rust_configure = _nixpkgs_rust_configure
Expand Down
5 changes: 5 additions & 0 deletions testing/python/MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,8 @@ use_repo(non_module_deps, "nixpkgs_python_toolchain")
use_repo(non_module_deps, "nixpkgs_python_configure_test")
use_repo(non_module_deps, "nixpkgs_python_toolchain_python3")
register_toolchains("@nixpkgs_python_toolchain//:all")

use_repo(non_module_deps, "generated_poetry_packages_deps")
use_repo(non_module_deps, "poetry_packages")
use_repo(non_module_deps, "generated_vanilla_packages_deps")
use_repo(non_module_deps, "vanilla_packages")
Loading