compile_pip_requirements does not work when the requirements.in is a result of a rule

[ns-ggeorgiev]

🐞 bug report

Affected Rule

rule compile_pip_requirements

Is this a regression?

no

Description

when I provide requirements.in file that is generated from a rule, compile_pip_requirements fails file not found, because it is looking for the file in the workspace, instead of bazel sanbox area.

🔬 Minimal Reproduction

genrule(
        name = "requirements_in",
        outs = ["requirements.in"],
        cmd = "echo six > \"$@\"",
    )

compile_pip_requirements(
    name = "requirements",
    requirements_in = ":requirements_in",
    requirements_txt = "requirements.txt",
)

🔥 Exception or Error

Updating bzl/requirements/requirements.txt
Usage: pip_compile.py [OPTIONS] [SRC_FILES]...
Try 'pip_compile.py -h' for help.

Error: Invalid value for '[SRC_FILES]...': Path 'bzl/requirements/requirements.in' does not exist.

[rickeylev]

I don't think this is possible. If I understand things correctly, repo rules can't use generated files ( bazelbuild/bazel#4069), and compile pip requirements has to be a repo rule (because it has to run external commands to resolve transitive deps and generate external repos) You might be able to approximate something by using Bazel run to generate the requirements.in file in your source tree, though.

…

On Sat, Oct 15, 2022, 10:29 AM George Georgiev ***@***.***> wrote: 🐞 bug report Affected Rule rule compile_pip_requirements Is this a regression? no Description when I provide requirements.in file that is generated from a rule, compile_pip_requirements fails file not found, because it is looking for the file in the workspace, instead of bazel sanbox area. 🔬 Minimal Reproduction genrule( name = "requirements_in", outs = ["requirements.in"], cmd = "echo six > \"$@\"", ) compile_pip_requirements( name = "requirements", requirements_in = ":requirements_in", requirements_txt = "requirements.txt", ) 🔥 Exception or Error Updating bzl/requirements/requirements.txt Usage: pip_compile.py [OPTIONS] [SRC_FILES]... Try 'pip_compile.py -h' for help. Error: Invalid value for '[SRC_FILES]...': Path 'bzl/requirements/requirements.in' does not exist. — Reply to this email directly, view it on GitHub <#860>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIEXQ6Q62NNUEOYLBHAIGVLWDLSYPANCNFSM6AAAAAARF7W7P4> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[jvolkman]

Is this fixed by #829?

[rickeylev]

Oh, I see. I stand corrected!

[groodt]

Oh, I see. I stand corrected!

Only in a sense. The repository rules for locking dependencies probably need a bit more documentation written around them to list the caveats. There are quite a few problematic areas (relative file paths, non-hermetic, platform specific, coupled to a format that might change and rules don’t control) that I think as maintainers we should have the freedom to change if necessary.

[ns-ggeorgiev]

The way I solved the same was just adding the following just before changing the current directory to the workspace directory

        if not shutil._samefile(requirements_in, os.path.join(os.environ["BUILD_WORKSPACE_DIRECTORY"], requirements_in)):
            requirements_in = os.path.abspath(requirements_in)

but #829 seems to resolves the issue as well.

I am closing this.