Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bitbucket Pipelines — GPU support #1015

Open
0x2b3bfa0 opened this issue May 24, 2022 · 2 comments
Open

Bitbucket Pipelines — GPU support #1015

0x2b3bfa0 opened this issue May 24, 2022 · 2 comments
Assignees
@dacbd @casperdcl
Labels
ci-bitbucket cml-runner Subcommand enhancement New feature or request external-request You asked, we did gpu Inexplicably convoluted drivers p2-nice-to-have Low priority technical-debt Refactoring, linting & tidying

Comments

@0x2b3bfa0
Copy link
Member

0x2b3bfa0 commented May 24, 2022 •
edited
Loading

Continuation of #798 (review)

As per the official documentation, self-hosted Bitbucket Pipelines runners for Linux don't “provide access to the host file system” although that limitation is “subject to change in the future”

Linux-only

Can’t access the local files on the runner host from a build script: In order to provide isolation and reproducibility, a runner doesn’t provide access directly to the host file system, we recommend using SCP / Rsync, if necessary.

The limits described above are subject to change in the future.

In practice, this means that every single step is containerized, and users don't have any control over container options like docker run --gpus, even when the NVIDIA runtime is installed.
@0x2b3bfa0 0x2b3bfa0 changed the title Bitbucket Pipelines cml runner follow-up Bitbucket Pipelines — GPU support May 24, 2022
@0x2b3bfa0 0x2b3bfa0 added enhancement New feature or request cml-runner Subcommand ci-bitbucket technical-debt Refactoring, linting & tidying labels May 24, 2022
@casperdcl casperdcl added the blocked Dependent on something else label May 24, 2022
@0x2b3bfa0
Copy link
Member Author

Related

@0x2b3bfa0 0x2b3bfa0 mentioned this issue May 25, 2022
Merged
@RCdeWit RCdeWit mentioned this issue Jul 25, 2022
Closed
@casperdcl
Copy link
Contributor

casperdcl commented Jul 29, 2022 •
edited
Loading

potential work-around: use (TP)I task (BB Pipelines -> TPI, exporting/forwarding BB env vars -> AWS EC2 -> user script && CML report). Likely depends on resolving iterative/terraform-provider-iterative#630.

moved to iterative/terraform-provider-iterative#633

@casperdcl casperdcl added external-request You asked, we did and removed blocked Dependent on something else labels Jul 29, 2022
@casperdcl casperdcl mentioned this issue Aug 2, 2022
Open
@casperdcl casperdcl added the gpu Inexplicably convoluted drivers label Aug 3, 2022
@casperdcl casperdcl self-assigned this Nov 29, 2022
@casperdcl casperdcl mentioned this issue Nov 29, 2022
Open
13 tasks
casperdcl added a commit to iterative/cml.dev that referenced this issue Dec 1, 2022
@casperdcl
bitbucket: add GPU examples
620eceb 
- fixes iterative/cml#1015
- fixes #349
@casperdcl casperdcl mentioned this issue Dec 1, 2022
Open
5 tasks
@dacbd dacbd added the p2-nice-to-have Low priority label Feb 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dacbd dacbd

@casperdcl casperdcl

Labels
ci-bitbucket cml-runner Subcommand enhancement New feature or request external-request You asked, we did gpu Inexplicably convoluted drivers p2-nice-to-have Low priority technical-debt Refactoring, linting & tidying
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dacbd @casperdcl @0x2b3bfa0