-
Notifications
You must be signed in to change notification settings - Fork 280
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
restore sapling getdeps so eden/mononoke can depend on it #950
Conversation
@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
@ahornby has updated the pull request. You must reimport the pull request before landing. |
@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Summary: fix getdeps generate-github-actions byt adding missing process_project_dir_arguments. Without this couldn't specify any project_install_prefix other that the default /usr/local mononoke installs directly to getdeps inst dir so needs a project_install_prefix of / for the artifact discovery to work regenerated the gh actions with `python3 fbcode/opensource/fbcode_builder/getdeps.py generate-github-actions mononoke --os-type=linux --allow-system-packages --output-dir fbcode/eden/oss/.github/workflows --job-file-prefix mononoke_ --job-name-prefix "Mononoke " --free-up-disk --project-install-prefix mononoke:/` Differential Revision: D63031271
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef and badf6d1 Pull Request resolved: facebook#950 Test Plan: Setup a local ubuntu 22.04 toolbox with: ``` toolbox create --distro ubuntu --release 22.04 toolbox enter ubuntu-toolbox-22.04 sudo apt update sudo apt install g++ libssl-dev pkg-config curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ``` Build locally with: `python3 build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. sapling --project-install-prefix sapling:/usr/local` Test locally with: `python3 build/fbcode_builder/getdeps.py --allow-system-packages test --src-dir=. sapling --project-install-prefix sapling:/usr/local` Generate github actions with: `./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --free-up-disk --os-type=linux --src-dir=. --output-dir=.github/workflows --job-name "Sapling CLI Getdeps " --job-file-prefix=sapling-cli-getdeps_ sapling` CI --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/950). * facebook#951 * __->__ facebook#950 Differential Revision: D62978511 Pulled By: ahornby
This pull request was exported from Phabricator. Differential Revision: D62978511 |
Summary: Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef and badf6d1 Pull Request resolved: facebook#950 Test Plan: Setup a local ubuntu 22.04 toolbox with: ``` toolbox create --distro ubuntu --release 22.04 toolbox enter ubuntu-toolbox-22.04 sudo apt update sudo apt install g++ libssl-dev pkg-config curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ``` Build locally with: `python3 build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. sapling --project-install-prefix sapling:/usr/local` Test locally with: `python3 build/fbcode_builder/getdeps.py --allow-system-packages test --src-dir=. sapling --project-install-prefix sapling:/usr/local` Generate github actions with: `./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --free-up-disk --os-type=linux --src-dir=. --output-dir=.github/workflows --job-name "Sapling CLI Getdeps " --job-file-prefix=sapling-cli-getdeps_ sapling` CI --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/950). * facebook#951 * __->__ facebook#950 Differential Revision: D62978511
Summary: X-link: facebookincubator/zstrong#995 Bring back mononoke getdeps integration tests. This is a refesh of the previously working export-D34186407 branch from 2022 Main changes since: * depend on restored sapling manifest from previous commit * bring back selected manifests needed by mononoke that were deleted in D51869247. I added the sqlite binary packages from this to the sqlite3 manifest rather than bring back the sqllite3-bin manifest * add manifests for new tools used: ripgrep, git-lfs, and zstd cli (we already had the zstd libs) * fix a few test expecations that where too closely tied to git cli or TLS version etc (ubuntu 22.04 is on older version) * getdeps MakefileBuilder.run_tests() improvements * fix error status reporting, was not failing if tests failed * pass of --num-jobs to tests to stop it OOMing my machine * pass of --filter to tests so can iterate on one test more easily Can to iterate local execution for one test with: ``` python3 ./build/fbcode_builder/getdeps.py test --num-jobs 4 --allow-system-packages --no-facebook-internal --src-dir=. mononoke_integration --retry 0 --filter server/test-gettreepack.t ``` Pull Request resolved: facebook#951 Test Plan: Setup a local ubuntu 22.04 toolbox with: ``` toolbox create --distro ubuntu --release 22.04 toolbox enter ubuntu-toolbox-22.04 sudo apt update sudo apt install g++ libssl-dev pkg-config sudo locale-gen en_US.UTF-8 curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ``` Build dependencies: ``` python3 ./build/fbcode_builder/getdeps.py install-system-deps --recursive mononoke_integration python3 ./build/fbcode_builder/getdeps.py build --allow-system-packages --no-facebook-internal --src-dir=. --no-tests sapling python3 ./build/fbcode_builder/getdeps.py build --allow-system-packages --no-facebook-internal --src-dir=. --no-tests mononoke ``` then test locally: ``` ./build/fbcode_builder/getdeps.py build --no-deps --num-jobs 16 --allow-system-packages --no-facebook-internal --src-dir=. mononoke_integration ./build/fbcode_builder/getdeps.py test --num-jobs 6 --allow-system-packages --no-facebook-internal --src-dir=. mononoke_integration ``` Generate the github actions yaml ``` ./build/fbcode_builder/getdeps.py generate-github-actions --allow-system-packages --test --os-type linux --job-file-prefix mononoke-integration_ --job-name-prefix "Mononoke Integration " --output-dir .github/workflows mononoke_integration --num-jobs 4 ``` --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/951). * __->__ facebook#951 * facebook#950 Differential Revision: D62978526 Pulled By: ahornby
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef03f2d10e45a36aaa2354b4f722cd71e07 and badf6d159dada727acfb5394bd17de223b336975 X-link: facebook/sapling#950 Reviewed By: quark-zju Differential Revision: D62978511 Pulled By: ahornby fbshipit-source-id: b614e3f10d254283a4904e9e1a6e97870a56b612
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef03f2d10e45a36aaa2354b4f722cd71e07 and badf6d159dada727acfb5394bd17de223b336975 X-link: facebook/sapling#950 Reviewed By: quark-zju Differential Revision: D62978511 Pulled By: ahornby fbshipit-source-id: b614e3f10d254283a4904e9e1a6e97870a56b612
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef03f2d10e45a36aaa2354b4f722cd71e07 and badf6d159dada727acfb5394bd17de223b336975 X-link: facebook/sapling#950 Reviewed By: quark-zju Differential Revision: D62978511 Pulled By: ahornby fbshipit-source-id: b614e3f10d254283a4904e9e1a6e97870a56b612
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef03f2d10e45a36aaa2354b4f722cd71e07 and badf6d159dada727acfb5394bd17de223b336975 X-link: facebook/sapling#950 Reviewed By: quark-zju Differential Revision: D62978511 Pulled By: ahornby fbshipit-source-id: b614e3f10d254283a4904e9e1a6e97870a56b612
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef03f2d10e45a36aaa2354b4f722cd71e07 and badf6d159dada727acfb5394bd17de223b336975 X-link: facebook/sapling#950 Reviewed By: quark-zju Differential Revision: D62978511 Pulled By: ahornby fbshipit-source-id: b614e3f10d254283a4904e9e1a6e97870a56b612
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef03f2d10e45a36aaa2354b4f722cd71e07 and badf6d159dada727acfb5394bd17de223b336975 X-link: facebook/sapling#950 Reviewed By: quark-zju Differential Revision: D62978511 Pulled By: ahornby fbshipit-source-id: b614e3f10d254283a4904e9e1a6e97870a56b612
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef03f2d10e45a36aaa2354b4f722cd71e07 and badf6d159dada727acfb5394bd17de223b336975 X-link: facebook/sapling#950 Reviewed By: quark-zju Differential Revision: D62978511 Pulled By: ahornby fbshipit-source-id: b614e3f10d254283a4904e9e1a6e97870a56b612
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef03f2d10e45a36aaa2354b4f722cd71e07 and badf6d159dada727acfb5394bd17de223b336975 X-link: facebook/sapling#950 Reviewed By: quark-zju Differential Revision: D62978511 Pulled By: ahornby fbshipit-source-id: b614e3f10d254283a4904e9e1a6e97870a56b612
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef03f2d10e45a36aaa2354b4f722cd71e07 and badf6d159dada727acfb5394bd17de223b336975 X-link: facebook/sapling#950 Reviewed By: quark-zju Differential Revision: D62978511 Pulled By: ahornby fbshipit-source-id: b614e3f10d254283a4904e9e1a6e97870a56b612
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef03f2d10e45a36aaa2354b4f722cd71e07 and badf6d159dada727acfb5394bd17de223b336975 X-link: facebook/sapling#950 Reviewed By: quark-zju Differential Revision: D62978511 Pulled By: ahornby fbshipit-source-id: b614e3f10d254283a4904e9e1a6e97870a56b612
Summary: X-link: facebookincubator/zstrong#995 Bring back mononoke getdeps integration tests. This is a refesh of the previously working export-D34186407 branch from 2022 Main changes since: * depend on restored sapling manifest from previous commit * bring back selected manifests needed by mononoke that were deleted in D51869247. I added the sqlite binary packages from this to the sqlite3 manifest rather than bring back the sqllite3-bin manifest * add manifests for new tools used: ripgrep, git-lfs, and zstd cli (we already had the zstd libs) * fix a few test expecations that where too closely tied to git cli or TLS version etc (ubuntu 22.04 is on older version) * getdeps MakefileBuilder.run_tests() improvements * fix error status reporting, was not failing if tests failed * pass of --num-jobs to tests to stop it OOMing my machine * pass of --filter to tests so can iterate on one test more easily Can to iterate local execution for one test with: ``` python3 ./build/fbcode_builder/getdeps.py test --num-jobs 4 --allow-system-packages --no-facebook-internal --src-dir=. mononoke_integration --retry 0 --filter server/test-gettreepack.t ``` Pull Request resolved: #951 Test Plan: Setup a local ubuntu 22.04 toolbox with: ``` toolbox create --distro ubuntu --release 22.04 toolbox enter ubuntu-toolbox-22.04 sudo apt update sudo apt install g++ libssl-dev pkg-config sudo locale-gen en_US.UTF-8 curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ``` Build dependencies: ``` python3 ./build/fbcode_builder/getdeps.py install-system-deps --recursive mononoke_integration python3 ./build/fbcode_builder/getdeps.py build --allow-system-packages --no-facebook-internal --src-dir=. --no-tests sapling python3 ./build/fbcode_builder/getdeps.py build --allow-system-packages --no-facebook-internal --src-dir=. --no-tests mononoke ``` then test locally: ``` ./build/fbcode_builder/getdeps.py build --no-deps --num-jobs 16 --allow-system-packages --no-facebook-internal --src-dir=. mononoke_integration ./build/fbcode_builder/getdeps.py test --num-jobs 6 --allow-system-packages --no-facebook-internal --src-dir=. mononoke_integration ``` Generate the github actions yaml ``` ./build/fbcode_builder/getdeps.py generate-github-actions --allow-system-packages --test --os-type linux --job-file-prefix mononoke-integration_ --job-name-prefix "Mononoke Integration " --output-dir .github/workflows mononoke_integration --num-jobs 4 ``` --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/951). * __->__ #951 * #950 Reviewed By: quark-zju Differential Revision: D62978526 Pulled By: ahornby fbshipit-source-id: 0070a67d798bb23ee9e78e1a5149ba5364d548c9
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef03f2d10e45a36aaa2354b4f722cd71e07 and badf6d159dada727acfb5394bd17de223b336975 X-link: facebook/sapling#950 Reviewed By: quark-zju Differential Revision: D62978511 Pulled By: ahornby fbshipit-source-id: b614e3f10d254283a4904e9e1a6e97870a56b612
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef03f2d10e45a36aaa2354b4f722cd71e07 and badf6d159dada727acfb5394bd17de223b336975 X-link: facebook/sapling#950 Reviewed By: quark-zju Differential Revision: D62978511 Pulled By: ahornby fbshipit-source-id: b614e3f10d254283a4904e9e1a6e97870a56b612
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef03f2d10e45a36aaa2354b4f722cd71e07 and badf6d159dada727acfb5394bd17de223b336975 X-link: facebook/sapling#950 Reviewed By: quark-zju Differential Revision: D62978511 Pulled By: ahornby fbshipit-source-id: b614e3f10d254283a4904e9e1a6e97870a56b612
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef03f2d10e45a36aaa2354b4f722cd71e07 and badf6d159dada727acfb5394bd17de223b336975 X-link: facebook/sapling#950 Reviewed By: quark-zju Differential Revision: D62978511 Pulled By: ahornby fbshipit-source-id: b614e3f10d254283a4904e9e1a6e97870a56b612
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef03f2d10e45a36aaa2354b4f722cd71e07 and badf6d159dada727acfb5394bd17de223b336975 X-link: facebook/sapling#950 Reviewed By: quark-zju Differential Revision: D62978511 Pulled By: ahornby fbshipit-source-id: b614e3f10d254283a4904e9e1a6e97870a56b612
Summary: X-link: facebookincubator/zstrong#994 Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon How: * started with backout of 73302ef03f2d10e45a36aaa2354b4f722cd71e07 and badf6d159dada727acfb5394bd17de223b336975 X-link: facebook/sapling#950 Reviewed By: quark-zju Differential Revision: D62978511 Pulled By: ahornby fbshipit-source-id: b614e3f10d254283a4904e9e1a6e97870a56b612
Summary:
Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon
How:
Test Plan:
Setup a local ubuntu 22.04 toolbox with:
Build locally with:
python3 build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. sapling --project-install-prefix sapling:/usr/local
Test locally with:
python3 build/fbcode_builder/getdeps.py --allow-system-packages test --src-dir=. sapling --project-install-prefix sapling:/usr/local
Generate github actions with:
./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --free-up-disk --os-type=linux --src-dir=. --output-dir=.github/workflows --job-name "Sapling CLI Getdeps " --job-file-prefix=sapling-cli-getdeps_ sapling
CI
Stack created with Sapling. Best reviewed with ReviewStack.