-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[build] Check versions availability in rules/*.mk at init and use apt-get source instead of dget or wget in Makefiles #13379
Closed
guillaumelambert
wants to merge
18
commits into
sonic-net:master
from
guillaumelambert:checkversions3
Closed
[build] Check versions availability in rules/*.mk at init and use apt-get source instead of dget or wget in Makefiles #13379
guillaumelambert
wants to merge
18
commits into
sonic-net:master
from
guillaumelambert:checkversions3
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- create an a apt alternate configuration directory during 'make init' - use 'apt-get source' instead of dget in makefiles - adapt packages versions accordingly in rules/XX.mk files SONiC images compilation is currently designed to be performed with a non-root user under Ubuntu systems. As a result, there is no guarantee that debian repos are declared in /etc/apt/sources.list on the Ubuntu host system used. Although for this reason 'apt-get source' can not be directly used (out of the box) to that purpose, the build process recompiles packages for various GNU+Linux Debian distributions. And the packages sources used are retrieved from more or less official repositories since the current solution in SONiC is to use dget and URLs from various locations. As discussed in the following URL https://serverfault.com/questions/447457/use-apt-get-source-on-a-debian-repo-without-using-etc-apt-source-list this is the simplest solution but it has several drawbacks from a maintenance and security standpoint, for example : * maintaining manually these URLS since they can be removed by external sites owners sonic-net#13281 * potential loose authentication of packages sources sonic-net#13302 A more complex solution is prosposed (on the same URL) and adress these drawbacks by using an alternate apt configuration directory that is writable by non-root user. It allows here to: * identify packages sources with official Debian Keyring GPG signatures * use 'apt-get source <pkgsrc_name>' to retrieve packages without maintaining several specific URLs * in a next step use 'apt-cache showsrc <pkgsrc_name>' to retrieve a list of available versions to ckeck packages availability during the 'make init' phase Signed-off-by: Guillaume Lambert <[email protected]>
- add a script to check packages sources versions declared in rules/isc-dhcp.mk - call this script durint the 'make init' phase Files with the extension .mk in "rules/" folder are parsed several times during the 'make configure' phase. Thus, it is preferable to check and optionnaly set the package versions declared there before during the previous 'make init' phase. This will prevent from 'apt-get source' calls failures. Signed-off-by: Guillaume Lambert <[email protected]>
…ation - declare bash and debootstrap in automatic version detection script i.e. scripts/check_rules_mk_files_versions.sh - add a notice in script comments - adapt consequently corresponding rules/*.mk and src/*/Makefile files i.e rules/bash.mk rules/debootstrap.mk src/debootstrap/Makefile Signed-off-by: Guillaume Lambert <[email protected]>
…nfiguration - declare kdump-tools and lm-sensors in scripts/check_rules_mk_files_versions.sh - adapt consequently corresponding rules/*.mk files Signed-off-by: Guillaume Lambert <[email protected]>
Signed-off-by: Guillaume Lambert <[email protected]>
Signed-off-by: Guillaume Lambert <[email protected]>
Signed-off-by: Guillaume Lambert <[email protected]>
Signed-off-by: Guillaume Lambert <[email protected]>
Signed-off-by: Guillaume Lambert <[email protected]>
Signed-off-by: Guillaume Lambert <[email protected]>
Signed-off-by: Guillaume Lambert <[email protected]>
Signed-off-by: Guillaume Lambert <[email protected]>
Signed-off-by: Guillaume Lambert <[email protected]>
Signed-off-by: Guillaume Lambert <[email protected]>
Signed-off-by: Guillaume Lambert <[email protected]>
Signed-off-by: Guillaume Lambert <[email protected]>
…sions.sh Signed-off-by: Guillaume Lambert <[email protected]>
Signed-off-by: Guillaume Lambert <[email protected]>
guillaumelambert
force-pushed
the
checkversions3
branch
from
January 16, 2023 16:08
285b4e9
to
d418b65
Compare
wrong branch repo: using orangeopensource repo + fix socat Makefile |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Available versions can be retrieved with 'apt-cache showsrc'
If target version is no more available, there is no point to pursue the build.
Cf [build] Fix isc-dhcp full version in rules.mk #13288
SONiC images compilation is currently designed to be performed with a
non-root user under Ubuntu systems.
And the build process recompiles packages for various GNU+Linux Debian
distributions.
As a result, there is no guarantee that debian repos are declared in
/etc/apt/sources.list on the Ubuntu host system used.
And 'apt-get source' can not be directly used out-of-the-box.
Currently, the packages sources are retrieved from more or less official URLs
and locations with dget and wget.
As discussed in the following URL
https://serverfault.com/questions/447457/use-apt-get-source-on-a-debian-repo-without-using-etc-apt-source-list
dget is the simplest solution but it has several drawbacks from a
maintenance and security standpoint, for example :
sites owners
[Build] Failed to build isc dhcp #13281
A more complex solution is proposed (on the same URL at serverfault) and address these
drawbacks by using an alternate apt configuration directory that is writable
by non-root user. This PR leverages this solution and allows here to:
several specific URLs
list of available versions to ckeck packages availability during the
'make init' phase
If no candidate version is available, the script will exit with an error code and stops the whole compilation process.
This will prevent starting a long build process condemned to fail and thus spare precious time and computing resources.
Issue #13281
Which release branch to backport (provide reason below if selected)
potentially all
Description for the changelog
[build] Check versions availability in rules/*.mk at init and use apt-get source instead of dget or wget in Makefiles
A picture of a cute animal (not mandatory but encouraged)