diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 8761e39b..4360864d 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -1 +1 @@ -* conda/conda-core +* conda/steering-council diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml new file mode 100644 index 00000000..ce0896e5 --- /dev/null +++ b/.pre-commit-config.yaml @@ -0,0 +1,24 @@ +# disable autofixing PRs, commenting "pre-commit.ci autofix" on a pull request triggers a autofix +ci: + autofix_prs: false +# generally speaking we ignore all vendored code as well as tests data +exclude: | + (?x)^( + tests/data/.* | + conda_libmamba_solver/mamba_utils\.py + )$ +repos: + # generic verification and formatting + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v5.0.0 + hooks: + # standard end of line/end of file cleanup + - id: mixed-line-ending + - id: end-of-file-fixer + - id: trailing-whitespace + - id: check-merge-conflict + - repo: meta + # see https://pre-commit.com/#meta-hooks + hooks: + - id: check-hooks-apply + - id: check-useless-excludes diff --git a/README.md b/README.md index 15bebb78..4938e3cb 100644 --- a/README.md +++ b/README.md @@ -10,23 +10,24 @@ for conda's implementation, all major changes should be submitted as | CEP | Title | | --- | ------- | -| [1](cep-1.md) | CEP Purpose and Guidelines | -| [2](cep-2.md) | Add plugin architecture to conda | -| [3](cep-3.md) | Using the Mamba solver in conda | -| [4](cep-4.md) | Implement initial conda plugin mechanism | -| 5 | Does not exist | -| [6](cep-6.md) | Add Channel Notices to conda -| 7 | Does not exist | -| [8](cep-8.md) | Conda Release Schedule | -| [9](cep-9.md) | Conda Deprecation Schedule | -| [10](cep-10.md) | Conda Version Support | -| [11](cep-11.md) | Define the menuinst standard | -| [12](cep-12.md) | Serving run_exports metadata in conda channels | -| [13](cep-13.md) | A new recipe format - part 1 | -| [14](cep-14.md) | A new recipe format – part 2 - the allowed keys & values | -| [15](cep-15.md) | Hosting repodata.json and packages separately by adding a base_url property. | -| [16](cep-16.md) | Sharded Repodata | -| [17](cep-17.md) | Optional python site-packages path in repodata | +| [0000](cep-0000.md) | CEP template | +| [0001](cep-0001.md) | CEP Purpose and Guidelines | +| [0002](cep-0002.md) | Add plugin architecture to conda | +| [0003](cep-0003.md) | Using the Mamba solver in conda | +| [0004](cep-0004.md) | Implement initial conda plugin mechanism | +| 0005 | _Does not exist_ | +| [0006](cep-0006.md) | Add Channel Notices to conda +| 0007 | _Does not exist_ | +| [0008](cep-0008.md) | Conda Release Schedule | +| [0009](cep-0009.md) | Conda Deprecation Schedule | +| [0010](cep-0010.md) | Conda Version Support | +| [0011](cep-0011.md) | Define the menuinst standard | +| [0012](cep-0012.md) | Serving run_exports metadata in conda channels | +| [0013](cep-0013.md) | A new recipe format - part 1 | +| [0014](cep-0014.md) | A new recipe format – part 2 - the allowed keys & values | +| [0015](cep-0015.md) | Hosting repodata.json and packages separately by adding a base_url property. | +| [0016](cep-0016.md) | Sharded Repodata | +| [0017](cep-0017.md) | Optional python site-packages path in repodata | ## References @@ -39,9 +40,9 @@ Community members are encouraged to author a CEP to suggest changes *before* any code is written to allow for the community to discuss the proposed changes. The formal process by which CEPs should be authored and how they are reviewed -and resolved is specified in [CEP 1](https://github.com/conda/ceps/blob/main/cep-1.md). +and resolved is specified in [CEP-1](https://github.com/conda/ceps/blob/main/cep-0001.md). -A template for new CEPs is given in [CEP 0](https://github.com/conda/ceps/blob/main/cep-0.md). +A template for new CEPs is given in [CEP-0](https://github.com/conda/ceps/blob/main/cep-0000.md). ## Conda capitalization standards diff --git a/cep-0.md b/cep-0000.md similarity index 100% rename from cep-0.md rename to cep-0000.md diff --git a/cep-1.md b/cep-0001.md similarity index 99% rename from cep-1.md rename to cep-0001.md index 7463b5f2..3d266238 100644 --- a/cep-1.md +++ b/cep-0001.md @@ -55,7 +55,7 @@ or as part of a conda community meeting, is the best way to go about this. Once the champion has asked the conda community whether an idea has any chance of acceptance, a draft CEP should be written following the -template in [CEP 0](cep-0.md) and described below. +template in [CEP 0](cep-0000.md) and described below. While working on the CEP prior to submission, please set its status to *draft*. diff --git a/cep-2.md b/cep-0002.md similarity index 100% rename from cep-2.md rename to cep-0002.md diff --git a/cep-3.md b/cep-0003.md similarity index 100% rename from cep-3.md rename to cep-0003.md diff --git a/cep-4.md b/cep-0004.md similarity index 95% rename from cep-4.md rename to cep-0004.md index 833cb79f..17ea7b67 100644 --- a/cep-4.md +++ b/cep-0004.md @@ -16,7 +16,7 @@ In order to enable customization and extra features that are compatible with and A plugin mechanism in `conda` would provide many benefits, including (but not limited to) the coverage of underserved use cases, the added ability to extend `conda` internals via official APIs, as well as lowering the barrier of entry for contributions from other stakeholders in the `conda` community and ecosystem. -The [initial proposal to add this plugin architecture](https://github.com/conda-incubator/ceps/blob/main/cep-2.md) has been [officially approved](https://github.com/conda-incubator/ceps/issues/23); this current CEP will discuss the specific way that the plugin mechanism will actually be implemented. +The [initial proposal to add this plugin architecture](https://github.com/conda-incubator/ceps/blob/main/cep-0002.md) has been [officially approved](https://github.com/conda-incubator/ceps/issues/23); this current CEP will discuss the specific way that the plugin mechanism will actually be implemented. ## Specification @@ -130,7 +130,7 @@ It should be noted that a request for change was recorded in the pull request ab ## Reference - [`pluggy` Documentation](https://pluggy.readthedocs.io/en/stable/index.html) -- [CEP 2: "Add plugin architecture to conda"](https://github.com/conda-incubator/ceps/blob/main/cep-2.md) +- [CEP 2: "Add plugin architecture to conda"](https://github.com/conda-incubator/ceps/blob/main/cep-0002.md) - [`conda` Issue #11112: Introduce a Plugin System](https://github.com/conda/conda/issues/11112) ## Copyright diff --git a/cep-6.md b/cep-0006.md similarity index 94% rename from cep-6.md rename to cep-0006.md index 52ef637d..7b871666 100644 --- a/cep-6.md +++ b/cep-0006.md @@ -11,9 +11,9 @@ ## Abstract -In order to facilitate better communication between users of `conda` and channel owners, we want to +In order to facilitate better communication between users of `conda` and channel owners, we want to implement a notification feature. -Specifically, this feature will give channel owners the ability to send users a small message (about the size +Specifically, this feature will give channel owners the ability to send users a small message (about the size of a single Twitter post). How often this message displays will be set by channel owners and will be displayed for each channel a user is actively using (i.e. everything under "channels" in a user's `.condarc`). @@ -66,12 +66,12 @@ Channel "defaults" has the following notices: [info] -- Tue May 10 11:50:34 2022 This is a test message. It is not very long, and could have a link to a longer post: https://example.com/short-link - + ``` ### How else can our users access this message? -Additionally, because our users may wish to see this message on demand, we will add a new sub-command called `notices`. +Additionally, because our users may wish to see this message on demand, we will add a new sub-command called `notices`. The following are a couple examples to show exactly how it would function: **Basic usage:** grabs notices for all current channels: @@ -105,8 +105,8 @@ Channel "defaults" has the following notices: ### What file format will this message have, and what will it contain? -The notification message will be in the JSON file format. This will allow us to not only store the message itself but -also metadata about the message, including information about how often the client should display the message (more on +The notification message will be in the JSON file format. This will allow us to not only store the message itself but +also metadata about the message, including information about how often the client should display the message (more on this in the next section). Here's an example of the `notices.json` file which will be stored in the root of the channel directory structure. @@ -137,7 +137,7 @@ Detailed overview of the JSON fields: ### How often will these messages appear? -How often these messages appear will be configurable by the channel owners and the client. This will be accomplished by +How often these messages appear will be configurable by the channel owners and the client. This will be accomplished by the `expires_at` field in the `notices.json` file itself. When a message has expired, it will trigger a refresh from the server at which point it will show new messages if there are any. @@ -155,7 +155,7 @@ The motivation for this CEP came about as channel owners (Anaconda specifically) users of their channels. These messages may contain specific notices for particular users (e.g. identified by IP address) or general messages for the wider audience of a particular channel. -Additionally, this new notification space can also provide a place for us to relocate `conda update conda` reminders +Additionally, this new notification space can also provide a place for us to relocate `conda update conda` reminders to a more visible spot (at the end of command output versus in the middle of the output). On top of this, other channels can use these notices as a way to share news with their users or requests for help in maintaining their channels. @@ -170,7 +170,7 @@ channels may choose to dynamically serve this JSON file and this specification a The other consideration was keeping these messages to a minimum for our users and allowing them to easily disable these messages if they do not want to see them while running commands such as "install" or "create". Although -we feel these messages are important for users to see, we also do not want to clutter their terminal output. +we feel these messages are important for users to see, we also do not want to clutter their terminal output. Ultimately, this is why we are choosing to turn this on by default but are also providing a way to disable it. @@ -184,14 +184,14 @@ We do not expect any backwards compatibility issues for this new feature. - **Show notices at the beginning of environment activation:** This was deemed too intrusive/annoying. - **Show notices at the beginning of command output:** Users may miss this if placed here, especially for commands with lots of output -- **Message in an HTTP header when retrieving any file from the repository:** This would be a better option for some kinds +- **Message in an HTTP header when retrieving any file from the repository:** This would be a better option for some kinds of messages, like download rate limiting or other blocks due to abuse, since it could be turned on by a rule in a CDN. - However, this use case is probably better addressed by having a standard way for `conda` to display errors on the + However, this use case is probably better addressed by having a standard way for `conda` to display errors on the console from HTTP status codes like 429 (Too Many Requests) and 403 (Forbidden). Additionally, serving custom headers - is challenging unless the repository owner has more control over the web server than is usually given by services + is challenging unless the repository owner has more control over the web server than is usually given by services like Github Pages. - **Add notices to the repodata.json file:** The repodata.json file is already being fetched, so adding some - notices would reduce the number of requests. However, the repodata.json file is way too big already, so it + notices would reduce the number of requests. However, the repodata.json file is way too big already, so it would require clients to download a fairly large file before even being able to show the notification. - **Create a generic metadata.json file containing a notices key:** This could be appealing for creating a space for other kinds of channel metadata, but in order to keep things as simple as possible at the moment diff --git a/cep-8.md b/cep-0008.md similarity index 100% rename from cep-8.md rename to cep-0008.md diff --git a/cep-9.md b/cep-0009.md similarity index 98% rename from cep-9.md rename to cep-0009.md index 7c070cd4..94ccd387 100644 --- a/cep-9.md +++ b/cep-0009.md @@ -9,7 +9,7 @@ -[cep8]: https://github.com/conda-incubator/ceps/blob/main/cep-8.md +[cep8]: https://github.com/conda-incubator/ceps/blob/main/cep-0008.md [django]: https://docs.djangoproject.com/en/dev/internals/release-process/#deprecation-policy [voting]: https://github.com/conda-incubator/governance#enhancement-proposal-approval diff --git a/cep-10.md b/cep-0010.md similarity index 97% rename from cep-10.md rename to cep-0010.md index d1c1536b..b51de5a0 100644 --- a/cep-10.md +++ b/cep-0010.md @@ -10,7 +10,7 @@ ## Abstract -This CEP introduces an official version support policy to promote transparency and certainty +This CEP introduces an official version support policy to promote transparency and certainty about which versions of conda are supported. The policy itself states that only the latest released version of conda will be supported. Instead of patching previous releases, maintainers will only focus their efforts on new and current releases. The implications of @@ -21,7 +21,7 @@ this and how this looks will be outlined below. The latest version of conda will be the only officially supported version of conda. This relates only to the conda project itself (i.e. conda-build is not included). This means that the only time patch releases will be made are soon after a release goes out. For example, if a -bug is identified soon after releasing conda `23.11.0` and fixed, a subsequent `23.11.1` +bug is identified soon after releasing conda `23.11.0` and fixed, a subsequent `23.11.1` release will be made. Patch releases for conda versions older than the current version will not be made. @@ -41,15 +41,15 @@ a particular version of conda will be supported. We do not expect our users to always be running the latest version of conda, but we also do not want to give users the impression that every version of conda will be supported indefinitely. Therefore, we felt it necessary to outline with this CEP exactly which versions -will be supported and how. This not only helps set expectations for our users but -also eases our development efforts knowing we no longer have to support older versions +will be supported and how. This not only helps set expectations for our users but +also eases our development efforts knowing we no longer have to support older versions of conda. ## Rationale For many projects (more information here: https://endoflife.date), either the latest version is supported or the projects have a sliding window of time -for their supported versions. This window is a guarantee saying that the +for their supported versions. This window is a guarantee saying that the version in question will be supported for a specific amount of time. For most who use a version window, they further specify the types of fixes a version will receive as it ages. @@ -91,4 +91,4 @@ Helpful websites and articles: All CEPs are explicitly [CC0 1.0 Universal](https://creativecommons.org/publicdomain/zero/1.0/). -[deprecation-schedule]: https://github.com/conda-incubator/ceps/blob/main/cep-9.md \ No newline at end of file +[deprecation-schedule]: https://github.com/conda-incubator/ceps/blob/main/cep-0009.md diff --git a/cep-11.md b/cep-0011.md similarity index 98% rename from cep-11.md rename to cep-0011.md index 4f5cabb6..89264277 100644 --- a/cep-11.md +++ b/cep-0011.md @@ -55,7 +55,7 @@ simplified overview of all possible keys and their default values: "platforms": { # To create the menu item for a fiven platform, the key must be present in this # dictionary. Presence is enough; the value can just be the empty dictionary: {}. - "linux": { + "linux": { # See XDG Desktop standard for details # https://specifications.freedesktop.org/desktop-entry-spec/desktop-entry-spec-latest.html#recognized-keys "Categories": None, @@ -98,7 +98,7 @@ simplified overview of all possible keys and their default values: #: see https://developer.apple.com/documentation/bundleresources/entitlements "entitlements": None, #: symlink a file (usually the executable in 'command') into the .app directory - "link_in_bundle": None, + "link_in_bundle": None, #: shell logic that will run when an Apple event is received "event_handler": None, }, @@ -141,9 +141,9 @@ Placeholder | Value `HOME` | Path to the user directory (`~`). `ICON_EXT` | Extension of the icon file expected by each platform. `png` in Linux, `icns` in macOS, and `ico` in Windows. Note the dot is _not_ included. **macOS-only** | -`PYTHONAPP` | Path to the `python` executable installed in `PREFIX`, assuming the `python.app` conda package is installed. Equivalent to `{{ PREFIX }}/python.app/Contents/MacOS/python`. -**Windows-only** | -`SCRIPTS_DIR` | Path to the `Scripts` directory in `PREFIX`. +`PYTHONAPP` | Path to the `python` executable installed in `PREFIX`, assuming the `python.app` conda package is installed. Equivalent to `{{ PREFIX }}/python.app/Contents/MacOS/python`. +**Windows-only** | +`SCRIPTS_DIR` | Path to the `Scripts` directory in `PREFIX`. `BASE_PYTHONW` | Path to the `pythonw.exe` executable in `BASE_PREFIX`. `PYTHONW` | Path to the `pythonw.exe` executable in `PREFIX`. @@ -190,12 +190,12 @@ resources (XML files on Linux, Registry on Windows), these MUST be undone too. ## CLI interface The implementor CLI is _not_ defined in this document. However, the integrations with `constructor` -SHOULD be standardized if they are going to be kept in use. +SHOULD be standardized if they are going to be kept in use. The proposed CLI (inspired by what's already in use to introduce minimal changes) is: ```shell -${IMPLEMENTOR} constructor --prefix ${PREFIX} [--base-prefix ${BASE_PREFIX}] [--mode user|system] [--make-menus | --rm-menus] [pkg-name ...] +${IMPLEMENTOR} constructor --prefix ${PREFIX} [--base-prefix ${BASE_PREFIX}] [--mode user|system] [--make-menus | --rm-menus] [pkg-name ...] ``` - `--make-menus` will create the menu items for the JSON files found in `$PREFIX/Menu`. @@ -365,7 +365,7 @@ implemented. Per-platform options are allowed but only when strictly necessary. - A binary launcher is required for correct system integration (see reasons [`conda/menuinst#123`](https://github.com/conda/menuinst/issues/123)). This is placed at `.app/Contents/MacOS/`. The proposed launcher simply guesses its own location to find - the `*-script` file, which is spawned in a subprocess. + the `*-script` file, which is spawned in a subprocess. - In some cases, if an external binary is required, it needs to be symlinked into the `.app` directory to ensure keyboard integrations work (see [`conda/menuinst#122`](https://github.com/conda/menuinst/issues/122)). diff --git a/cep-12.md b/cep-0012.md similarity index 98% rename from cep-12.md rename to cep-0012.md index f0223b61..cf047da3 100644 --- a/cep-12.md +++ b/cep-0012.md @@ -2,7 +2,7 @@ Title Serving run_exports metadata in conda channels Status Accepted Author(s) - + Jaime Rodríguez-Guerra <jrodriguez@quansight.com> Created May 4, 2023 @@ -27,11 +27,11 @@ Since `conda-index` already processes the package metadata to generate `repodata ## Precedence and role in the pinning resolution process -This file is not meant to replace the `run_exports` metadata already present in the package archives. It merely presents that information in a more convenient way. +This file is not meant to replace the `run_exports` metadata already present in the package archives. It merely presents that information in a more convenient way. `conda-build`-like clients are free to use either the `run_exports` metadata in the archives or the one in `run_exports.json`, since they MUST be equivalent. Special keys like `pin_run_as_build` MUST keep their behavior, since `run_exports.json` does not add a new level of precedence in the pinning resolution process. Again, it's an equivalent source for information already present in the archives. -This means that `run_exports.json` MUST NOT be patched (like it is done with `repodata.json`). It MUST always reflect the metadata present in the archives. +This means that `run_exports.json` MUST NOT be patched (like it is done with `repodata.json`). It MUST always reflect the metadata present in the archives. > Note this only applies to the served `run_exports.json` file. It does not try to regulate what > `conda-build`-like tools can do at environment creation time. They might need to apply @@ -57,7 +57,7 @@ Each `run_exports` metadata `dict` can contain the following fields; each field - `noarch` ```json -{ +{ "info": { "platform": "string", "arch": "string", diff --git a/cep-13.md b/cep-0013.md similarity index 99% rename from cep-13.md rename to cep-0013.md index 5d62e35b..e6570eca 100644 --- a/cep-13.md +++ b/cep-0013.md @@ -139,7 +139,7 @@ source: then: url: http://path/to/windows/source - if: (unix and cmp(python, "2")) - then: + then: url: http://path/to/python2/unix/source - if: unix and cmp(python, "3") then: @@ -225,7 +225,7 @@ there will be several shortcomings. For example, using a `{% for ... %}` loop is prohibited. After searching through the conda-forge recipes with the Github search, we found for loops mostly used -in tests. +in tests. In our view, for loops are a nice helper, but not necessary for many tasks: the same functionality could be achieved in a testing script, for example. At the @@ -277,7 +277,7 @@ source: build: number: 0 # note: in the new recipe format, `skip` is a list of conditional expressions - # but for the "YAML format" discussion we pretend that we still use the + # but for the "YAML format" discussion we pretend that we still use the # `skip: bool` syntax skip: ${{ true if (win and vc14) }} @@ -322,4 +322,3 @@ extra: recipe-maintainers: - some-maintainer ``` - diff --git a/cep-14.md b/cep-0014.md similarity index 99% rename from cep-14.md rename to cep-0014.md index 4e2e0a91..3142afdd 100644 --- a/cep-14.md +++ b/cep-0014.md @@ -22,7 +22,7 @@ In this CEP, we define the allowed keys and values for the recipe. ## Motivation The conda-build format is currently under-specified. For the new format, we try to list all values -and types in a single document and document them. This is part of a larger effort (see [CEP-13](/cep-13.md) for the new YAML syntax). +and types in a single document and document them. This is part of a larger effort (see [CEP-13](/cep-0013.md) for the new YAML syntax). ## History diff --git a/cep-15.md b/cep-0015.md similarity index 100% rename from cep-15.md rename to cep-0015.md diff --git a/cep-16.md b/cep-0016.md similarity index 95% rename from cep-16.md rename to cep-0016.md index 217ead54..d3347021 100644 --- a/cep-16.md +++ b/cep-0016.md @@ -30,18 +30,18 @@ The current repodata format is a JSON file that contains all the packages in a g ### JLAP -In a previously proposed CEP, [JLAP](https://github.com/conda-incubator/ceps/pull/20) was introduced. +In a previously proposed CEP, [JLAP](https://github.com/conda-incubator/ceps/pull/20) was introduced. With JLAP only the changes to an initially downloaded `repodata.json` file have to be downloaded which means the user drastically saves on bandwidth which in turn makes fetching repodata much faster. However, in practice patching the original repodata can be a very expensive operation, both in terms of memory and in terms of compute because of the sheer amount of data involved. JLAP also does not save anything with a cold cache because the initial repodata still has to be downloaded. This is often the case for CI runners. -Finally, the implementation of JLAP is quite complex which makes it hard to adopt for implementers. +Finally, the implementation of JLAP is quite complex which makes it hard to adopt for implementers. ### ZSTD compression -A notable improvement is compressing the `repodata.json` with `zst` and serving that file. In practice this yields a file that is 20% of the original size (20-30 Mb for large cases). Although this is still quite a big file it's substantially smaller. +A notable improvement is compressing the `repodata.json` with `zst` and serving that file. In practice this yields a file that is 20% of the original size (20-30 Mb for large cases). Although this is still quite a big file it's substantially smaller. However, the file still contains all repodata in the channel. This means the file needs to be redownloaded every time anyone adds a single package (even if a user doesnt need that package). @@ -51,12 +51,12 @@ Because the file is relatively big this means that often a large `max-age` is us We propose a "sharded" repodata format. It works by splitting the repodata into multiple files (one per package name) and recursively fetching the "shards". -The shards are stored by the hash of their content (e.g. "content-addressable"). +The shards are stored by the hash of their content (e.g. "content-addressable"). That means that the URL of the shard is derived from the content of the shard. This allows for efficient caching and deduplication of shards on the client. Because the files are content-addressable no round-trip to the server is required to check freshness of individual shards. Additionally an index file stores the mapping from package name to shard hash. -Although not explicitly required the server SHOULD support HTTP/2 to reduce the overhead of doing a massive number of requests. +Although not explicitly required the server SHOULD support HTTP/2 to reduce the overhead of doing a massive number of requests. ### Repodata shard index @@ -82,8 +82,8 @@ The contents look like the following (written in JSON for readability): } ``` -The `shards_base_url` is the base url for the shards. -Both the `base_url` and the `shards_base_url` can be absolute or a relative url. +The `shards_base_url` is the base url for the shards. +Both the `base_url` and the `shards_base_url` can be absolute or a relative url. The urls are relative to the shard index. The index is still updated regularly but the file does not increase in size with every package added, only when new package names are added which happens much less often. @@ -94,13 +94,13 @@ We suggest serving the file with a short lived `Cache-Control` `max-age` header ### Repodata shard -Individual shards are stored under the URL `.msgpack.zst`. -Where the `sha256` is the lower-case hex representation of the bytes from the index and `shards_base_url` is defined in the shard index. -It is a zstandard compressed msgpack file that contains the metadata of the package. +Individual shards are stored under the URL `.msgpack.zst`. +Where the `sha256` is the lower-case hex representation of the bytes from the index and `shards_base_url` is defined in the shard index. +It is a zstandard compressed msgpack file that contains the metadata of the package. The files are content-addressable which makes them ideal to be served through a CDN. They SHOULD be served with `Cache-Control: immutable` header. -The shard contains the repodata information that would otherwise have been found in the `repodata.json` file. +The shard contains the repodata information that would otherwise have been found in the `repodata.json` file. It is a dictionary that contains the following keys: **Example (written in JSON for readability):** @@ -162,7 +162,7 @@ It is a dictionary that contains the following keys: } ``` -The `sha256` and `md5` from the original repodata fields are converted from their hex representation to bytes. +The `sha256` and `md5` from the original repodata fields are converted from their hex representation to bytes. This is done to reduce the overall file size of the shards. Implementers SHOULD ignore unknown keys, this allows adding additional keys to the format in the future without breaking old versions of tools. @@ -174,7 +174,7 @@ Although these files can become relatively large (100s of kilobytes) typically f To fetch all needed package records, the client should implement the following steps: 1. Fetch the `repodata_shards.msgpack.zst` file. Standard HTTP caching semantics can be applied to this file. -3. For each package name, start fetching the corresponding hashes from the index file (for both arch & and noarch). +3. For each package name, start fetching the corresponding hashes from the index file (for both arch & and noarch). Shards can be cached locally and because they are content-addressable no additional round-trips to the server are required to check freshness. The server should also mark these with an `immutable` `Cache-Control` header. 4. Parsing the requirements of the fetched records and add the package names of the requirements to the set of packages to fetch. 5. Loop back to 1. until there are no new package names to fetch. @@ -194,7 +194,7 @@ SHA hashes are non-compressable because in the eyes of the compressor it is just ### Shorter SHA hashes -Another approach would be to only store the first 100 bytes or so of the SHA hash. This reduces the total size of the sha hashes significantly but it makes the client side implementation more complex because hash conflicts become an issue. +Another approach would be to only store the first 100 bytes or so of the SHA hash. This reduces the total size of the sha hashes significantly but it makes the client side implementation more complex because hash conflicts become an issue. This also makes a future implementation based on a OCI registry harder because layers in an OCI registry are also referenced by SHA256 hash. @@ -211,7 +211,7 @@ To improve compression we investigated storing the index file as a struct of arr { "name": "", "hash": "", - } + } ] ``` @@ -230,9 +230,9 @@ This did yield slightly better compression but we felt it makes it slightly hard ### Authentication -With this approach a client could do hundreds of requests concurrently. -Authenticating all these requests would impose a non-negligable overhead to each requests and will also require more server resources. -Initially a one-time authentication flow similar to OCI registry tokens was added to the CEP but this has been removed to reduce the scope of the CEP. +With this approach a client could do hundreds of requests concurrently. +Authenticating all these requests would impose a non-negligable overhead to each requests and will also require more server resources. +Initially a one-time authentication flow similar to OCI registry tokens was added to the CEP but this has been removed to reduce the scope of the CEP. Another CEP will be introduced to amend this CEP with a performant authentication flow. ### Remove redundant keys diff --git a/cep-17.md b/cep-0017.md similarity index 100% rename from cep-17.md rename to cep-0017.md