Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for querying OCI graph #99

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Commits on Nov 1, 2024

  1. fcos-graph-builder: rename updates variable

    It's confusingly named `stream_metadata_url` but actually is the URL to
    the update guidance.
    jlebon committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    68de10b View commit details
    Browse the repository at this point in the history
  2. fcos-graph-builder: add trace log for requests

    Found this helpful in debugging.
    jlebon committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    564d6ed View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2024

  1. graph-builder: scrape only once per stream

    Currently, we're starting a total of N x M scrapers, where N is the
    number of streams, and M the number of arches. So right now, this would
    be 3 x 4 = 12 scrapers.
    
    This is very wasteful because each scraper individually downloads the
    release index and update metadata every 30 seconds, even though that
    metadata is not different per architecture. I think the reason it was
    set up this way is in case we wanted to host separate e.g. release
    index or update files _per_ architecture in S3 instead of all together.
    This can be seen by the fact the code supports templating those URLs
    with `basearch`. However, it's unlikely we'll be changing that design
    decision, so let's just do the saner thing and rework the scraping to
    be stream-based.
    
    This is done by changing the scraper to host not one single `Graph`
    object, but instead a `HashMap<String, Graph>` which maps architectures
    to graphs. Then, when a request for a graph comes in, we lookup in our
    cache keying off of the requested architecture.
    
    This is prep for adding another dimension to the matrix, which is
    whether the OCI version of the graph was requested. If we didn't do this
    cleanup first, it would have blown up the number of scrapers to 24.
    
    Part of coreos/fedora-coreos-tracker#1823.
    jlebon committed Nov 2, 2024
    Configuration menu
    Copy the full SHA
    7c0228c View commit details
    Browse the repository at this point in the history
  2. Add support for querying OCI graph

    When parsing the release index, check for the new `oci-images` key. If
    present, also build up a separate graph with only nodes containing OCI
    information. In that case, the node payload is the pullspec and the
    scheme declared in the node metadata is `oci`.
    
    When a client requests a graph, check if the `oci=` URL parameter was
    set. If so, return back the OCI graph instead of the OSTree one.
    
    Part of coreos/fedora-coreos-tracker#1823.
    jlebon committed Nov 2, 2024
    Configuration menu
    Copy the full SHA
    5329303 View commit details
    Browse the repository at this point in the history
  3. tree: fix clippy warnings

    This is a mix of both new and old clippy warnings. Just doing it in one
    commit to make it easier.
    jlebon committed Nov 2, 2024
    Configuration menu
    Copy the full SHA
    6b2985a View commit details
    Browse the repository at this point in the history