Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract all data acquisition from ResourceService #1288

Merged
merged 37 commits into from
Dec 11, 2023

Commits on Dec 8, 2023

  1. Extract all data acquisition from ResourceService

    This creates a new class `KubernetesDataSource` that encapsulates all code related to obtaining data from kubernetes in the `ResourceService`. This cleans the service code up a lot and makes the responsibilities clearer.
    
    The use of `Channel<ResourceChange>` has been removed. Instead, code invocations are made directly, skipping queuing. We still have two other channels that ensure front/back are decoupled. The kubernetes channel will ultimately also be removed, leaving just a channel for the subscriber.
    
    Added some API docs.
    drewnoakes committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    9696462 View commit details
    Browse the repository at this point in the history
  2. Check cancellation token

    drewnoakes committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    ff69825 View commit details
    Browse the repository at this point in the history
  3. Remove Channel<T> for kubernetes objects

    This is the second removal of a `Channel<T>` from this component. Now, updates are passed by direct code invocations, skipping queuing. We still have a downstream channel for the subscriber to decouple front/back.
    drewnoakes committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    5425254 View commit details
    Browse the repository at this point in the history
  4. Inline method

    drewnoakes committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    4fc397c View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    2aae053 View commit details
    Browse the repository at this point in the history
  6. Simplify inclusion of docker environment

    A few changes here:
    
    - Use more consistent names throughout. They're not "extra" or "additional" arguments. In fact they are a complete replacement. In their absence, the "spec" environment is used.
    - Move creation of docker inspection task to view model method, making the various "handle * update" methods more uniform, for future refactoring.
    - Remove special handling in `ProcessKubernetesChange` for one scenario. Simplifies the signature, and the code a fair bit.
    drewnoakes committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    b9c8a83 View commit details
    Browse the repository at this point in the history
  7. Specify list capacity

    drewnoakes committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    db66a05 View commit details
    Browse the repository at this point in the history
  8. Merge ObjectChangeType Add and Modified

    We only need to know if the value is "upserted" (updated or inserted), or "deleted".
    
    Recent changes to threading/queuing in the code here changed some timing. The arrival of one kind of resouce can trigger the publication of another, and these would always be "modified", however they could arrive before that resource's stream published that instance. The rest of the code handles these happening out-of-order. We just need to treat add and modified the same way, so they've been merged.
    drewnoakes committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    a8f68f7 View commit details
    Browse the repository at this point in the history
  9. Use semaphore to serialize kubernetes data processing

    There are multiple kubernetes resource types, each with its own monitoring stream. Across these resources, updates arrive concurrently. The update flows for each type of resource can interact with state stored for other resources. Therefore we use a semaphore to ensure that only one resource update can flow through the system at a time.
    
    Another option would have been to make the collections concurrent, or use explicit locking. Concurrent collections are heavy. Explicit locking is tricky to get right. This mutual exclusion via top-level semaphore seems like a safe and elegant approach for now.
    drewnoakes committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    b6663ef View commit details
    Browse the repository at this point in the history
  10. Avoid redundant linear scans

    When this method is passed the same collection twice, the `FromSpec` value will always be true, because every item in the list is in the list. Instead we pass `null` and consider all items as from the spec.
    drewnoakes committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    8bf4672 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    b255e1d View commit details
    Browse the repository at this point in the history
  12. Avoid switch on every update

    Each instance of `WatchKubernetesResource<T>` produces a single type, that would flow through `ProcessKubernetesChange` to look up the relevant handler. Instead, pass the handler in to `WatchKubernetesResource<T>` so it can be invoked directly.
    drewnoakes committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    bcc255f View commit details
    Browse the repository at this point in the history
  13. Formatting

    drewnoakes committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    6d50eb0 View commit details
    Browse the repository at this point in the history

Commits on Dec 10, 2023

  1. Configuration menu
    Copy the full SHA
    6416388 View commit details
    Browse the repository at this point in the history

Commits on Dec 11, 2023

  1. Rename class to avoid name conflict

    This object is a snapshot of a service's state, not a service itself. We have another class which is a service. Append "Snapshot" to differentiate. Future work will extend this concept of snapshots more broadly.
    drewnoakes committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    5872436 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    24f7d9d View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    abefbdb View commit details
    Browse the repository at this point in the history
  4. Make ResourceViewModel.Environment immutable

    Note that it's just the collection that becomes immutable here. The elements are still currently mutable. That will change when we split front/back ends, and have a snapshot on the backend with a view model on the front end.
    drewnoakes committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    f0afba7 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    a451352 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    07778d7 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    cc0e748 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    9e15ead View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    fc57efe View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    a34c768 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    e515c5e View commit details
    Browse the repository at this point in the history
  12. Source container variables from status

    This data is now available from DCP, so we don't have to launch processes to query docker for this data any more, which simplifies things quite nicely.
    drewnoakes committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    0f0b74b View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    03083cd View commit details
    Browse the repository at this point in the history
  14. Extract duplicate code

    drewnoakes committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    dd2d0a2 View commit details
    Browse the repository at this point in the history
  15. Further deduplicate code

    drewnoakes committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    177948d View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    66eabe9 View commit details
    Browse the repository at this point in the history
  17. Configuration menu
    Copy the full SHA
    e81a95c View commit details
    Browse the repository at this point in the history
  18. Merge project/executable snapshot construction

    Now that project snapshots derive from executable snapshots, we can unify a bunch of construction logic.
    drewnoakes committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    665df71 View commit details
    Browse the repository at this point in the history
  19. Configuration menu
    Copy the full SHA
    4f31609 View commit details
    Browse the repository at this point in the history
  20. Reorder methods

    drewnoakes committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    029ca5b View commit details
    Browse the repository at this point in the history
  21. Start using "snapshot" naming in DCP data source

    We still have some types named ViewModel, but they'll be renamed later in the split.
    drewnoakes committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    fad004b View commit details
    Browse the repository at this point in the history
  22. Rename ObjectChangeType to ResourceChangeType

    It's not clear what object the name refers to, so make it more specific. It's internal, so we can always rename it again later. This name pairs nicely with `ObjectChange` too.
    drewnoakes committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    da62fc6 View commit details
    Browse the repository at this point in the history
  23. Further consolidate data processing

    The methods that handle executable and container updates are largely the same. Extract that commonality to a new method.
    drewnoakes committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    7c657cb View commit details
    Browse the repository at this point in the history