Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for custom resource logging #2508

Merged
merged 4 commits into from
Feb 28, 2024

Conversation

davidfowl
Copy link
Member

@davidfowl davidfowl commented Feb 28, 2024

  • Resources can call WithResourceLogger to add an annotation that allows them to publish logs to a live stream. These will flow through the resource server to any consumer (including the dashboard).
  • Share circular buffer between otlp log storage and aspire.hosting.
  • Added tests

Fixes #2498

custom-resource-logs.mp4
Microsoft Reviewers: Open in CodeFlow

- Resources can call WithResourceLogger to add an annotation that allows them to publish logs to a live stream. These will flow through the resource server to any consumer (including the dashboard).
- Share circular buffer between otlp log storage and aspire.hosting.
- Added tests
@dotnet-issue-labeler dotnet-issue-labeler bot added the area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication label Feb 28, 2024
@davidfowl davidfowl added this to the preview 5 (Apr) milestone Feb 28, 2024
if (annotation._backlog.Count > 0)
{
// REVIEW: Performance makes me very sad, but we can optimize this later.
yield return annotation._backlog.ToList();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think console logs need the concept of paging. Returning all the data and displaying all the data is a performance problem, especially showing 1000s of lines in the browser.

Virtualization means only a subset of the data is needed.

I noticed this earlier when I created #2507

tests/Aspire.Hosting.Tests/CustomResourceLoggerTests.cs Outdated Show resolved Hide resolved
@mitchdenny
Copy link
Member

Two thoughts on this:

  1. Given we are constraining height of the logging pile (10000 entries). Should we also define a maximum width to the message so we know that on a per resource basis memory consumption cannot exceed X?
  2. Should we also be looking to output these entries to the AppHost console log?

@mitchdenny
Copy link
Member

Tentatively approving pending the fix on the test failure and responses to those questions above. They can be follow up PRs if needs be.

- Support completing the log stream
- Added tests
@davidfowl davidfowl merged commit 87a2fd7 into main Feb 28, 2024
8 checks passed
@davidfowl davidfowl deleted the davidfowl/logging-custom-resources branch February 28, 2024 15:41
@davidfowl
Copy link
Member Author

@mitchdenny

Given we are constraining height of the logging pile (10000 entries). Should we also define a maximum width to the message so we know that on a per resource basis memory consumption cannot exceed X?

I don't think so, at least not by default. We'll need something more durable eventually though. I'm not a fan of opening a temp file bug it might be required for longer log streams.

Should we also be looking to output these entries to the AppHost console log?

Nope, see #2396

@github-actions github-actions bot locked and limited conversation to collaborators Apr 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Added support for logging for custom resources
3 participants