Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

detected fields API #12339

Open
trevorwhitney opened this issue Mar 25, 2024 · 0 comments
Open

detected fields API #12339

trevorwhitney opened this issue Mar 25, 2024 · 0 comments
Assignees
Labels
component/api type/feature Something new we should do

Comments

@trevorwhitney
Copy link
Collaborator

Is your feature request related to a problem? Please describe.
Since "detected fields" (ie. logfmt or json fields in a log line) are only known at query time, users exploring their logs can only get information about fields present in the first 1000 (or log limit) log lines. This often pictures a very limited picture. Since indexed label/values are often more about infrastructures, fields in log lines can more important to users debugging an application who may be less knowledgeable about the infrastructure.

To enable these use cases, as well as better visualizations of fields present in logs, it would be nice to have an API that returned information about detected fields.

Describe the solution you'd like
An API that returns information about useful detected fields, where usefulness is determined by:

  • for strings, is the cardinality of values below some configured threshold?
  • for "unwrappable" values, what can it be unwrapped into (ie. duration, bytes, etc.)

The response should include the label names, and should be ordered by usefulness (ie. common fields come first)

Describe alternatives you've considered

  • A time range could be broken up and sampled to get a better representation of a wider time range, however this still wouldn't include information about usefulness with regards to cardinality and unwrap type.

Additional context
Future work may include storing this information in an index, but that's not a requirement for this iteration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/api type/feature Something new we should do
Projects
None yet
Development

No branches or pull requests

2 participants