-
Notifications
You must be signed in to change notification settings - Fork 8.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Telemetry] Usage collectors not using isReady
correctly
#81944
Comments
Pinging @elastic/kibana-telemetry (Team:KibanaTelemetry) |
Thanks @chrisronline! |
@chrisronline the actions, alerts and lens collectors explicitly comment that they're fine with not returning any usage data the first time that their collectors' |
Sure, but we don't have to settle for that and I don't know why we would. We have a way to ensure the telemetry data can always be consistent. My guess is that the authors of those collectors aren't aware that a "miss" might mean there is no telemetry data for that user the entire day. |
@chrisonline Gotcha! I'll update the README and change the example code to make it clearer. |
FWIW, we've had this ability for some time now and there is some history in the original PR: #36153 |
@chrisronline The updated README gives more info on using |
@TinaHeiligers Yea that's great, thanks for adding that! cc @mikecote for any thoughts on if the alerting team should update the alerts/actions collectors |
cc @wylieconlon Lens You might want to reconsider the implementation of |
@TinaHeiligers this is intentional - our telemetry collection can take a long time (minutes). The advice from the telemetry team (IIRC) was to not block telemetry collection on startup. Is the recommendation now different? |
The telemetry collectors will wait up to 1 minute before attempting to send usage data without that collector; however, AFAIK, that time period is somewhat arbitrary and possibly changeable. @dgieselaar Just curious, can you provide more information on why it takes so long? |
I created this issue #82947 for the alerting team to explore / fix this. |
@chrisronline heavy queries (aggregations on billions of documents), and we run them sequentially to be on the safe side as to not overloading the cluster. |
Thanks @dgieselaar. Once this data comes back, is it persisted in memory so that the next telemetry collection cycle picks it up? If so, does it persist in memory until Kibana is restarted? Is it ever re-fetched? |
@chrisronline AFAIK, they use |
That's correct @afharo, thank you - and apologies to @chrisronline for missing this earlier :). We run our queries every 12hrs. |
I see, okay. It seems we need to make an exception for APM. I'll update the issue |
@wylieconlon Have you had a chance to review the Lens usage collector's implementation of |
Pinging @elastic/kibana-core (Team:Core) |
@TinaHeiligers I think this fell through the cracks, is there a deadline for this? |
@flash1293 There isn't a strict deadline from the Core team as such, since it's up to plugin owners to decide how important their data is for them. As with most things telemetry-related, the sooner we ensure we're capturing consistent data, the better 😄 |
I checked the code and it seems like we have the same structure as other consumers listed (although implemented a little different) @gmmorris Seems like you implemented this, could you confirm it doesn't matter whether |
@flash1293 you are right it doesn't matter as the approach, because you are doing the same check if the Task Manager is ready or not and |
The following collectors need to change their
isReady
to properly wait for the task manager state to exist:APMAPM is unique, as their telemetry data is not something easily computedThey are intentionally catching errors where the task manager hasn't ran their task yet, but that should be used to indicate if the usage collector is ready or not.
cc @TinaHeiligers
The text was updated successfully, but these errors were encountered: