How do we ensure that we have all the available data to solve bugs appearing in the wild

[oreHGA]

Summary of the new feature/enhancement

Problem

As PowerToys usage increases, there'll likely be more issues appearing in the wild that the team won't have caught - even after extensive tests. How do we ensure that we have all the available data to know when these issues occur without users needing to report them & guide solving them?

Potential Impact

• Improved customer satisfaction as the team is now one step ahead of issues, potentially leading to quicker turn to fixing them
• Reduced dev hours trying to repro/triage issues raised by users :)
• Team can potentially rank issues based on impact on users, criticality of scenarios...

Proposed technical implementation details

I don't have the all the technical technical worked out, but my suggested approach is:

A framework that is able to collect key traces when scenarios don't meet certain defined measures while still ensuring user privacy is upheld. Afterwards, parsing/exploring these traces to highlight & suggest possible root causes, getting rate of occurrence & gauging impact to users etc. This will probably need to be spec'd out based on what telemetry currently exists at the moment, existing work that can be leveraged and what the team's suggested approach is.

What do you think?

[crutkas]

Tracked against #726. this is in progress