Backtrace now enables you to track trends in your software’s stability as observed by unique end users.
This builds on the ability to compute error rates against metrics you provide / upload into our system (as explained in this document). The functionality allows you normalize your errors by useful metrics such as “number of sessions”, “minutes played”. or any other stability metric that you, as a developer, would want to track in your app.
Now, Backtrace’s SDKs have built in functionality to generate these metrics automatically! This means that you will be able to see session and user stability metrics like “Crash free sessions” in the Backtrace Web Console out of the box! Our APIs also allow you to create your own events so you can track custom metrics and have them sent to your Backtrace instance as well!
How Will This Feature Help Me?
Often it can be difficult to assess an end user’s experience by looking at crash/error data alone. Stability metrics allow you as a developer, producer, project manager, etc… to more easily monitor your project’s health on those terms.
By default, Backtrace’s SDKs are configured to capture session and user metrics and use them to calculate and display the following on your project’s Overview page:
- Crash free session: What % of sessions went without crashing. A great measure of your app’s aggregate session stability
- Crash free users: What % of unique users have not experienced crashes. Perfect for tracking user’s experiences
- Crash free session % through time: A line chart showing crash free session % through time to help you more closely monitor the evolution of your app’s stability
Note that this functionality also works in conjunction with the Web Console’s filtering. As such, you filter your data set to get a sense for these stability metrics within a more specific domain. For example, if you wanted to see the stability of a specific release, you could add that to the filter bar as shown below!
Note: In order for filtering with a specific attribute to work, that attribute must be configured properly to be associated with the metric group in question (in the case of Crash free users / sessions, the "Application Launches" metric group). For more information on how to configure this, see here.
Another common use case would be to exclude non-fatal errors to see these metrics with regard to only fatal errors (Crashes, hangs). The exact filter you would use to do this depends on which attributes you are indexing / SDK you are using. (errorType = “crash” for example).
Prioritizing Crash/Error Resolution
The Web Console’s Triage and Explore views allow you to normalize your fingerprints against these metrics as well. This will allow developers, tech leads to prioritize resolving crashes/errors which have the most impact on your app’s stability.
Once your SDK is sending these metrics, use the drop down at the top right of the Triage and/or explore view to normalize your errors by the metric of your choosing.
In this example, we have selected to normalize our errors by sessions. Now, in the “Errors by sessions” column, you can see the impact of each fingerprint relative to the number of sessions it occurred in. In the screenshot below you can see that even though the fingerprint “8206af2” had the most occurrences, it only manifested in 0.0034% of sessions (1 / 29,220 total sessions in this time period). So it may make more sense for an engineer to investigate and resolve fingerprint “002e365” instead which is present in a higher percentage of sessions.
Advanced Use Cases, Customization
We understand that different applications are likely to require different metrics to properly monitor stability. While our SDKs provide session and unique user metrics out of the box, we built this set of features to allow users to define custom metric events to suit the needs of their applications.
For example, you could configure metrics to normalize for:
- How many minutes of app usage were crash/error free
- How many “levels” in your game a crash/error occurred in
Specific configuration steps for this functionality will vary by SDK so please refer to each supported SDK’s readme file for instructions to take advantage of this functionality. See below for links to SDK-specific documentation.
SDK Support Links
As of August 2021, only our Unity SDK supports this functionality. Through Q3 and 4 of 2021, we intend to continue extending support first to our Unreal Engine SDK, then native Android and iOS.