Using Real-User Monitoring to Understand API Performance
In our previous article we talked about Synthetic Monitoring—a technique used to automate the testing and monitoring of resources by performing simulated user interactions and API calls. Now we're going to look at a complimentary technique called real-user monitoring which takes a more passive hands-off approach.
What is real-user monitoring?
Real user monitoring (RUM) is a way of monitoring the performance and stability of your application by keeping track of how users are actually using it. It happens in real time as end users navigate through your website, interact with your application, or make requests for resources. As opposed to simulated, or synthetic monitoring approaches that test how an application works, real user monitoring follows a user through an application from first byte all the way to the end of their visit.
Sometimes this is referred to as end user experience monitoring, or user session monitoring, but in each case it is often part of a suite of tools that check for performance issues, drop-off points in the customer experience, and other metrics that can have a business impact.
While commonly thought of as part of an application performance management tool (APM) that is used to track performance within your own application, real-user monitoring can also be applied to third-party dependencies that your application consumes. For example, if a user initiates an action that pulls from a third-party API, real-user monitoring can keep track of the response time and error rate of that API—and more importantly how it affects specific users.
Benefits and use cases of real user monitoring
Since real-user monitoring can cover everything from website performance to full-scale application performance monitoring, there are adjacent use-cases that can benefit from it.
Most commonly, RUM is split into a few key areas:
- Capturing data as it happens: User visits a page, performs an action, and data is captured about the app and interaction.
- Aggregating data: Rather than needing to parse all user sessions, an overall aggregate displays important metrics that matter to your team.
- Issue identification: At Bearer we call these anomalies. A good RUM system will identify problems as they occur and track them. This can be user-facing, or in our case API focused.
- Notification: While on-demand assessment is great for teams to look into the performance of an application, a good monitoring solution notifies the necessary teams right away when a problem occurs.
Track site behaviors
The simplest form of RUM is site tracking. It enables developers to monitor how a user moves through a website, and tracks site visitors. Tools like Google Analytics and Fathom focus on this aspect. They aren't concerned with technical performance, but rather the user journey, referral details, navigation, and content performance.
Track application performance
The next step up is application performance. APMs and similar tools monitor how an application, website, or service performs as users interact with it. This can be everything from CPU usage, site performance, operating system (OS) problems, and more. While synthetic monitoring can simulate a service under load, real-user monitoring offers baseline values for the most common scenarios.
The focus is on actual problems
Directed or synthetic monitoring is influenced by what teams think they should focus on. Real user monitoring identifies areas where the end user experience suffers and can help refocus teams toward areas that have the most noticeable effect on an application's value.
This is a much easier sell when considering engineering resources. Hypothetical performance issues are important for edge cases, but data gathered by monitoring real user interactions can make it easier to identify a high-impact business case.
Directed tests happen on your timeline. Real-user monitoring occurs in real time whenever an end user, or even your application, interacts with a resource. This means you can set up alerts, notifications, and even remediate problems as they happen. The live nature of this makes visibility into the root cause of a problem easier to debug. It also means you can reach out to any affected stakeholders faster.
Is the site down? Know right away. Is there a runaway resource that might cause a cascade of failures? Notify the ops team so they can get a workaround into the production environment fast. RUM makes this possible in ways that a synthetic monitoring approach can't.
Better SLA tracking
Need hard data to ensure third-party APIs are meeting—or failing to meet—their service level agreements (SLAs)? Real user monitoring makes it easier. Whenever an API is down, the system is already set up to detect and log the problem. If performance drops below a guaranteed threshold, you'll know. More importantly you can schedule reports to keep track of the long-term performance of a third-party service to better inform future decisions on whether to keep it or go with an alternative.
Downsides of real user monitoring
Like any approach, there are some areas that it won't handle, as well as some pain points to watch for.
Personal data retention
Depending on how deeply your monitoring solution follows a user, you may run into some privacy concerns with how you are storing and using data about your users. Ensure that you are abiding by regional data privacy laws, like GDPR or CCPA, when handling data about the user even if it may seem like non-identifying data.
Usefulness is dependent on activity
Unlike synthetic monitoring and other simulated usage approaches, real-user monitoring requires active users. If your product is new or users only use a subset of the features you offer, you may find real user monitoring does not adequately cover all aspects of your application. In this case, it is best to pair it with a synthetic solution in order to get the most complete picture of your application's performance.
An abundance of data
More data means a better chance at finding problems and understanding trends. It also means a fire hose of information. Finding a solution that filters out specific data, and only shows you what you care about is a big hurdle for many developers when using real user monitoring. For third party APIs, at Bearer we developed the concept of Log Collections to ensure that our users can find the signal amongst the noise, even as we monitor all of their API calls.
Using the right approach for your needs
For most developers, a solid mix of synthetic and real user monitoring will be the right solution. At Bearer, we're big proponents of the passive approach for third-party dependencies, as it opens up the possibility of identifying outages and performance issues that may not always show when performing a direct test. For activity within your own app, there are many application performance monitoring tools that can help identify issues and add visibility to your application.
Regardless of your specific use case, real user monitoring is an excellent choice for further understanding the performance of not only your app, but also that of the APIs that it relies on.