Testing uptime, response time, and other performance metrics in applications can take a few different shapes. One common technique is an approach known as synthetic monitoring. This form of performance monitoring doesn't rely on real users interacting with a service, and instead uses automated tools to mimic interactions. Then, the results are recorded and parsed just like other solutions.
Synthetic monitoring is often used for testing your own internal applications, but can it be used to check the performance of third-party services and APIs? In this article we'll explore synthetic monitoring, it's use cases, and how we can apply it to the full landscape of our services—including those we don't have control over.
What is synthetic performance monitoring?
Sometimes called active or directed monitoring, synthetic performance monitoring is a way to simulate the actions of users programmatically in order to test parts of an application or service. This might happen as part of an end-to-end or integrated testing suite, or on a schedule to ensure an application is working as expected.
The main difference between synthetic monitoring and what's known as "real user" or passive monitoring is the lack of actual end users. Because it doesn't rely on a user to perform an action, synthetic monitoring can test websites, web services, internal and third-party APIs, and even full task flows without any input from a user. This is particularly useful for identifying problems with lesser-used parts of an application.
The common use-cases and benefits
There are a variety of situations where synthetic monitoring can be beneficial. Not only can it allow you to simulate many of the situations your customers may experience, but it also allows you to measure performance in ways that are more consistently reliable than real-user monitoring. Let's look at a few of them.
Very common and low effort, uptime testing is a common use case. By hitting specific servers, endpoints, or URLs on a regular basis, you can automate the process of ensuring that a service is up and running.
Allows for easy benchmarking
Because synthetic monitoring isn't affected by the variability of user connections and hardware, you can generate a more accurate baseline of what normal performance looks like. This allows you to benchmark services with the same set of constants. The performance information in this situation is particularly useful for comparing resources, perhaps across different regions, or even for comparing different third-party service providers.
Since spinning up global servers is easier than ever, you can take advantage of a distributed testing suite to more accurately predict performance issues that your core team or audience may never experience. Many companies start with a narrow user base, generally located in one or two regions. There is the potential to only hear about problems within that region, while completely ignoring performance concerns in other parts of the world. Synthetic monitoring allows you to identify issues globally without any adjustments other than the location of your monitoring server. When speaking about servers connecting to other servers, this may not be as valuable compared to end-users connecting to servers, but it can allow you to make better decisions on where to locate your own resources. For example, if most of your API dependencies are located in a specific region, it may be more beneficial to locate your resources there as well.
Where real-user monitoring requires traffic to exist from real users, synthetic testing enables you to debug problems on your schedule. If you have a lower user base, or you're trying to investigate issues with a specific service that isn't used by all of your customers, having the ability to trigger a synthetic test and monitor it can make debugging considerably easier.
Synthetic monitoring and testing allows for an interesting benefit over the traditional "check if a resource is working" approach. Developers can simulate full task flows, or transactions, that a user might take. This allows you to test code logic, dependencies related to the task, and API calls all in one contained piece. This is useful for distinguishing between an error caused by user data versus an error caused by an API outage.
Simpler SLA accountability
When consuming third-party APIs, service level agreements (SLAs) are the contract between you and the provider. Many ensure a minimum uptime guarantee, but that can be hard to calculate on your own. Creating a schedule of synthetic monitoring events to check for availability can make it much easier to hold web service providers accountable.
Downsides versus real-user monitoring
While synthetic monitoring is convenient and puts the control in the hands of the developer, it does have some drawbacks.
Perhaps the largest drawback over real-user monitoring relates to your ability to know which problems to look for. Synthetic monitoring relies heavily on knowing where a problem might be and deeply understand how your app uses services. This is why the concept of coverage is important. Unless you have tests created for all dependencies, synthetic monitoring won't identify every problem. This is especially important for third-party APIs. If specific regions or endpoints are affected, but your tests only cover a subset of them, you won't always know when an issue arises.
Another, related downside, is that synthetic monitoring requires the developer to create all the testing scenarios. The process of running synthetic tests can be automated, but creating them takes time and a plan. As mentioned, full coverage of all API usage, endpoints, and the connections between your app and other services is required to avoid missing a crucial problem. Real-user monitoring on the other hand detects problems that occur as users encounter them. Often through in-app agents.
Your best options
For most companies, the best option is a mix of synthetic and real-user monitoring. They compliment each other nicely, and add a layer of awareness into how your application works. At Bearer, we think real-user monitoring is essential for knowing how third-party APIs are performing and impacting your application. We also know that it's important to use synthetic monitoring to keep tabs on your application's internal resources, and to be proactive with testing third-party services that may not be as widely used within your business.