Central digital architecture “normal response times” for APIs

Background

The code of connection sets out some requirements of pension providers and schemes regarding technical response times. Additionally, the technical standards define the actual interactions which pension providers and schemes must make within the CDA to support core journeys for pensions dashboards.

When combined, these standards form the basis of what the pension provider or scheme must do and stipulate any constraints which they must operate in when performing these tasks.

As an example, the technical standards show that a pension provider or scheme must first use the CDA JWKS API endpoint when responding to a find pension request, to ensure the request has legitimately come from the CDA itself. CoCo2.1.1 adds that 99.9% of operations must respond (by means of ACK) to any find request within 2 seconds of the request being received.

The code of connection requirements were written on the assumption that the CDA is working under “normal” operating conditions, and therefore the pension provider or scheme would have sufficient time to complete its work as per the technical standards and code of connection requirements. These timings are designed to give pension providers and schemes sufficient time to perform their tasks, whilst providing an ecosystem which can be used to deliver performant dashboard services for users.

However, to date there has not been guidance as to what happens when the CDA is not performing under “normal” operating conditions, how a pension provider or scheme can tell this is the case, and what they should do in this scenario. In the above example, if the JWKS API call itself does not respond within 2 seconds, then clearly the pension provider or scheme cannot meet its obligation to respond within 2 seconds at no fault of its own. This guidance aims to provide clarity on this issue.

Code of connection – Affected APIs

There are 3 main areas where the code of connection specifies required response times but where response actions first involve making API calls into the CDA:

• CoCo2.1.1: Must acknowledge receipt of find requests by the find interface, by means of ACK (see technical standards) in <2 seconds (99.9% of ACK responses to acknowledge receipt of a find request to be returned in <2 seconds, measured over a 24-hour period).
• CoCo2.1.2: Must complete responses to find requests, including registering any pension identifiers (see technical standards) within 60 seconds for positive matches (including both matches made and possible matches; negative responses are not required) following sending of the ACK.
• CoCo2.1.3: Must respond to view requests in <10 seconds (99.9% of view data payloads retrieved from systems and returned to the dashboard that issued the view request in <10 seconds, measured over a 24-hour period).

For each of these calls, there are different CDA API endpoints which must be called, which are listed below:

(*) Covers path where valid RPT has been provided

As can be seen from the above, multiple CDA API endpoints must be called to meet a single overarching response time. For example, OSCP, token and rreguris are all required as part of the PEI registration journey.

Each API will naturally have different “normal” response times to each other based on the nature of their operations. For example, the rreguri endpoint will make an update to the CDA database, which should typically take longer than the JWK API, which is a read-only API where the CDA can use caching to improve performance.

Internal CDA benchmarking

The CDA is an internet-facing service, where response times are influenced by factors beyond the CDA itself. Network latency, routing, geographical distance between the pension provider or scheme system and the CDA, internet service provider, performance, internet congestion, mTLS connection establishment, and the use of intermediary components such as proxies, firewalls, and content delivery networks can all contribute to the overall time observed by a consuming application. Response times, measured from a client system to a server typically represent end-to-end performance under this variety of operating conditions rather than the processing time of the service alone run in isolation.

Each pension provider or scheme will have its own set of factors in play when calling the CDA, and such benchmarking “normal performance” of API calls to the CDA will differ between each provider and scheme. To deliver a common baseline, MaPS has benchmarked performance of the CDA when run in a controlled environment, giving a baseline view of how the CDA performs without the variation in network path and devices employed at each pension provider or scheme. This has been executed against every API used to support a timed code of connection requirement.

These figures are internal performance testing figures and cannot be taken as firm values for universal CDA API response times, where (as mentioned) access to the CDA is influenced via all factors employed in the full round trip taken from client to server in place for each pension provider or scheme. As such, these internal performance figures cannot be taken as firm values for universal CDA API response times, but are useful inputs for validating pension providers' and schemes' own benchmarking activity (see later in this document).

Our benchmarking uses the top 95% of API performance (P95) for assessing “normal” response times over a specific time window (in this case, over one day). P95 is the 95th-percentile response time, where 95% of measured successful responses complete at or below that duration.

Benchmarking “normal” CDA response times

Given that the CDA is an internet-facing service, a “normal” CDA response time for each pension provider or scheme would be dependent on not only the CDA performance but also the specific network routing which is used to connect these client systems to the CDA. Industry should additionally perform its own benchmarking for the general performance of the CDA which they see within their own production systems when connecting to the CDA, where all factors which can influence actual performance can give a representative view of what “normal” performance is at their own systems.

For consistency, pension providers' and schemes' own benchmarking should also use the P95 approach, where 95% of measured successful responses complete at or below that duration. The pension provider or scheme can use the values shown in the internal CDA benchmarking to validate the baseline figures they see are representative and reasonable given the underlying CDA P95 performance.

This benchmarking activity can be used to assess whether the CDA took longer than normal (that is, longer than the benchmarked P95 performance speeds) during a pension provider or scheme operation when in production. However, it should be noted that a P95 benchmark represents a performance baseline where approximately 5% of observations in the baseline distribution exceed this figure. Therefore, a single occurrence of a CDA call exceeding this P95 figure would not, in itself, demonstrate that the CDA was operating abnormally. Instead, more than 5% of all CDA calls during a given period would need to exceed this figure for the CDA to be classed as not performing to “normal” operating conditions.

Use of caching

Caching is an effective way to reduce the reliance on CDA APIs where the pension provider or scheme system needs to perform an operation within a specific duration. Within the context of this document and the associated code of connection requirements, the only API response which could be cached at the pension provider or scheme end is the JWKS endpoint, where caching is actively encouraged, as per the guidance on Public Key Infrastructure (PKI) and cryptographic materials:

This document builds on this guidance in stipulating that, where caching is used, caches for keys returned in the JWKS endpoint should be refreshed at 24-hour intervals.

What to do when the CDA is not operating in “normal conditions”

Where pension providers and schemes identify that the CDA is not operating within “normal conditions” to the tolerances stated against each code of connection requirement, pension providers and schemes should use this as a basis to investigate where the delay occurred. Given (as stated previously) that the overall “normal response times” of the CDA is influenced by both the CDA and internet-based considerations within the pension provider or scheme’s systems, analysis and diagnosis should occur to determine if the delay was at the CDA end given the internal benchmarking stated in this guidance. Industry should consider what appropriate monitoring and observability tools are best placed to perform this analysis.

Monitoring ongoing performance

The benchmarking figures presented from internal testing are with simulated user loads. We will continue to monitor performance with real users as we scale the service. We will update this guidance as necessary in response to ongoing performance monitoring.

Support