Reporting system test pack

Overview

This guidance assists directly connected organisations (DCOs) through the system testing phase to ensure the required quality standards are met in preparation for acceptance into the Pensions Dashboards Programme (PDP) pre-production environment for integration testing. 

The approach covers where system testing fits in the reporting process, the reference documents to be used in conjunction with the test pack, test scripts, the benefits of using the PDP test harness, and the support process offered.

The test harness administrator guide, test harness testing guide, detailed test scripts and test evidencing submission document will be provided as separate documents (see 'Downloads' below). 

Downloads

Download reporting system test pack materials

This zip folder contains:

• Reporting_System_Testing_Scenarios_v1.1.xlsx 
• PDP Combined technical and reporting standards system testing evidence v1.1.docx

Context

Reporting system testing

This phase of DCO system testing will be based on the reporting standards version 2.2. 

Reference documents

Before starting reporting system testing, DCOs need to be completely familiar with the standards:

1. Data standards
2. Technical standards (including supporting JSON schemas and OpenAPI specs) 
3. Reporting standards
4. Code of connection

The standards can be found on the PDP website.

Test pack contents

Test cases

Test cases for DCO reporting system testing have been written against the draft reporting standards v2.2 and are included in the test scenarios spreadsheet. 

The spreadsheet contains 1 work sheet: 

• System Test Scenarios: Lists the 42 test scripts detailing the endpoint, TC ID, test definition, criteria of test and expected results. The test scripts cover the following 6 endpoints:
  • POST /service-availability/find
  • POST /service-availability/view
  • POST /view-response/request-number
  • POST /view-response/response-time
  • POST /view-response/calculations
  • POST /view-response/unavailable

Test harness

A PDP test harness is available to DCOs to assist with system testing in the DCOs’ own environment. 

Deliverables supporting the test harness include: 

• CDA stub 
• source code with build instructions
• test harness administrator guide
• test harness testing guide for how to make assertions in CDA stub
• mTLS guide
• cryptographic pack and supplementary cryptographic material for system testing

Test harness outcomes

• CDA stub is verified to behave correctly
• all DCOs will interact with the same API
• eliminate the risk of a DCO’s own stub implementation having incorrect assumptions or misinterpretation of the spec
• CDA stub performs strict validation
• checks that messages sent by system-under-test are valid
• CDA stub supports the system testing test pack’s test scenarios and test data
• behaviour of the CDA stub will match what is needed for each test scenario (for example, a given scenario and test data simulating the necessary error responses)
• CDA stub can support advanced scenarios (for example, error handling and retry scenarios)
• errors will be hard to simulate and test in the integration environment
• the test pack, combined with the CDA stub can test these
• CDA stub can contribute towards evidence of testing (a prerequisite for integration testing)
• the “call log” shows what happened during the test scenarios 

Test harness administrator guide

The test harness administrator guide is the guide for the PDP test harness CDA stub. 

The intended audience is administrators and testers at DCOs, responsible for setting up and operating the environment for system testing. 

The document explains how to access, deploy, configure, and operate the CDA stub to facilitate system testing. This includes instructions on how to access the pre-built image. 

The prerequisites section notes a mechanism for running containers is required. This could be a simple machine running Docker or an alternative container runtime compatible with Linux images, or a full container orchestration platform such as Kubernetes. 

DCOs need to choose the mechanism which meets their organisational need. 

Alternatively, the CDA stub can be built from source. The guide provides relevant information to support as needed. 

The user credentials to access the image will be required and should be requested from MaPS. 

Test harness testing guide

The test harness testing guide describes how to run the tests and use the CDA stub to assert the system-under-test has the correct behaviour in the test scenarios. 

The system-under-test makes API calls to the CDA. The CDA stub can be installed in a location accessible to the system-under-test to receive and respond to these PDP API calls. 

The CDA stub is designed to behave in accordance with the test scenarios and test data, and to allow a tester to assert that the correct calls were made to the CDA stub. 

It also supports the collection of evidence that these tests were executed correctly. 

mTLS and Online Certificate Status Protocol (OCSP) certificate revocation 

Mutual TLS (mTLS) is used in PDP to ensure that a server will authenticate the client when a TLS connection is being established. This is enforced throughout the PDP architecture. Using mTLS to authenticate clients relies on a chain of trust checking that the client’s certificate is signed by the PDP certificate authority. 

The test pack supports mTLS, in a similar way to the real system. Benefits of this include: 

• testing the use of mTLS during system testing will give earlier feedback, helping to reduce errors and delays during the subsequent onboarding and integration testing 
• without mTLS support, the system-under-test would have to be configured into a “test-mode” that did not enforce mTLS 

Additional information is provided as part of the mTLS guide to assist DCOs with system testing. 

Note that mTLS OCSP is not supported for system testing in the test pack. 

Reporting standards endpoint testing

There are 6 new endpoints defined in the reporting standards which have been added to the stub CDA:

• /view-response/request-number (number of view requests received)
• /view-response/response-time (view response times over 10 seconds)
• /view-response/calculations (view response pension data unavailable calculations with specific deadlines)
• /view-response/unavailable (view response pension data unavailable calculations without specific deadlines)
• /service-availability/find (list of find endpoint outages in the previous day)
• /service-availability/view (list of view endpoint outages in the previous day)

A seventh endpoint that is stub-only has also been added to allow the DCOs to test their error handing. This /configuration endpoint allows any of the 6 reporting endpoints listed above to return specific error responses a set number of times. The endpoint can also be used to reset the reporting endpoints back to correct behaviour.

All the reporting standards endpoints will only accept the POST method.

The CDA stub provides an output of logs which will be copied for test evidence.

Submitting test evidence to PDP

System test evidence will be required to be submitted through the connection portal for checking by the PDP test team prior to approval being granted to perform integration testing in the PDP pre-production environment. 

Evidence must be collected in the reporting system testing evidence document. Given the file limit size of 25MB, please copy-paste the relevant text to show the success, rather than using screenshots unless the specific test case specifies to attach the screenshot. 

A test evidencing document provided by PDP will need to be submitted covering test results for all the test scripts provided by PDP. 

The test evidencing form criteria will cover:

• test script numbers
• tick box confirming if the PDP test harness was used in executing the test script
• pass/fail result
• call log evidence from the PDP test harness 

On receipt of the test evidence report submitted through the connection portal, the PDP test team will advise if the test results have been accepted. 

Support

Throughout the development and execution of DCO system testing, the PDP testing team will provide testing-related support for queries raised via the support hub.

On the support page, select “Support for organisations connecting to pensions dashboards”.

For test-related questions, select ‘Queries and feedback’.

Defect management

A PDP defect analyst will manage all defects raised by DCOs through triage, development, fix, retesting and closure. 

Queries and observations received from DCO testing and connections will be triaged to determine if they are a valid defect. The severity and priority will be set, and the defect will be assigned for fixing. 

Changelog

Last updated:01/05/2026