System test pack

Introduction

These guidelines assist participants through the system testing phase to ensure the required quality standards are met in preparation for acceptance into the PDP pre-production environment for integration testing.

The approach covers where system testing fits into the onboarding process, reference documents to be used in conjunction with the test pack, test script and test data high-level information, the benefits of using the Pensions Dashboards Programme (PDP) test harness, and the support process offered.

The test harness administrator guide, test harness testing guide, detailed test scripts including test data, test evidencing submission document and defect form will be provided as separate documents.

Downloads

Download system test pack materials

This zip folder contains:

• PDP VP system test cases v1.9.xlsx
• MaPS System Testing.postman_collection September 2024.json
• PDP system testing evidence v1.3.docx

Context

System testing for successful onboarding

The first phase of participant system testing will be based on technical standards v1.1.

Test scripts, test data and the test harness will all be based on the same version. There will be a requirement to perform additional testing against future technical standards and reporting standards, but these will be separate to this current system test phase.

This test pack will be used to assist participants with their testing development and test execution during system testing, a separate test pack will be issued for integration testing in the PDP pre-production environment.

Test phases

Participant testing and test execution within the connection journey

The system testing development and test execution phases performed in the participants own environments are supported by this test pack.

The process to support the integration testing phase within the PDP pre-production environment will be documented in the integration test pack.

1. Local development and unit testing.
2. System testing.
3. Pre-production connection.
4. Integration testing.
5. Operation acceptance testing.
6. Production connection.

End-to-end connection journey

Reference documents

Before embarking on system testing, participants need to be completely familiar with the standards documents:

1. Data standards.
2. Technical standards (including supporting JSON schemas and OpenAPI specs)
3. Reporting standards.
4. Code of connection.

Draft versions of standards can be found on the PDP website.

Test pack contents

Test cases

Test cases for participant system testing have been written against technical standards v1.1 and will be provided in a separate document.

The test case spreadsheet contains 7 work sheets:

• Document _CTRL: Document control listing the reference documents and versions used.
• PRE_TEST_SETP: Details the pre-test setup for the resource server and postman collection.
• TEST_CASE _STEPS: Lists the 30 test scripts detailing the test scenarios number, functional area, test name, pre-requisites, test data, test steps, expected results and the instructions to resource tester. The test script functional areas of find and view are broken down into 6 areas:
  • find request
  • validate the contents of the find request
  • persist find request
  • update match status
  • delete a resource
  • view request
• TEST_DATA_DETAILS: Contains the test data.
• VIEW_DETAILS_JSON: Details the view details for JSON.
• PUBLIC_KEY-JWT-SIGNATURE: Details the public key for JWT Signature.
• GLOSSARY: Glossary for all terms used in the test script sheet.

Test data

The participants will be supplied with the test data necessary to carry out the system tests. This takes the form of:

• A Postman collection, containing the find and view requests that will be sent to the resource server under test.
• A table of users and their identifiers. Participants will use this to set up data in the resource server under test so that the desired match is achieved by the resource server’s matching criteria when a find request is processed.
• An example of the view data that can be returned. It is not necessary to use this particular example; participants have the option of returning their own view data. What is important is that the view data for a possible match conforms to that part of the schema, and the view data for a full match conforms to that part of the schema.
• The public key for validating the signature in the user_token. This is supplied in 2 ways:
  • In the test case spreadsheet.
  • In the response for a GET request to the JWKS (/jwk_uri) endpoint of the MaPS test harness.

Postman collection

There are find (POST) requests supplied for test cases 1 to 20. These will also be used to provide prior matches for test cases 21 to 30 view (GET) request.

The find requests (shown above as POST) do not require any changes from the Resource Server tester. They will require changes in the Resource Server itself.

The ‘Detailed Instructions’ tab in the ‘PDP VP system test cases.xlsx’ file informs which user needs to have data set up in the Resource Server so that the required match happens. ‘PDP VP system test cases.xlsx’ has a tab ‘Users’ that details the attribute values used for matching for each user.

The view requests (shown above as GET requests) will require changes from the Resource Server tester. This is inevitable, given that the asset guid is generated in real time by the Resource Server.

Once a match has been registered with the Stub Consent and authorisation server, the Resource Server tester should get the resource ID of that match from the Resource Server or from the test harness.

The final view request will also require a valid requesting party access token. For those participants using the test harness supplied by MaPS, there are instructions on how to get the requesting party access token for a specific resource ID in the test harness testing guide.

Table of users

The spreadsheet includes a ‘Users’ tab that lists the details used to match a citizen against a pension.

As every Resource Server has its own data storage and match criteria, it is up to the participant to arrange the extra data to give possible matches and full matches where directed in the test scripts.

Test harness

A PDP test harness is available to participants to assist with system testing in the participants own environment.

Deliverables supporting the test harness include:

• consent and authorisation stub
• source code with build instructions
• test harness administrator guide
• test harness testing guide for how to make assertions in consent and authorisation stub
• mTLS guide
• crypto pack and supplementary crypto material for system testing

Benefits of using the PDP test harness

• improve the speed of connections testing
• decrease the support required during system and integration test phases
• reduction of issues encountered during integration testing
• improved feedback and troubleshooting for failing scenarios

How the test harness delivers the benefits

• consent and authorisation Stub is verified to behave correctly
• all data providers will interact with the same API
• eliminate the risk of a data provider’s own stub implementation having incorrect assumptions or misinterpretation of the spec
• consent and authorisation Stub performs strict validation
• checks that messages sent by system-under-test are valid
• consent and authorisation Stub supports the system testing test pack’s test scenarios and test data
• behaviour of the consent and authorisation Stub will match what is needed for each test scenario (for example, a given scenario and test data simulating the necessary error responses)
• consent and authorisation Stub can support advanced scenarios (for example, error handling and retry scenarios)
• errors will be hard to simulate and test in the integration environment
• the test pack, combined with the consent and authorisation Stub can test these
• consent and authorisation Stub can contribute towards evidence of testing (a prerequisite for integration testing)
• the “call log” shows what happened during the test scenarios

Test harness administrator guide

The test harness administrator guide is the administrator guide for the PDP test harness consent and authorisation stub.

The intended audience is administrators and testers at pension data providers and integrated service providers (ISPs), responsible for setting up and operating the environment for system testing.

The document explains how to access, deploy, configure, and operate the consent and authorisation stub to facilitate system testing. This includes instructions on how to access the pre-built image.

The prerequisites section notes a mechanism for running containers is required. This could be a simple machine running Docker or an alternative container runtime compatible with Linux images, or a full container orchestration platform such as Kubernetes.

Participants need to choose the mechanism which meets their organisational need.

Alternatively, the consent and authorisation stub can be built from source. The guide provides relevant information to support as needed

The user credentials to access the image will be required and should be requested from MaPS.

Test harness testing guide

The test harness testing guide describes how to run the tests and use the consent and authorisation stub to assert the system-under-test has the correct behaviour in the test scenarios.

The system-under-test makes API calls to the consent and authorisation service. The consent and authorisation stub can be wired up to the system-under-test to receive and respond to these consent and authorisation calls.

The consent and authorisation stub is designed to behave in accordance with the test scenarios and test data, and to allow a tester to assert that the correct calls were made to the consent and authorisation stub.

It also supports the collection of evidence that these tests were executed correctly.

mTLS and Online Certificate Status Protocol (OCSP) certificate revocation

Mutual TLS (mTLS) is used in PDP to ensure that a server will authenticate the client when a TLS connection is being established. This is enforced throughout the PDP architecture. Using mTLS to authenticate clients relies on a chain of trust checking that the client’s certificate is signed by the PDP certificate authority.

The test pack supports mTLS, in a similar way to the real system. Benefits of this include:

• testing the use of mTLS during system testing will give earlier feedback, helping to reduce errors and delays during the subsequent onboarding and integration testing
• without mTLS support, the system-under-test would have to be configured into a “test-mode” that did not enforce mTLS

Additional information is provided as part of the mTLS guide to assist participants with system testing.

Note that mTLS OCSP is not supported for system testing in the test pack.

Submitting test evidence to PDP

System test evidence will be required to be submitted through Salesforce for checking by the PDP test team prior to approval being granted to perform integration testing in the PDP pre-production environment.

Evidence must be collected in a Word document (or equivalent), exported as PDF, and submitted to MaPS via the standard Salesforce approach. Given the file limit size of 25MB, please copy-paste the relevant text to show the success, rather than using screenshots unless the specific test case specifies to attach the screenshot.

A test evidencing document provided by PDP, will need to be submitted covering test results for all the test scripts provided by PDP.

The test evidencing form criteria will cover:

• test script numbers
• tick box confirming if the PDP test harness was used in executing the test script
• pass/fail result
• call log evidence from the PDP test harness

Guidance on the prerequisite process for additional information is displayed along with the upload document section to attach the system testing execution evidence.

On receipt of the test evidence report submitted through Salesforce, the PDP test team will advise if the test results have been accepted.

Support

Throughout the development and execution of participant system testing, the PDP testing team will provide testing related support.

Contacting the PDP test team

Throughout system testing and integration testing, participants will be required to use the PDP support hub for all testing related defects and to raise test related queries.

The web page will display two options, “common queries and feedback”, and “technical support”.

For test related questions select ‘common queries and feedback’. PDP will run regular testing sessions with participants to discuss and raise queries.

To raise a defect, select ‘technical support’ and follow the instructions presented to complete a ’raise defect’. (Do not select ‘technical support for our live services’).

Select ‘raise a test defect’ option which will present a defect form to complete within JIRA. (Note: you will need to create a JIRA account if you do not already have one).

Raising defects

Participants are required to complete a 'defect form' to raise defects during all phases of testing. Only defects related to the central digital architecture, PDP test harness, PDP test scripts and PDP test data can be accepted.

Required information includes:

• reported by (company name)
• defect title
• PDP test case number (if applicable)
• date reported
• summary
• defect details
• expected results
• actual results
• steps to reproduce the defect
• URL (if applicable)
• screenshots or supporting images
• additional notes

The defect information will then be used to raise defects in PDP Jira.

Defect management

A PDP defect analyst will manage all defects raised by participants through triage, development, fix, retesting and closure.

Queries and observations received from participant testing and connections will be triaged to determine if they are a valid defect, the severity and priority will be set, and the defect will be assigned for fixing.

The PDP test team will produce a weekly report detailing all defects raised and progress towards fixing, retesting and deployment, which will be available to all participants.

Context diagrams

The 3 context diagrams show how the environments for system testing, integration and production vary with the introduction of the test client, IDP/Cognito, and PFS for integration testing and then the addition of Salesforce for production.

Figure 1: PDP system testing context

Figure 2: PDP integration testing context

Figure 3: PDP production context

Changelog

First published:04/10/2024

Last updated:08/11/2024