Test harness administrator guide

Introduction

This is the administrator guide for the PDP test harness consent and authorisation stub. The intended audience is administrators and testers at pension data providers and integrated service providers (ISPs), responsible for setting up and operating the environment for system testing. This system testing is required before onboarding and integration testing.

This document explains how to deploy, configure and operate the consent and authorisation stub to facilitate system testing.

A separate document describes how to run the tests and use the consent and authorisation stub to assert the system-under-test has the correct behaviour in the test scenarios.

The system-under-test makes API calls to the consent and authorisation server. The consent and authorisation stub can be wired up to the system-under-test to receive and respond to these consent and authorisation calls. The consent and authorisation stub is designed to behave in accordance with the test scenarios and test data, and to allow a tester to assert that the correct calls were made to the consent and authorisation stub. It also supports the collection of evidence that these tests were executed correctly.

The current version of the consent and authorisation stub is based on technical standards V1.1.

Version

This is version 2.5 of the test harness administrator guide.

Downloads

Download error-scenarios-postman-collection.zip

This zip folder contains:

• 8. MaPS System Testing - Error scenarios.postman_collection (1).json

Prerequisites

The consent and authorisation stub is supplied as a Linux container image. A mechanism for running containers is required - be that a simple machine running Docker or an alternative container runtime compatible with Linux images, or a full container orchestration platform such as Kubernetes.

It is possible to run Linux container images in multiple operating systems, including Microsoft Windows, Apple macOS and Linux.

Alternatively, the consent and authorisation stub can be built from source (for example, as an alternative container image that meets the pension provider’s bespoke requirements, or as a Java application run in a different way). See appendix A: build instructions for more information.

The recommended resource requirements for the consent and authorisation stub container is at least 1GB of memory and 2 CPU cores.

To use mTLS (as per the technical specification), TLS must be enabled for the consent and authorisation API within the consent and authorisation stub. This requires a valid certificate to be injected into the container, issued for the domain where the container will be exposed to the system-under-test. See mTLS Configuration for more information.

No outbound network connectivity is required from the consent and authorisation stub. However, the container’s http(s) endpoint must be reachable (as an inbound connection) from the system-under-test. Note that this outbound network connectivity requirement may change in a future version of the consent and authorisation stub, when mTLS OCSP support is added.

Running the consent and authorisation stub

The following section covers how the consent and authorisation stub can be run as a Docker container. These instructions can be adjusted for alternative container orchestration platforms.

Container image download

The container image is provided by MaPS to all ISPs and pension data providers.

The instructions below assume the participants have been supplied a password directly from MaPS (<password> in the example command below).

To pull the container image from the repository, first authorise access to the container registry:

Then pull the image (download it locally) using the following command, for the desired version (note that latest can be used to get the most recent container image; or 20240927162317 can be used from PDP test harness):

Note: subject to updates

To confirm the image has been correctly downloaded, list the images with the following command:

The output should look something like below:

Note: subject to updates

Running the container

The consent and authorisation stub container can be run using the following command (explained in more detailed in the next configuration section):

Note: subject to updates

The expected output is shown below:

Note that the run command above binds a local directory casTestLogs to the container logs directory for convenient access. Further information on logs inspection is described in the following sections.

Container configuration

As a standard container, the consent and authorisation stub supports all the normal container configuration (see docker run documentation).

The most important configuration options are documented below.

Ports

Within the container, the consent and authorisation stub API listens on TCP port 8443 and 8081 for https and http traffic respectively. These ports need to be exposed on the host that it is running on (or via alternatives like Kubernetes networking configuration), in order to connect to it from the system-under- test.

To map this to a port on the host, you can use the docker -p argument, such as in the example below:

mTLS configuration

For further details of mTLS in system testing, see the mTLS guide. Also see additional documents from PDP about mTLS, which will give the authoritative description of mTLS in PDP.

This section describes establishing https mTLS transport channels between the system-under-test and consent and authorisation stub.

It is assumed that the administrator has a good understanding of mTLS.

High-level overview

The system-under-test is the “client”, and the consent and authorisation stub is the “server”.

• the client will verify that the server has a valid TLS certificate. It is expected that this is signed by a public trusted certificate authority
• for the client to send requests to the server, the client will use the private key and certificate issued in the crypto pack (signed by the PDP certificate authority Root)
• the consent and authorisation stub will validate that the client’s certificate is signed by the trusted certificate authority Root. If not, it will reject the TLS connection. Note that the consent and authorisation stub’s truststore is pre-populated with the PDP certificate authority Root’s certificate chain

Consent and authorisation stub configuration

The consent and authorisation stub needs to be configured with a server certificate for TLS. This certificate must match the domain on which the consent and authorisation stub will be reached.

It is assumed that this certificate will be obtained by the participant from a public certificate authority (if your organisation requires use of a private certificate authority for test environments, see the appendix in the mTLS guide).

The server certificate must be injected into the container at start-up time. This is done by mounting the certificate file (PEM format) and private key (with no passphrase). The expected paths within the container are: /server-cert/server-certificate.pem and /server-cert/server- certificate.key. For example, if the files on the on the host are in the current working directory, called server.pem and server.key respectively, then start the container with:

When the consent and authorisation stub container starts, it will write to the console a message about the certificates in use, and that it is listening for https connections on port 8443.

Disabling TLS and mTLS

For the consent and authorisation stub API, mTLS is enabled by default. For disabling TLS, and therefore mTLS, the environment variable ENABLE_SSL=false can be injected. Only http traffic through port 8081 will be available.

If TLS is disabled, binding the server certificate can be ignored.

It’s possible to disable only the client authentication (mTLS), keeping TLS. It can be achieved by injecting instead the environment variable ENABLE_MTLS=none.

Use of HTTP

Even when mTLS is enabled, the consent and authorisation stub will still also accept API calls to the unencrypted HTTP endpoint. This is to facilitate troubleshooting (for example to allow a tester to send requests to the consent and authorisation stub when exploring why a test scenario failed).

The call log will clearly indicate whether http was used, or https with mTLS.

The other features of the consent and authorisation stub (for example the web-console for browsing the database) will use only HTTP.

Exposing the volume for log files

Within the container, the consent and authorisation stub writes logs to the directory /cas/logs/. You can bind mount this directory to a directory on the host, to make the log files easier to access and to persist the logs beyond the lifetime of the container.

The example below mounts the host’s directory ./casTestLogs into the container at the path /cas/logs where the logs are written, by using the standard -v flag.

Database credentials

The consent and authorisation stub has an internal database to track creation/modification/deletion of resources. This can be inspected to assert the expected behaviour of the system-under-test.

The default database username is sa, with no password needed.

However, the administrator can configure a custom username and password for the database. This is done by passing environmental variables to the docker run command:

Usage

The consent and authorisation stub’s URL will depend on the domain name of the host it is run on, and the port it is exposed on. For simplicity, the documentation below assumes the domain name “example.com” and port 8443, with a client-trusted server certificate.

The consent and authorisation stub API

The consent and authorisation stub exposes the API, as per the technical specification. For example, the endpoint for ‘rreguri’ calls is https://example.com:8443/rreguri.

However, note that communication is over HTTP (not HTTPS, and not mTLS).

For the system-under-test to communicate with the consent and authorisation stub, the administrator must configure the system-under-test with the URL for reaching the container’s endpoint. The system- under-test must also be configured to accept the HTTP connection, rather than enforcing mTLS.

It is expected that HTTP requests are sent to the consent and authorisation stub by the system-under- test, rather than directly. However, a tester can send requests directly to check that the consent and authorisation stub is running correctly, and/or to troubleshoot errors. See the open API console section.

Accessing/viewing the database

Console-access to the internal database is at https://example.com:8443/data. For standard documentation on using this database, see the H2 database quickstart documentation.

The login form is pre-populated to match the default configuration:

• the user name and password (default user name is sa, with no password)
• the JDBC URL’s default is: jdbc:h2:file:./db/cas

Once the user is logged in to the database client, they will be able to see the relevant User table (CAS_USER) and Resource table (CAS_REGISTERED_RESOURCE). and to query them using standard SQL queries.

Viewing the records will help the users to analyse whether the relevant test scenarios have been run correctly and corresponding records have been added/updated/deleted as required.

Log files

The consent and authorisation stub helps testers to assert the system-under-test behaved correctly. This includes looking at the call log to see the http requests and responses.

The log file for this is named http.log (see configuration information in the section exposing the volume for log files for details of how to mount this volume in the container). This file contains all HTTP requests and responses. See the separate testing guide for how to interpret that log.

Additionally, the http requests are printed in the console output of the docker run command (if not using detached mode).

The file server.log is also created. This can be used for troubleshooting purposes, if there are issues with the consent and authorisation stub.

Open API console

The Open API console is at https://example.com:8443/swagger-ui/index.html, providing a console for making API calls against the endpoints.

This console can be used to send HTTPS requests directly to the consent and authorisation stub. This will typically not be used during system testing, the requests will be sent by the system-under-test.

However, it may be useful when investigating failing tests to send requests manually to the consent and authorisation stub. This allows individual requests to be tried, for example to see why a given response is returned and to explore variants of the request.

However, it is possible to manually make the HTTP requests for test/troubleshooting purposes. This can be done in multiple ways, either by using the provided Open API console, or a third party software allowing to make HTTPS or HTTP requests (for example, cURL, Postman).

To make the HTTP request using the Open API Console, navigate to https://example.com:8443/swagger-ui/index.html. This displays a user friendly UI allowing to make calls to all endpoints with customised input data. Note that some requests require sending the PAT token in the Authorization header to authorise the request, which can be provided by clicking on the Authorize button as seen in the top right corner of the screenshot below:

Troubleshooting

Troubleshooting the container

The server.log file (see log files section) contains details from the server, including warnings and errors. This is a recommended starting point when investigating issues with starting/running the container.

It is also worth verifying that the container is still running correctly. For example, if using Docker on the host then the command below will list all containers, and show their status:

Network connectivity

As described in the section on port configuration, within the container it listens on port 8443 and 8081. This needs to be mapped to a port on the host, or if running in a container orchestration platform such as Kubernetes then mapping this pod’s port to an externally accessible endpoint.

As described in the section on the consent and authorisation stub API, the URL (domain name and port) will depend on how and where this container is run, and where it is accessed from.

A simple way to check network connectivity is the curl command below, which should return a 200 response (this example command uses example.com, so assumes it is being run from the host where the container is running):

To see if the HTTPS port is reachable, the curl command below can be used. However, when using mTLS this connection should be rejected (but the rejection will still look different from if there were no web server listening on that port).

cURL can be used also for testing mTLS connectivity providing the certificate information, in this example, form the provided Crypto Pack for system testing:

Troubleshooting failed tests

For tips on troubleshooting failing test scenarios, see the separate testing guide document.

Additional material

This document is not intended as an introduction, or as instructions for how to carry out the system testing. As such, it complements the other PDP material from MaPS.

Appendix A: build instructions

The main way to run the consent and authorisation stub is to use the pre-built container image. However, it is possible to build the consent and authorisation stub from source, and thus to package it in whatever way is desired.

The consent and authorisation stub is a Spring Boot app. It requires Java version 21, and is built using Gradle 8.7 or newer.

Code repo

There is a git repository for the consent and authorisation stub, which includes instruction files for building and details of the code design of the consent and authorisation stub.

This git repo can be found at: https://github.com/moneyadviceservice/pdp-test-harness/

To clone the git repository, the user needs to authorise access and then execute the following command:

The main branch contains the very latest code - this could include code under active development. Each released version is tagged. For example, to view all the tags, run the command:

To view the source code for version 1.1.2, run the command:

Container image build instructions

To build the consent and authorisation stub, and to package it as a container image, run a command like that below:

The dockerfile contains the build commands used to compile the code, and the commands used to create the container image.

If an organisation has requirements that mean the pre-built container image cannot be used (For example, only pre-approved hardened base container images can be used), then this Dockerfile can be adjusted as appropriate to build and use a custom container image.

Java build instructions

The project can be built directly (not using Docker) using Gradle. These commands assume that Java 21 and Gradle 8.7 or newer are pre-installed.

The JAR file is created in ./build/libs/.

Running the JAR file

If building the Java JAR file from source, one can run the application directly. The command below will run the application:

Environment variables can be passed in the standard way with the -D argument. For example, the command below configures the listener port, database user and database password:

With the application run using the JAR file, the log files (http.log and server.log) are available under ./logs/directory, relative to where the JAR file is run from.

Baking in the server certificate into a new container Image

When running the consent and authorisation stub container in a cloud environment or in an on-prem Kubernetes cluster, the need to mount the server certificate and key files complicates things.

A workaround to this is to build a new container image, using the existing container image as the base. This allows one to bake in the server certificate and key files into a new container image, for example with Dockerfile commands like those below:

The Dockerfile in the code repo shows how the current image is built.

Appendix B: hosting the consent and authorisation stub container

For hosting the consent and authorisation stub container, requirement include:

1. The consent and authorisation stub container must be run somewhere that the system-under- test can reach it.
2. The consent and authorisation stub must have a valid server certificate that matches the domain.
3. The server certificate and key must be injected into the container at start-up by attaching a volume (alternatively, see Baking in the server certificate into a new container Image).
4. The consent and authorisation stub must be able to do mTLS client authentication. This means one cannot put a load balancer or reverse proxy in front of it, which would do TLS termination.

There are many ways to run a container (in the cloud, or on-prem). However, the requirements above can make this complicated.

A simple way to run the consent and authorisation stub is to run a virtual machine with the container on it. In brief, this would require the following steps:

1. Provision a VM (here we assume Linux).
2. If the VM does not have a domain, then create a DNS A-Record (using the IP of your VM that is reachable from the system-under-test).
3. Generate a server certificate for the VM’s domain (for example, one could use Let’s Encrypt; note that domain validation will require an inbound network connection to the VM from Let’s Encrypt).