Webinar: Phase 1 research findings for the MoneyHelper Pensions Dashboard

Chris Curry

Okay, it's now one minute past, so I think it's time for us to get started. Welcome to this afternoon's webinar. Thank you very much for joining us.

My name is Chris Curry. I am principal of the Pensions Dashboards programme, and it's a pleasure to see so many of you, over 250 so far, joining us today.

And so the session we're running today is about user testing of the Money Helper Pensions Dashboard. As you may know, our phase one research is now concluded.

And for this session, I'm pleased to be joined by Adam Gifford, who is a policy and propositions lead at the Money and Pensions Service, who will walk through the key findings from phase one.

So these will include outlining our approach to testing with real users, where we'll cover the methodology used, the scale of the research, participant characteristics and the limitations to be aware of.

Key insights, such as what the research told us, what worked well, and of course, some of the challenges that we've encountered and users have encountered during testing and the actions we're taking as a result of these findings and the next steps.

And as always, with these sessions, there'll be some time at the end for Q&A.

Just to note that today's session is being recorded and we will publish both the recording and the slide deck on the PDP website after this event.

If you have any questions for that Q&A session that I mentioned, please do submit them using the Q&A tool in Teams. Please don't use the chat because we won't be monitoring it. So please do it through the Q&A tool. And we'll do our best to address as many of these as possible following the presentations.

If there are any that we're unable to cover, either where further consideration is needed or because we've run out of time, I know there generally are lots of questions here, then we will take those away and we will publish responses to those questions as well.

So a full set of questions and responses will be published alongside the recording and slides on our website at a later date.

So with that, end of the housekeeping, I'm delighted now to hand over to Adam to get us started with the presentation.

Adam, over to you.

Adam Gifford

Perfect. Thanks, Chris. And good afternoon, everyone. Clair, if you could move us on to our first slide, the next one, if that's okay.

So we thought we'd kick off today's session just with a quick reminder. Sorry, if you go back to just a quick reminder of the journey that we've been on in terms of our user testing.

So as many will know, but some may not, we actually began our user testing back in the summer of last year, where we started with some internal testing to begin.

That entailed putting some internal project people through the end-to-end service. This was really the first time that anyone had ever used the service in anger. So it was really important that we made sure that all of the end-to-end systems worked and could talk to each other in that journey.

So could we, for example, pass somebody through from the Money MoneyHelper lending page through into One Login? Could they verify their identity? Could they be returned? Could we then add some additional information, notably their National Insurance number?

Could we package up all that information, send it through to the pension providers, get them to check their records, confirm that there was a match, and then could we then retrieve all of that information and show it on a dashboard? And we found that whilst there was a few niggles here and there, overwhelmingly that period of testing went pretty well.

That gave us then the confidence to move on to what we referred to as industry expert testing, where we were then able to build our confidence of the end-to-end connectivity of the different systems.

But really it was then a first chance for us to get to and understand whether the data being returned was as expected from the data providers.

Once we were able to get through that testing, that really then built further confidence and allowed us to move into this first phase of consumer testing, which I'm going to be providing the findings for today.

And then that's paved the way for us then moving into the second phase of high volume testing, which is where we are right now. And we'll be then providing a further report in the coming weeks, months about how the second phase is going. So Claire, if you could move us on then.

So just in terms of the testing that we undertook in this first phase, if you can move us on one more. Great. So this first phase took place over a period of three months. And our aim during this phase was really to obtain those deep qualitative insights with a view to ensuring that the service was working as expected.

We wanted to identify any critical pain points and blockers that may impact users early on, particularly anything that would block users from completing key tasks. That way, any key issues could be addressed before we then moved up to and scaled in the second phase of testing.

Now in terms of the sample, we worked with just under 40 participants and the group was deliberately diverse, covering a wide range of ages, regions, levels of pensions engagement and digital confidence.

This helped us ensure that we were testing the service with users who affect real world conditions rather than a narrow or overly digital audience. Now I'll provide a little bit more clarity over the next few slides over the characteristics of our users.

But before we move on, just wanted to outline that most participants are recruited through employees of pension providers in this initial phase. And that was done primarily for speed of access to testers.

However, from round three, we also did work with a specialist recruiter, which allowed us to include users who rely on assistive technology to access digital services. really ensuring that accessibility considerations were embedded into our learnings early on, which is a key consideration for maps and government services in general.

In terms of the how, in order to obtain the rich insights that we needed, we opted to carry out moderated user testing, and this was facilitated by a user researcher, and then tell us watching users go through the end-to-end service, allowing us to observe, probe, and explore their interactions as they progressed through the service.

Now, sessions typically lasted between 60 and 90 minutes, during which we closely documented user actions, behaviours, and comments. This gave us both qualitative insight into user experience and clear evidence of where the performance worked well, but also where it required some improvement. Claire, if you could just move this on. So who do we test with?

Well, as you can see from the screen, testing was carried out over 4 rounds with round one taking place between the 6th and 10th of October. It was made-up of users recruited from three pension providers, namely Willis Towers Watson, Watson, Legal and General and Royal London.

Now, despite our best efforts to recruit people who reported lower levels of pensions engagements, we did actually find, and this was the same round two actually as well, that self-reported levels didn't necessarily match our categorisation upon observation.

Now, this isn't perhaps wholly surprising given they're recruited from providers, because even if they do work not necessarily in the pensions area of the business, we probably could have expected them to have a generally higher awareness of money and pensions than perhaps those that might work in different industries.

So that in itself was a bit of a learning for us from round three forward and we've adapted our recruitment style to reflect that. And around 2 took place between the 27th and 31st of October with users recruited from AJ Bell and Nest.

Round 3 took place 3 weeks later between the 18th and 21st of November. Although a real crucial difference as I've already mentioned was that in this round we saw users that required the use of assistive technology in order to interact with digital services.

And we actually recruited a further 3 participants around 4 with the same characteristics. There was a little bit of a gap between round three and round four, and that was primarily due to Christmas, but also a small technical issue that we wanted to resolve and fix before we went into round four.

That took place between the end of January, so the 28th of January and the 18th of February, and that also included users from British Telecom Pension Scheme. And as you can see from the screen, we saw a difference in device types being used across the testing.

So in the first round, that was primarily laptops. And then we had a mix of mobile laptop and desktop in subsequent rounds, although the overall majority across the rounds was the preferred use of laptops.

And then finally, the key insight probably to highlight is that rounds one to three focused on and return the state pension and defined contribution pensions, with only round 4 returning DB pensions.

If you could move us on, Claire, we'll just give an overview of the participants that we did the testing with over the first four rounds.

So we had an equal split of male and female testers, and the majority were located in England with a couple from Scotland and a couple from Wales.

Unfortunately, we didn't manage to recruit anyone from Northern Ireland, which we do recognise as a gap. So we're actively seeking to address that in phase two, where we will also be continuing with rounds of moderated research in addition to the higher volume unmoderated testing.

We had a split of ages with approximately one sixth aged 50 to 64, around two fifths and indeed the highest proportion between 35 and 49, around one quarter aged 25 to 34 and then finally one twentieth aged 18 to 24, which again we do also recognise as an under-representation in terms of the sort of national averages and we are planning a dedicated round of moderated research with that particular group in phase two.

Last couple of slides then just on the characteristics, if you could move us on one more. So in terms of the digital confidence of our participants, we had a reasonable spread. However, the majority were at the higher end of the digital inclusion scale, as can be seen on the screen now.

In terms of the digital inclusion scale, it goes from a rating of 1, which is somebody who is defined as never having used and never will use digital services, all the way then through to users who are considered experts in interacting with digital services, such as those that use them for work every day and have advanced skills.

It's estimated that around 14% of the population have no or low digital skills and confidence, and just under 10 occasionally reluctant to go online and do specific things, with around 80% of the population then considered to sit somewhere on the processing basic digital skills right through to expert.

Now we assessed most of our participants, as you can see on the screen, as having the basic digital skills and some we considered being confident and some expert with a small number falling in that reluctant and task specific bracket.

In fact, we had one user who was unwilling and unable to actually use digital services independently. If you can move us on. One more, just a final piece to cov er in terms of our characteristics of our participant base. So as you can see on the screen here, we actually rated the user's pension engagement scores.

And you can see that was done in conjunction with an assessment of the UK national average. So notably in round one, as I've sort of alluded to already, we found that users had a much higher pension engagement score, so 22 out of 25, which is obviously pretty high.

Whereas in round three, where we recruited those that require the use of assistive technology, their average score was around 14, which isn't too much higher than the UK national average. And for comparison, the average score that we had on our panel at the end of December was 15, which has now, I think, reduced slightly given that we've now scaled up to a much larger number on our internal panel.

And if you can move someone more, Claire, just in terms of the limitations of this first phase of research, which we do think it is important to call out. So notably, as I've just mentioned, we have high levels of anticipated pensions knowledge and financial knowledge which is something that we'll be looking to address with further testing.

We also had a limited coverage of the pension schemes, limited to State Pension, DC and then with the exception of round 4, a very small number of DB pensions returned, which is again something that we've been actively working to address in subsequent rounds.

And we also, just to call out, have a slightly smaller sample size than we'd planned for, which we had to restrict due to various technical issues along with some time and resource constraints.

Okay, so that's enough about what we did, who we tested with and limitations. Now let's move on to providing an overview of the key insights to be obtained in this space. So if you can move us on, Claire, I think we're going to start with what worked well.

Right. Okay, so I think one of the really key learnings for us is that most users were able to complete the end-to-end journey without support, which also includes navigating the One Login part of the service, which can be a little bit more tricky for some. We found that there was a lot of value placed on and a real positive reaction actually from users in seeing their pensions in one place.

The ability to see both state and private pensions in one place is really highly valued, as well as also the time potential saved from being able to go to one single place rather than having to navigate to numbers, lots of different third party systems in order to gain access to all that pensions data or potentially even have to go through your annual benefit statements to try and pull that together yourself.

In terms of the matching, we found that most users who accessed the service had multiple pensions sharing, which is again quite a positive finding. And we found that most users that went through the end-to-end journey expressed high levels of satisfaction with the service overall and would be highly likely to recommend to a friend, although we'll just put a bit of a hygiene rating across that in terms of that higher domain knowledge that we've called out earlier on.

And so we're obviously monitoring that as we move into the second phase to see whether that also maintains itself at a high rating as we get into a much larger and diverse group of users. The State Pension, we found that users find the information generally valuable, including specifically some callouts that we added to alert users to the fact that they will need to actually claim their State Pension, which many suggested that was something that was new that they needed to do.

And also they recognised then the difference between their State Pension and their workplace pensions a bit better than they did previously, which was a really positive finding. And then lastly, we got some positive feedback from users just on the overall design of the dashboard, as well as generally high levels of trust being expressed in both the GOV.UK and MoneyHelper branding, which I think provides a level of assurance to users in terms of its security and impartiality.

Okay, so that's a summary of what worked well generally. I think we're going to now move on to talking about general pain points encountered through the service. So one of the issues that we encountered was in relation to One Login. As I say, the majority of users were able to get through the end-to-end service, but we did find that some users did struggle a little bit at time, particularly to navigate through the multiple steps that are required as part of that overall journey.

That was more in the case where they actually needed to set up a One Login account for the first time. Where they were, where they'd perhaps already set up a One Login account before and even gone through that verification process, it did work much quicker and easier for people in that situation.

And whilst it could be a little bit lengthy and time consuming for people to get through all of those steps, most users did suggest that they would return. So having persevered, got to the end, they did recognise that, you know, they've now done that.

And so then going forwards, it would be a much simpler process. I think we should also keep in mind in relation to the findings that obviously One Login provides access to multiple government services and therefore by the very nature of the services it's providing and the integrity of it, by design some users will fail verification.

So we did observe that through the testing, but obviously it's a balance to be struck between obviously making the service user friendly, but also making it not so simple that it introduced risk from bad actors that might try and get into the system. In terms of the pensions shown, so not all users found the pensions they were expecting, which did create some anxiety amongst some.

Some indicated that they didn't recognise the pensions returned, particularly where they were possible matches, which again isn't necessarily unsurprising and something that we know the industry will be working over time to improve.

Some questioned why they needed to provide a NINO [National Insurance number], particularly where it didn't result in a full match.

And therefore, they suggested that if it couldn't find, if it could find that, why couldn't it find others fully? So that's something that we've taken away and looking and see how we can sort of make that better for users. And then finally, we had a small number who did actually report sort of big feeling anxious of the potential to be scammed, particularly when they were shown a pension that they didn't recognise.

So again, that's something that we've taken away and thinking about how we could position a little bit more upfront in the service. In terms of the pensions found categorisation, so users who saw a pension returned in the red channel, which is the pension that need action channel, or indeed the amber channel, which is where there's a definite match for the user, but some of the information is missing.

Users indicated that they weren't necessarily clear at all times about the next steps they would need to take. So what could they then do in terms, what do they need to do in order to go and contact the provider to resolve those issues? In terms of comprehension, so the initial pension summary was a little bit confusing for some.

Users do need a little bit more context about what's included in the summary and the assumptions that it's based on. So that's where, for example, at the moment, the initial summary anchors to the State Pension age and it wasn't necessarily clear to some what was included in terms of the values, whether it included the State Pension or didn't include State Pension, etc.

Timeline future had mixed reactions. Users struggled in some cases to interpret the timeline, particularly when it was shown, well, as it's shown by ear, year. They naturally think of their term in terms of age.

So they were sort of mentally trying to compute what age they would be at those various ages because it wasn't shown on the screen. And then also we found that the State Pension page in particular was a bit tricky for users to digest.

So if you've seen the State Pension, you will know that there is a block of text that we show right underneath the graphics. Users reported that it was a little bit difficult for them to digest that all in one go and they would prefer that to be broken up. They also lacked clarity on the difference between the forecast and estimate.

And particularly in the first two rounds, that was exacerbated by the fact that we actually showed two sets of graphics. They were the same, but that caused users some confusion about whether that was intentional or whether it was duplicated. And so we've actually removed that, which I think has helped reduce some of that confusion going forwards.

And then lastly, just a note on defined contribution pensions, users were sort of split on the importance of seeing pot versus income up front in the summary. That's perhaps not unsurprising given the user base that I've mentioned and those higher levels of domain knowledge.

So that is something that we are keen to review and reassess as we go through this next phase of testing. Last couple of points here. So in terms of data accuracy, some data was missing or not shown correctly. Not necessarily unexpected. This is a really big task that we're working through. There is going to be lots of bits and pieces that we need to work with providers to address.

So it's really key that we identify those and feed that back. We did also get some feedback in terms of data recency. Users did highlight or indicate, I should say, that they were a little bit concerned that some of the values were older and what that meant and whether the dashboard was wholly useful as a result.

But again, I would question whether we'll see the same feedback once we get into that much larger population base, where they're perhaps not as au fait with going into their portals and checking their information on a regular basis.

And then the last point here, just in terms of help and support. So what we found was that the support options that we've got available through the service, and I'll talk a little bit more in a couple of slides time about this in more detail, but we found that users didn't necessarily always know where to go when they needed support and then how they could best get the support they needed.

So that's the general pain points. We've just got a couple more slides now if we move on, Claire, just in terms of further insights.

So particularly, we've got a slide on accessibility, some insights on One Login, and then a final slide on help and support. So in terms of accessibility, in terms of what worked well, so it was really positive that we found that dyslexic users in particular found the content really clear and readable and without having necessarily to resort to text to speech to help them understand that, which was real positive, showing that we're getting the right balance in terms of that plain English approach.

Where users did struggle, however, was that there was limited upfront, or at least they reported having limited upfront explanation, which then increased the cognitive load later in the journey as some of these things built and as they needed to concentrate more and know exactly what was expected of them.

When it came to assistive technology, we found that screen readers generally could progress independently through most of the steps in the service. So that was a real positive, although there was some variation between the different screen reading devices that can be used. So that is a bit of feedback that we've taken on board and are looking to improve.

Where users did struggle, however, was some of the transitions, particularly across the One Login process, where there's multiple steps that you've got to undertake. It could involve uploading photos of ID, facial scans, device switching, as well as switching between an app as well.

And users found at times that it didn't necessarily have all the information that they needed and were expecting up front and to know when to expect those handoffs between the different devices and systems.

And then lastly, just in terms of the sort of instructions and guidance that we've got in the service, so the core navigation and form controls that we have in the service, they were generally found to be accessible for individuals.

However, where they did struggle a bit was particularly where some of the sort of instructions rely on visual cues. And that was particularly more of an issue for One Login where, for example, there might be a reference to an action being completed and therefore turning green.

But if you're partially impaired, sorry, if you're sight impaired, you're not necessarily going to see that contrast on the screen. So if we move on from accessibility, we've got some findings in terms of One Login here, so a bit more information. So in terms of what worked well, and even though it can take a bit of time to get through the service, users are usually able to get through all of the necessary steps and report it being much better than perhaps some of the older services that they may have encountered, such as gov.verify in the past.

The majority of users are able to successfully verify their identity. And again, they're able to do that without support. So that's a real positive for the system. And that was obviously gave us a lot of confidence in them being able to open up the service more generally. I think important to call out as well that One Login does offer a number of different methods to verify an individual.

So that really enables people to choose the method that best fits the information that they can provide. So that's a real positive part of the service, meaning that people can get through depending on the route that's best for them. Users are generally pleased with the readability of the end-to-end service as I've already mentioned.

And they also like the real time updates in terms of progress as they progress through the service. So as they're completing some of the various tasks that are required, so maybe they're completing the scan of their face as part of their passport, there's then an indicator and a progress update to let them know that that's then been completed, which again just really helps instruct the user that they complete that task and they can move on to the next one in that chain.

In terms of areas for improvement, we identified that users at times were confused about whether government gateway in particular and or government verify if they've used that in the past was the same thing and whether this would replace One Login would replace those services in the future.

Obviously, we are aware that will reduce over time naturally as more and more users transition across and more government services transition across to using Gov.Gateway, sorry, One Login.

But we probably think there is something that could be done just to help position this a little bit better for users to help them understand the difference between what they may have used and have already set up. Testing with blind and low vision users identified that they do face some barriers in getting through the service.

So for example, we did encounter where individuals are perhaps trying to take a scan of their passport, that can be difficult for them to actually see and get things all in the right place.

So again, there's some areas that could be improved just to make that a bit easier for those specific cohort of users.

And then lastly, what we found was that although more accessible routes through One Login do exist in terms of verifying your identity, some users did struggle to locate these as they often weren't necessarily discoverable immediately to people.

So they may have, for example, had to have attempted one of the more standard routes through. And then when they couldn't get through that, then they defaulted more to one of the other routes through which were more accessible to them.

So that's something that perhaps could be improved, particularly for that cohort of users that are going to struggle through those more standardized routes. If you move us on, Claire, I think we've got, yes, we've got some insights just to help and support.

So users indicated that their first point of contact in seeking help and support would typically be MoneyHelper which wasn't necessarily unsurprising to us. However, they did also indicate that they would reach out to pension providers, particularly where they thought there was an issue with the data and how that was coming through.

We found that some users when they were unable to get through One Login and were directed to get support, they weren't necessarily clear on the information or actions that they might need to take, particularly where there were steps that needed to work through in order to complete that journey.

So again, that's something that we're working with One Login to see how we can improve as we are sort of the first port of contact for that initial support. Users reported feeling uncertain once they got through to the dashboard and can see their information.

They weren't entirely sure what next steps to take when the information was missing or incorrect, whether they should contact MoneyHelper or should go to the pension provider. So we're looking at how we can again make that slightly clearer for the user.

Instinctively, users want to self-help, or at least the cohort of users that we tested with in this first they tended to, as I say, caveat that most of the users that we tested with were more financially savvy and also perhaps on that higher digital inclusion scale.

So we'll be keeping that under review going forwards and seeing, trying to get that right balance between self-help, but also those other access channels that users may want, such as telephone, web chat, et cetera.

However, overall, the sort of help and support section seems like it identifies the right types of support that users might need.

We're now just trying to work out, do users identify, is it prominent enough, is it in the right place? And then how could we improve that going forwards?

OK, I think that's it for these findings. I think we've just got a couple more slides to wrap up, just to provide a quick overview in terms of what we've actually done with all this feedback.

So whilst this feedback has been really helpful, it's obviously not that meaningful unless we take it and actually do something with it to improve the overall user experience. So in that regard, we've made a number of key changes. Some of those are on the screen here.

There's other things that we've also done outside of this, but we wanted to highlight some of the key ones here. So, first and foremost, in relation to the State Pension page, as I mentioned, we've removed the table, which was causing confusion, whether it was duplicated information.

We've also added in a little bit more context to help people understand how the State Pension works in general. We're also planning a further ideation and workshop with the State Pension and PDP team to look at how we can present the data in a better way going forwards.

In terms of the pensions that need action page, we are currently iterating that to make it much clearer to users what next steps they need to take and specifically what actions they might need to do when they get to the provider to, for example, resolve a possible match or where perhaps they might have been asked to contact the provider.

And we've been working with the industry to understand what their processes look like to make sure that we're giving the right information to users going forwards. On the summary page, we have adjusted the description of the summary, again, just to try and make that slightly clearer in terms of what it's showing and what it's not showing.

However, we are also exploring some further research, particularly around the comprehension of that, but also as we start to layer in more complexity in terms of the pensions that can be shown, that might also cause us to look at some changes to that page in general.

In terms of One Login, we've fed all of the insights that we are gaining through, back through to Login, and they've been sort of prioritised and added to their own backlog. And we've also had some workshops with them specifically to look at things like the support routing and handling of queries between both MHPD and One Login to see how we can prove those interactions going forwards.

And then obviously as we progress into the second phase of testing in particular, we'll be looking to monitor any drop off between as they go through that journey and what those circumstances might be to inform our test plans going forwards.

Last couple of points then. So in terms of data recency, we've added some further information to the dashboard, or I think actually it's about to go into the product within the next sprint. And that'll add some further clarity on the information that people are seeing, why it might be a little bit in the past, or perhaps might be more recent in some cases.

And then we're also, we've been working with providers to feedback any data display issues, and also been working with the industry on how we can improve the display logic, particularly where there's been issues that in terms of the way we've received the information and then shown that on to the user.

So that I think is sort of the key insights and actions that we took. I think we've then just got one final question before, sorry, one final slide before we open up to questions. So, I think we just really here just wanted to reflect on the sort of the learnings that we had overall from this phase.

So I think my overall reflection here is that this has given us a really solid foundation from which to base the next phase of development There's still a lot of insights that we need to gain. But crucially, we have got quite a lot of strong insights. And by strong insights, we don't necessarily mean that all the feedback was positive, by the way.

We just more so mean that we've got good strength and depth of insights in those areas. So that in particular is in relation to users' understanding of the service intent and journey, which seems overall to be positive. Comprehension of the pensions info and data availability. Again, we've got some strong insights in this space.

That's not to say that it's all working because there's lots more that we need to improve in this space, but we've got a good sense of direction of how we can make that better. We've also gained some really strong insights about how people navigate the end-to-end service, including through One Login, Find and then back into MoneyHelper.

Also, we've gained insights around users' understanding of various error messages that we've shown to users at various stages of the journey, as well as those initial disclosures and warnings that we provide to users which contextualize the information that they're seeing. And then we've also gained some really strong insights around users what they need in terms of support and how we can better improve that for them.

And then just in terms of partial insights, as you can see on the screen, so we gained partial insights in relation to their pensions found outcomes in particular. So there's a lot more to do in that space as we start to scale up the volumes. That will give us a much better indicator of whether people are finding the pensions that they're expecting and overall levels of data completeness and whether that's an acceptable level for the user or whether that causes any concerns or issues around the satisfaction of the service overall. Data matching, again, there's more to do there, but obviously I think the key thing here is that we need volume to really cement the insights in that space. From 40 users, it's really difficult to get a good understanding of whether the data matching is all working as expected.

We gained some insights around user behaviour on NI number. We haven't got it on the slides here, but I think in general, most users are impinting their National Insurance number first time through. But again, we're really keen to see how that trends at scale as people start to use it.

And then we got a good understanding of the different device and browser types. Although I think one of the things that we are very conscious of is that in moderated testing, perhaps that's not necessarily reflective of how users might interact with services in the real world.

So again, once we get into the higher volume, we'll really start to see the different devices that people are using and the propensity to use those.

And then obviously a lot more to understand in terms of those critical pain points and issues that might occur and how we might resolve those going forwards.

I think that's it, Claire. That's what we had to share for you. Hopefully that's what you'll whistle a little bit and there'll be a lot more that we can share as we start to scale up the volumes.

I guess, Chris, I'm handing back to you for any questions.

Chris Curry

Brilliant. Thank you very much, Adam. Extremely comprehensive and lots and lots of information there.

In fact, it's so good you've answered some of the questions which have come in already. So I'll just pick up a couple of those to give you a little bit of a chance to get your voice back.

But I think we've got about 10 minutes or so, a bit less than that. So we won't get through all the questions that we've got. But as I said earlier, we'll be publishing answers to those questions that we aren't able to cover when we put the slides and the recording up on the website.

So one which I'll pick up is about what changes have been enacted on the user experience on the back of phase one testing and when will the next updates be rolled out.

And I think this is one of the ones where the last couple of slides that Adam has covered covers an awful lot in terms of the key changes which we're already making to some of the designs and pages as we're going through. But of course, we'll continue to develop and iterate as we get more insight throughout phase two of the testing as well. So watch this space.

There'll be future events like this where we'll be updating on what else has been doing as we go through phase two. So going back to some of the earlier things that you covered in the presentation, Adam, one of the some of the questions we've come in are about kind of how much information we'll be flagging to users.

So one in particular is about we will be flagging up that those schemes with less than 100 deferred and active member entitlements probably won't be connected to the dashboard. So people won't be completely surprised if they don't see all of their pensions.

Adam Gifford

So in fact, on the landing page that you just start up, we do have some information about some of the gaps in the service and what they might not see in the dashboard and what the reasons for that might be.

I think one of our observations would be at this stage is that there's probably a little bit more that we can do around the prominence of that in terms of that. But also, obviously, it's then if people don't, some of this won't necessarily mean that much to a user because they won't necessarily know whether they're in that bracket or not.

So it's how can we improve that by working with users over time. So I think at this stage, we've got some insights, not as much as perhaps we would want.

Hence, in further rounds, we'll probably look to probe a bit more further on, you know, do people see that? Do they understand it? And then when they get to the end, does that kind of match what they're expecting?

Or, you know, if they didn't see a pension they were expecting could that correlate to the earlier information that was shown? But certainly something that we do include and we'll be looking to monitor.

Chris Curry

Very good, thank you.

There are a couple of questions coming in about One Login as well, perhaps unsurprisingly, given it's such an important part of the journey with what we're doing.

So a question is asked, will there be any enhancement or changes to the One Login service before the public launch or increased user support demand being planned into business readiness plans?

Adam Gifford

I'm not sure I can answer that currently just because obviously One Login have their own backlog that they work through. So we're one service that interacts and uses One Login.

There are lots of other services that are also using One Login as their identity service. So they are also feeding back insights to One Login all the time. I think what I can say is that they is a service that is in beta.

They are evolving and iterating the service continually based on that user's feedback and based on the prioritization that they're doing, you know, based on the number of issues they're seeing and the nature of those things.

So we've certainly got a regular dialogue with them where we're sharing our insights and learnings and they're also then sharing back how they've worked on certain things to improve that.

So definitely I would expect that things will improve and will continue to improve, probably beyond the point at which we launch our service.

Chris Curry

Yeah, that's a really important point that we are in very close contact with One Login and they are a service which is in development as well at the same time.

And another quick question related to that, which I think you might have already covered a little bit, is what would happen if someone did not have a photo ID and therefore didn't have the ability to get it? Can they still use One Login?

Adam Gifford

Yeah, absolutely. There are other routes through that don't rely on a photo ID. So users can select the route that best fits the information they can provide.

One, for example, relies more on credit checks with banks and stuff, which is a bit more accessible, particularly in that situation for users.

And I imagine, again, that might be something that will change and evolve over time as well.

Chris Curry

OK, brilliant. Another question, probably more broadly about what will happen when dashboards are launched which says so once the dashboards are made publicly available, will there be a central point of contact for say the support, help and guidance?

I think you mentioned that at the moment people were coming first to Money Helper. So how will we be using that and evolving that?

Adam Gifford

So that is something that we are working through to really understand what's the user needs. Also, how would we, how could that be facilitated? What's the best way to do that?

There's a lot of different parties that are obviously providing the overall experience to the user, be that the MoneyHelper front end dashboard, we've got the data that's being provided, we've got the industry that's facilitating some of those data flows and then we've also got the One Login piece as well. So we're certainly aware of the challenges in that space.

And I think the key thing over the next sort of this next phase of testing and beyond is then to look at what does that longer term operation will look like and possibly wouldn't need to look like beyond MoneyHelper as well once we possibly then get to the point of introducing other dashboards into this space where that need and that support model might need to evolve again.

Chris Curry

Yeah, that's a very good point because of course the MoneyHelper dashboard is the first dashboard that's going to be launched, but it won't be the only dashboard after some time. So yeah, we need to have a support system which is going to work short term and long term as part of that.

Now we've got a question that's come in asking for a little bit more detail about the use of the National Insurance number or the NINO in matching, where the common approach in industry is to require a NINO to match in order to get a full match without it quite often you'll get a partial match instead.

So are we seeing that there's a big difference if people are entering it or not?

Adam Gifford

Difficult to say. Most users that go through first time around do input their National Insurance number. And of course, that's then what the providers will do their initial search and match on.

We have seen examples of users come in and then perhaps the second time through not input their National Insurance number.

But of course, that initial match sort of search and match by the provider has already been done and therefore it doesn't sort of change that initial result.

I think, as I mentioned, one of the key things in this next phase as we scale up is then to look at the prevalence of the input around the National Insurance number and then see if we can look at the correlation between any matching and partial matching data that we're then getting at scale.

As I say, on the basis of the number of users that we had, most did input it, and then also it's small numbers, so it doesn't give you a really good indication of how that's working overall.

Chris Curry

Yeah, I think it's one of those things that we'll definitely be looking at as we go through phase two and as those numbers are scaling up. So it's an important thing.

And I suspect that will also be the answer to this final question, which I'm going to put to you, Adam, which is have we been able to gain any insight to users preference about what when they use the when they use the service?

So, for example, do they prefer to use it in the evening or in the morning? Or is there any particular hot time that people like to use it?

Adam Gifford

Yeah, again, we didn't really gain too many insights in that regard during the moderated testing, but just by nature of the testing and what we were seeking to obtain.

And obviously, even if we ask questions to users about that, there's often a difference in terms of what they say and what they might do in reality.

And that's where you say in the next phase, that's really key for us to start to understand that more natural use of the service as people are getting and receiving a link and can use it as their own volition.

We are looking for trends to see, you know, when are the common times that people are using them and what's the different split of devices as well.

Chris Curry

Brilliant. Thank you very much for asking those questions, Adam. That's been incredibly helpful. We've got a number still that unfortunately we haven't got through, but we are now out of time. It's already quarter two.

So I'm going to just bring us to a close and I'd like to close by highlighting how you can stay up to date with the latest news on pensions dashboards and how to get in touch with us if you have any further questions as well as those ones you've submitted today or about any part of pensions dashboards going forward.

So we are continuing to explore opportunities to engage with industry in person, building on the success of the number of webinars we've hosted and we will continue to host webinars in the future. For example, as we go through phase two of the testing, there'll be more information coming out.

And our aim is to create more forums for networking, discussion and sharing the latest programme updates. Outside of the webinars, if you'd like to raise a query, you can always contact the programme via the support page on our website.

And our website is also home to a wide range of articles, videos and reports, which we publish regularly to help provide updates and general information to keep you informed of what is happening.

You can also follow us on social media and sign up to our newsletter to receive the latest updates directly to your inbox.

With that, I will bring the session to a close. Thank you very much for joining us. I hope you found that insightful and helpful, and we look forward to seeing you on the next one.

Thank you very much and have a good afternoon.