Preparing User Research for an Alpha Assessment
While working at Nomensa, I was assigned to a government arm’s length body (ALB) as a client with a high profile financial product (potentially 38 million users) that needed to be assessed according to Government Design Standards (GDS) in order to be permitted to progress though design and development stages (Discovery, Alpha, Beta, Live). I joined the project as it was commencing the Alpha phase following a pause of nearly two years. My role was to lead the user research and to ensure it was done to an appropriate standard to pass the Alpha assessment.
These are some of the requirements we needed to fulfil:
Establish who the likely users are.
Create evidenced based personas or other ways of categorising users.
Identify user needs and ensure they continually inform design.
User Profiles
When I came onto the project, a full discovery round had been conducted more than two years previously, but there was no full report for this research, only a high level ppt presentation. There was also no record of the participant characteristics, nor were there any recordings of research sessions or presentations. Given the stringent requirements for passing GDS assessments, we decided to run a mini discovery round of 18 in-depth interviews to test original findings, especially with regard to the four personas that had been created.
For the most part, our research confirmed what had come previously. However, we changed the personas to be more high level and considered them more as stages that a user might move into and out of over the course of a lifetime. We dropped one of the original four (‘Life is hard’) because we did not interview anyone who fell into this category and they were unlikely to use the product until them moved into one of the other stages anyway.
The following is a screenshot from the Miro board where we edited the profiles following the mini discovery round.
Wrangling the Quantitative Data
There was a strong desire within the team to include a quantitative account of who the users are. In addition, one of the requirements for this task was that our users for this specific product be mapped to the marketing segmentation for the entire market for the ALB, which was perceived to be the entire UK population. I created the following chart in order to fulfil this requirement.
Line 1: Segmentation
This line shows the ALB’s own segmentation, which accounts for the entire population. The three categories were developed through research conducted by the in house marketing team and show a progression from less comfortable to more comfortable across the categories.
Line 2: UK adults with pensions in accumulation.
The data on this line were taken from the 2021 national FCA Financial Lives survey. It shows how many adults have pensions in accumulation (58% or roughly 38 million). This is the total potential user group for this product. The grey area on the left shows the FCA’s estimate of those who are not in retirement, are not users of the product and why (according to their survey data). The grey area on the right shows those who are in retirement (and therefore not users of the product) as well as those who are in retirement and are not users for other reasons.
Line 3: This line shows how our user profiles map to the two other data breakdowns. Note that our profiles are placed mainly within the 57% with pensions in accumulation and that we’ve kept Life is hard in the chart even though bringing them into the other profiles is a challenge.
Continually Testing the Profiles
In order to ensure we’d made the right decisions with regard to our user profiles and to make sure we picked up on any emerging variations on this as we worked our way through research rounds, we carried out pre-interviews with every moderated testing session that allowed us to establish where participants sat within the scale. Following is a chart showing where participants fell throughout 8 rounds of research.
Keeping the User Needs at the Forefront
One of the most important requirements of the GDS process is to keep collecting, tracking and tacking account of user needs during design iterations and testing. We developed a way of doing this involving collecting user needs during every research round, storing them in a spreadsheet, and then holding workshop sessions where we would determine whether user needs were in or out of scope and then prioritise them for potential development.