Research
Problem Statement
Some veterans are unaware of benefits and services offered by Veterans Affairs Canada (VAC). They can experience confusion when trying to find out which programs are relevant and would fulfill their needs. This can discourage them from applying and keep them from accessing services that could help them regain health or live better lives.
Context
Modernizing service delivery is an ongoing priority at VAC. The focus is on efficient, effective, user-centric and outcome focused service to Veterans and their families. VAC created the My VAC Account online portal and the Benefits Navigator application as part of this modernization effort.
The Online Services Directorate at VAC initiated a project to add a ‘Benefits at a Glance’ feature to complement its Benefits Navigator application. The Canadian Digital Service (CDS) has been engaged to support VAC’s efforts to help solve this problem in a way that will best meet Veterans’ needs.
Key Users
- Veterans, still-serving military members and family members of Veterans.
- VAC Frontline Staff
- Call centre agents
- Area office staff
- Veteran service agents
- Case managers
- VAC Internal Teams
- Online Services
- Communications
- Information Technology
Service Owner
- Janet Nichols, A/Director General, Online Services Directorate
- Michel Doiron, Assistant Deputy Minister, Service Delivery Branch
Lead contacts
- VAC: Farzad Sedghipour, Service Designer, VAC Online Services
- CDS: Courtney Claessens, Product Manager, CDS
Proposed outcomes
To develop a user-centered service that allows Veterans to browse relevant benefits. The service should encourage them to apply for these benefits and engage with VAC services through their preferred channel. This will reduce pressure on telephone and over-the-counter processing. Making this service available will improve awareness of benefits amongst Veterans and increase the number of qualified applications for benefits.
DISCOVERY PHASE
Overview
In December 2017, CDS in partnership with Code for Canada and VAC Online Services began a Discovery Phase to understand the problems facing Veterans in their journey to learning about VAC benefits and services and the challenges they face in applying for them.
Combining agile methodology and human-centred design, the discovery work was divided into two parts. The first part focused on the experiences of different staff groups within VAC and the second part on the experiences of Veterans. Field research was conducted throughout. Data analysis and findings were shared with the service owners and internal groups at VAC.
The discovery phase validated that there is a decreased uptake of services and that Veterans have a low familiarity with VAC benefits. Findings also informed the development of a minimum viable product (MVP) that would enable Veterans and their family members to quickly determine which of the 40+ benefits are relevant to them, and confidently take their next steps.
The discovery work has received highly positive feedback from VAC’s internal teams and senior executives as a model user research process to improve government service delivery. The discovery process and findings were disseminated by: i) sharing publicly on the CDS blog, ii) live tweeting by team members, iii) a presentation to Minister Scott Brison, President of the Treasury Board Secretariat and Digital Government, , and iv) a Code for Canada event in Toronto with the civic tech community. Finally, other federal teams have expressed interest in learning from the process.
People involved in discovery research
- Veterans, still-serving military members and family members of Veterans.
- VAC Frontline Staff
- Call centre agents
- Area office staff
- Veteran service agents
- Case managers
- VAC Internal Teams
- Online Services
- Communications
- Information Technology
- Field Operations
- Access to Information and Privacy (ATIP) Request
- Research Directorate
- Office of the Veterans Ombudsman
- External contractors
- Code for Canada
- Stiff Sentences, Ottawa ON
- Thinking Big, Charlottetown PEI
1. Veterans
Who is a Veteran? VAC defines a Veteran as
Any former member of the Canadian Armed Forces who successfully underwent basic training and is honourably released.
VAC serves a range of Veteran groups - from WWII Veterans to family and children of recently retired military members. This is a wide range of groups and consequently a wide range of needs. We must begin by clarifying who we are serving as end-users.
VAC serves veteran groups based on types of
- Service
- Organization
- Family members and caretakers of Veterans
Service Type - a way to categorize military professionals based on their career (e.g. Canadian Armed Forces, World War Veteran, Korean War). These include
- Still-serving members of Canadian Armed Forces (CAF) or Royal Canadian Mounted Police (RCMP)
- Veterans of Modern-Day Conflicts from CAF or RCMP
- Veterans of the Second World War and Korean War from CAF
- Allied Veterans now living in Canada
Organization types
- Canadian Armed Forces (includes the Navy, Army, Reserves, etc.)
- Royal Canadian Mounted Police (RCMP)
A snapshot of the current context that’s relevant to our discovery work:
“When people think of Veterans, many immediately picture someone who served in the First World War, Second World War or the Korean War. While many Canadians recognize these traditional Veterans, the same may not always be true for Canadian Armed Forces (CAF) Veterans—those who served Canada since the Korean War.
In fact, some former CAF members don’t even see themselves as Veterans. Veterans Affairs Canada (VAC) wants to change this and is working to ensure CAF Veterans receive the honour and recognition they have earned and so richly deserve.”
Source:
http://www.veterans.gc.ca/eng/about-us/definition-veteran
2. Veterans Affairs Canada (VAC) Frontline Staff
Veterans Service Agent (VSA)
A Veterans Service Agent (VSA) takes full responsibility for the service needs of Veterans and their families, and becomes their primary point of contact. A VSA analyzes a Veteran’s needs, coordinates and helps them navigate the VAC system to ensure that their needs are met. VSAs do not take on complex case files.
Case Managers
A Case Manager (CM) is a VAC staff member who handles Veterans services needs in complex situations or circumstances. Case managers help Veterans with planning and applying for benefits, and follow up with them nearly every 45 days. Veterans might need case management when they are coping with serious illness or adjusting to a loss such as a career or a loved one.
Area Office Staff
VAC staff in the regional area offices around the country meet with Veterans and their families to help them receive services and benefits. Their services are also provided in Service Canada Offices across the country, and in the territories and other northern communities.
Analysts at National Contact Center Network (NCCN)
VAC’s English and French toll-free lines enable callers to access the National Contact Centre Network (NCCN). NCCN provides a central point of contact for Veterans to obtain general information and request services. When the caller’s request goes beyond the authority of the NCCN analyst, the call is transferred to the appropriate area for response such as to connect to a VSA or CM.
A snapshot of the current context that’s relevant to our discovery work:
VAC has a new directive on ‘How services are delivered’
“It was clear the Department needed to make a fundamental change to the way it delivers services and benefits for the men and women who have served our country and rely on our services. Many Veterans feel they are missing out on services and benefits because they are unaware of what is available and simply don’t know the right questions to ask. As one Veteran so succinctly stated: “I do not know what I do not know.”
- Delivering on Service Excellence Veterans Affairs Canada, 2017
3. VAC Internal Teams
Service Delivery Branch - Branch responsible for delivering benefits and services and for providing social and economic support that respond to the needs of Veterans, other clients and their families. The branch also provides services in the areas of:
- Field Operations such as area offices
- Online Services Group/Directorate (OSG/OSD) - the section within the Service Delivery Branch primarily responsible for My VAC, a Veterans portal to apply for benefits, and some web presence/properties. Currently lead by Janet Nichols (Director and Service Owner for this project).
Strategic Oversight and Communications Branch
This branch is responsible for leading communications initiatives. They work in line with the Communications Policy of the Government of Canada that support the Department’s strategic objectives and priorities.
Chief Financial Officer and Corporate Services
This branch is responsible for supporting VAC in meeting Government of Canada requirements in areas such as financial stewardship, corporate reporting and key accountabilities. The branch also provides internal corporate services in the areas of:
- information technology
- access to information and privacy activities (relevant for our design research activities)
Strategic Policy and Commemoration Branch
This branch is responsible for:
- ensuring that VAC programs and policies remain relevant and meet the current and future needs of our clients.
- the Research Directorate (RD) supports departmental decision makers and planners by supplying timely scientific evidence related to military and Veteran well-being (relevant for our design research activities).
4. Office of the Veterans Ombudsman
The Veterans Ombudsman works toward ensuring that the sacrifices of Canada’s Veterans and their families are recognized through the provision of services, benefits, and support in a fair, accessible, and timely manner. The Ombudsman is an impartial and independent officer, reporting directly to the Minister of Veterans Affairs Canada. This independence is essential to ensure that our commitment to stakeholders is honored.
5. External Groups
Code for Canada
Code For Canada is a not for profit organization that pairs up experienced digital professionals from the private industry with government partners - in this case, Veterans Affairs and Canadian Digital Services.
Stiff Sentences, Ottawa
Stiff is a communications agency that works with clients in Canada and around the globe to create and manage brands, develop integrated communication strategies and go to market through traditional, digital and social media. VAC has contracted Stiff to work on the redesign of the VAC website and content design of VAC programs and services.
Thinking Big, PEI
Thinking Big is a digital services and UX Consulting organization in Charlottetown, PEI, working on the redesign of the My VAC Account and in preparing for VAC’s Web Renewal Initiative as part of the Government of Canada’s (GOC) Web Renewal Initiative.
INTERNAL SERVICE RESEARCH
Overview
Part I of the discovery phase validated the existing insight that Veterans have a decreased uptake of services and a low familiarity with VAC benefits. This process involved i) interviewing staff and organizational teams across the benefits process, ii) mapping VAC’s organizational goals, iii) studying the structure of the VAC benefits, iv) studying VAC’s IT infrastructure and web presence, and v) reading VAC’s reports on patron-focused research.
This research showed that VAC staff throughout the department face difficulty finding a single source of information regarding the benefits available to Veterans. These staff include case managers, front-line staff from call centre, benefit application adjudicators, benefit program managers, and the Veterans Ombudsman’s office.
What was learnt
1. Not all former military members identify with the term ‘Veteran’
Younger, modern-day Veterans may not identify with the term, ‘Veteran.’
“A lot of people don’t believe they can apply for benefits…like reservists, people that were never deployed.” - staff at The Royal Canadian Legion.
The Royal Canadian Legion is a non-profit Canadian organization serving former military members (veterans’ organization).
2. Eligibility of benefits is not well understood
Figuring out which benefits are relevant based on a veteran’s profile can be complex. It’s hard to determine at the outset what someone is eligible for.
“One-on-one briefing with someone who is knowledgeable is the most effective way to determine what members are eligible for and how to proceed.” - staff at Royal Canadian Legion
“You’re surprised at the outcome…I never know if it’s going to go through.” - VAC staff
3. Veterans have complex and changing needs
Veterans’ needs during the period of transition from the military and thereafter can be very complex. The challenges faced are both physical and mental. Many of their health conditions are also degenerative. The report ‘Life After Service Survey 2016’ highlights these conditions further:
“Compared to those with earlier releases, recently-released Veterans have higher rates of service in Afghanistan, poor self-rated mental health and less than a total of 10 years of military service; all factors associated with difficult adjustment to civilian life. Veterans reported chronic conditions, including arthritis (29%), depression (21%), anxiety (15%), and post traumatic stress disorder (PTSD) (14%).”
Any solution to these problems must build on and bring together past initiatives, and needs to be implemented in a way that can reach Veterans.
4. Veterans need an increased awareness of VAC benefits and services
This validates another established finding that there is a need to increase awareness of VAC services. Research tells us that most veterans are aware only of 3-4 benefits out of 40+. There are many veterans who need VAC services and aren’t currently accessing them.
“My Vac is great if only they knew about it.” - Former military member and current staff of VAC referring to the My VAC self service online portal.
5. Veterans need a central point of access for information on benefits
It can be hard to parse through the information and determine what is actually useful. There are lots of tools but information is fragmented and often not up-to-date. Too much information may have become as much of a problem as too little information can be.
“It might take you a half-hour; you might miss something.” - VAC call-center agent
“We have enough tools. They’re stretched really thin and sometimes they’re not updated” - VAC frontline staff
6. Lack of takeaways or references to relevant, useful information
The call centre generally does not follow up with the veteran on their enquiry or share a takeaway such as emailing a link to a relevant page on the website. The Benefits Navigator online tool doesn’t allow veterans to save or share information with friends or family or their case managers.
“What we want to give them is curated and useful” - VAC staff
7. Veterans need an easy way to determine which benefits are relevant to them
VAC’s current website doesn’t enable the user to get to relevant content by being able to answer two simple questions: who am I? And, what is my goal here? This prevents veterans and frontline staff from exploring what options are available to them.
“It’s all good stuff, but it’s just more stuff that’s coming to the folks. It’s hard to know what relates to me.” - staff from the Veteran Ombudsman
8. There is a need for collaboration and vision setting
There seem to be parallel initiatives ongoing at VAC to collaborate and share priorities. In order to fulfill Veterans needs, medical and otherwise, veterans may need to combine their benefits from VAC, CAF, SISIP (Serviceman’s Income Security Insurance Plan) and others provided by government organizations.
“Bigger problem is that we have to centralize the usability team” - staff from VAC Research Directorate
Methodology
Desk research
We studied research conducted by VAC and other organizations on the topics of
- Veterans in Canada; and
- VAC benefits and services
Some sources included reports and papers from the VAC Research Directorate (e.g. Life After Service Studies), surveys and studies from the Department of Statistics Canada, Department of National Defence, Public Opinion Research on Veterans in Canada (Library and Archives Canada) and the Auditor General of Canada (2013 Fall Report of the Auditor General of Canada).
Staff interviews
To gain a deeper understanding of internal staff members as users within the benefits process, we conducted in-person interviews in Charlottetown, Prince Edward Island (PEI), VAC headquarters. Individuals and teams were interviewed, including area office staff, veteran service agents, case managers and national call centre staff.
Business Model Mapping
To familiarize ourselves with VAC and the services it provides, we studied:
- VAC’s service performance reports;
- documents outlining strategy;
- plans and priorities for the department;
- past projects;
- policy guidelines
To synthesize all this information, we carried out a mapping exercise using the ‘business model canvas’. The business model canvas is a one page visualization of an organization’s business model operations. (More here: https://strategyzer.com/canvas/business-model-canvas).
Value Proposition Mapping
Following the business model canvas session, we held a workshop with the VAC staff (in PEI) using the ‘value proposition canvas’ as an artifact and technique to elicit insights on the value of VAC benefits for Veterans. The ‘value proposition canvas’ is a visual artifact and design technique to help teams understand how their product or service helps solve customers’ problems (relevancy), and delivers specific benefits (quantified value) to them.
Copies were printed for five teams (including one team participating online from the regional offices) to complete the facilitated exercise. The workshop ran for 3 hours and helped us gather information on staff pain points; if they feel empowered or disempowered by their roles and their thoughts and assumptions on the needs of Veterans.
Expert interviews
To learn from those who have undertaken similar exercises to improve government service delivery, we interviewed other government teams:
- U.S Department of Veterans Affairs (VA) - VA Center for Innovation
- Ad Hoc - software design and engineering company building government digital services for U.S Veterans Affairs
- Veterans’ Affairs New Zealand
Next steps
Lessons from part I of discovery research were shared with VAC’s internal teams and senior executives. Our next steps include developing a research plan for part II - interviewing Veterans and studying the benefits structure in greater detail.
Selected documents and presentations
Discovery Part I - Presentation
PART II: INTERVIEWING VETERANS
Overview
Part II of the discovery phase focused on interviewing Veterans. The goal of this work was to understand why the current system of benefits often seems inaccessible to Veterans.
Using snowball recruitment (or chain referral recruitment) we were able to connect with over 75 informal recruiters. They in-turn nominated Veterans who met our participant criteria for the interviews. We received over 40 Veteran nominations within the first 3 weeks and conducted a total of 9 semi-structured, hour-long interviews with Veterans. Over 200 pages of notes were generated. All members of the delivery team, including designers and developers, participated in this research.
As human centered design was new to VAC, CDS took the lead on conducting these research activities. Findings from this research were later shared with VAC.
What was learnt
1. VAC’s communications may not be reaching modern-day Veterans.
Stigma
Military culture makes it hard to ask for help, this makes applying for benefits a stigmatized subject, a sign of weakness. The feeling is magnified in new Veterans by the challenge of shifting from following orders and procedures to proactively navigating the VAC system, and having to care for oneself.
Outreach efforts
When VAC delivers outreach efforts such as transition interviews for releasing members, interviews indicate that Veterans may not be ready to respond. Firstly, leaving the military is a major life event. Just before their release, Veterans are busy. With multiple appointments and paperwork to complete they are not thinking deeply about applying for new benefits. Here, Veterans are generally preoccupied with their release and life changes.
Secondly, soon after the release period, some Veterans may be in a difficult financial or medical situation with an urgent need for help and VAC might seem like an attractive option to seek help. Although, when Veterans grasp the extended time commitment to complete VAC’s application process, they tend to move towards quicker, alternate options. Sometimes never to return to VAC.
Myths
Misconceptions lead Veterans to thinking that VAC is an organization meant to serve them. Some included:
- Limited amount of funds available to Veterans
- “There’s only a limited amount of money there, because there’s only so much - why bother with me?”
- Perception of gaming the system
- “If I apply for benefits, without having critical physical conditions, I’m going to be perceived as someone gaming the system.”
- Benefits are only for the injured
- “I’m not that injured, old and disabled person. I don’t even consider myself a Veteran, really.”
- Distrust in the service
- “I bet you I would be denied. Why start the process?”
Influence of social network
Many Veterans cope with their difficulty adjusting to civilian life by turning to their friends and family. People in the interviews shared that if their friends have applied or talk about applying for a benefit it would increase their interest in applying for the same benefit.
2. Hard to relate to website content
Veterans indicated that the content on the website is hard to understand as it is written for bureaucracy experts, not in plain simple English. Additionally that the content assumes that Veterans should know what they are looking for from VAC. Contrary to this assumption, especially newer, younger Veterans are still figuring out how to shape their lives as civilians and would like the website to help them connect where they are to where they should be.
3. Website is complex
Veterans indicated that when they are on the website - they tend to spend more time navigating through pages of information than they would like to spend learning about the benefits that might help them.
4. Website improvements are not enough
The VAC website is often a Veteran’s first touchpoint for learning about VAC benefits. However, Veterans stated that information on the website often isn’t sufficient to help them make the decision to apply for a benefit. Because applying for a benefit is a big decision, they would like information that is suited to their unique circumstances. Some Veterans seek this byphoning the call centre to get specific advice or going to an area office to meet face to face with VAC staff.
5. Lack of transparency
Veterans expressed their struggle with a lack of transparency with the benefits process. They have many concerns about missing key information that could help them in their decision making process and setting expectations are realistic, such as wait times and the amount of paperwork involved.
Methodology
Participant recruitment
Snowball sampling (or chain sampling, chain-referral sampling, referral sampling) was used for the discovery phase. Snowball sampling is a non-probability sampling technique where existing study subjects recruit future subjects from among their acquaintances. Thus the sample group is said to grow like a rolling snowball.
Semi structured interviews
Semi-structured interviews are used to gather focused, qualitative textual data. This method offers a balance between the flexibility of an open-ended interview and the focus of a structured ethnographic survey.
Analysis methods:
Grounded theory
A methodology that involves the collection, analysis, synthesis and conceptualization of qualitative data for the purpose of constructing meaning from the data.
Conceptually clustered matrix
A text table with rows and columns arranged to cluster the insights that are related thematically.
Journey mapping
Using the research data, we developed a journey map visualizing the process that a Veteran would go through in order to receive a benefit from VAC.
Reporting
Insights were presented in a powerpoint presentation outlining key insights and proposed solutions for moving ahead.
Documents and presentations
Consent and privacy documentation
https://docs.google.com/document/d/1zadYJuf6KQh4eJMV8vD6tuUBMkD-qzo1woIJEyu4zJE/edit#
Interview guide
https://docs.google.com/document/d/18ylT425JgygPQCc-zBomJPQxu6FpcA6IpiOizseltas/edit
Interview tips
https://docs.google.com/presentation/d/1a_qYVZnC9izoyUXP7dB5eIc8Mj343XiP4ucwNSWqtxQ/edit
Discovery: Research presentation
Guidance and resources
- Privacy Act Canada
- Public Opinion Research
- New Veterans Charter, 2006
- Education and Training Benefit, 2018
ALPHA PHASE
Overview
In the alpha phase the team developed design principles and feature level solutions towards addressing the needs elicited from discovery. The end result was a successful minimum viable product that helped Veterans, still serving military members and their families find relevant benefits to meet their needs. The alpha phase began in July and concluded in October, 2018.
What was learnt
Alpha identified that people use the product for three key functions
- Exploring - For example, a still-serving military member uses the tool to learn about the help they might receive once they have completed their military service.
- Assistance for a specific need - For example, a recently retired military member is looking for funding assistance to help them pay for school.
- Assistance for a specific benefit - For example, the daughter of a Veteran wants to learn more about the Veterans Independence Program to help her parent.
Insights
Guidance
Veterans need a simple introduction on benefits and information on how benefits might help them. This knowledge precedes their use of any tools or functions on the website.
Positive communication
The incentives to learn and apply for a benefit is greater when the message is welcoming and positive. Painting a hopeful scenario is important for those who are dealing with stress, medical conditions and disabilities.
Clarity and purpose
The objective and function of this tool was unclear to some participants. The product name and landing page content must communicate, in a simple and easy manner, what the tool can help Veterans achieve.
User flow
The path taken by participants during usability tests indicated that users can be unsure of why they performed certain actions and often returned to their previous actions in order to gain confidence that they’re indeed on the right path.
Simple and easy to understand
Veterans expect information on the website to be written in simple, plain language. They expect to see examples and descriptions to help them determine the relevance of the benefit to their needs.
Takeaways
When participants used the tool to learn about relevant benefits, they indicated a need to save, store and capture the information to prevent losing the data or sharing it with a family member or friend.
Combined benefits
The goal for some Veterans was to get a better sense of the cumulative benefits they could receive. Either as a lump sum monetary benefit or the number of services that they may be eligible for, similar to a shopping cart experience.
Visuals
Veterans expected to see more images and icons to help relate to the information. Too much text was described to make it harder for them to remember all the information.
Methodology
Service Design Blueprint
The service blueprint helps us to understand a service process from an end to end perspective - if we are studying the process of veterans applying for their benefits then the blueprint captures this with the veterans journey, their front end interactions and the backstage interactions from the department.
Usability testing
A total of 4 moderated usability testing sessions were completed during the alpha phase. The tool was tested with a total of 16 persons, including Veterans, still serving military members and their family members.
Test sessions were conducted in Ottawa at the CDS office or local coffee shops. Remote tests were also conducted with persons outside the NCR location.
Participant recruitment
Snowball sampling (or chain sampling, chain-referral sampling, referral sampling) was used for the alpha phase.
Documents and presentations
Alpha - insights and decision log
https://docs.google.com/presentation/d/1G5FnLgOEuJQ_gMeo3UYeP1STlaRW8Hyckjji2-sRHR4/edit#slide=id.p1
Test plan: test session 1
https://docs.google.com/presentation/d/1q5XeJ_RJn7Vo1Q2wGtkFtHq9nwPp1RE_5EoLgeo8FCk/edit#slide=id.p
Test plan: test session 2
https://docs.google.com/presentation/d/1MPkd2IEUHQq6dWoX4lajAfO3NiUAPkFTQnS5o1LXTIU/edit#slide=id.p
Test plan: test session 3
https://docs.google.com/presentation/d/1GIIA1SfjvsDUS9Z-CiDD7mRxO1ChuGEFirm5N383PrY/edit#slide=id.p
Test plan: test session 4
https://docs.google.com/presentation/d/1cpuZBsIttqFFWDxrNjv6dolI0uJhomFNRuncxB1JeVk/edit#slide=id.p
Guidance and resources
Usability testing
https://cds-snc.github.io/design-research-handbook/usability-testing/
BETA PHASE
Overview
After building an MVP in the alpha phase, research in the beta phase focused on improving the product with regular usability testing.
An integral part of the beta phase was for the CDS delivery team to build capacity and onboard a multi-disciplinary product delivery team from VAC (based out of VAC Headquarters in Charlottetown, Prince Edward Island). For research, this meant recruiting a design or UX researcher with applied research skills and onboarding them to understand the progress of the product and team thus far.
Having successfully been onboarded, Emily Coffin took over the team’s research practice. This included designing test plans, conducting field research and analysis, sharing insights and particularly in bringing the other team members along for the research journey. The goal was met to empower the VAC team (and therein the VAC department) to continue the work with the principles and practices of agile development and human centered design.
Another important highlight during the beta phase was conducting a usability testing session entirely in French with Francophone Canadians. All federal services are mandated to be provided in both official languages, so testing products in French and English is integral to achieving good product outcomes. Due to the shortage of French speakers from the product team, volunteers from the broader CDS organization were mentored and trained to moderate the usability tests and participate as note-takers.
Results from the usability tests conducted in French identified similar usability issues to those found in English testing. However, French speaking participants completed fewer tasks than english speaking participants, and reported a lower confidence level. This could be due to language and cultural differences. Another explanation for the difference could be that the French moderator was less experienced in conducting the usability tests. However, these are hypotheses that would require further study to validate.
Research in the beta phase was also conducted with frontline staff members from VAC area offices (Ottawa, Gatineau and Halifax) as well as the NCCN office in Halifax. As frontline staff work closely with Veterans to help them apply for benefits, the tool is useful as a resource for them to direct Veterans towards. Research insights confirm that staff are experts on the subject of VAC benefits and services and have lower needs to directly benefit from using the tool. However in sharing the tool with the Veterans they serve, staff members can be of greater help by answering their clients’ better informed, personalized questions.
Methodology
Usability testing
A total of 4 moderated usability testing sessions were completed during the alpha phase, testing the tool with a total of 81 persons - i) Veterans, ii) still-serving military members, iii) family members of Veterans and still-serving military members, and iv) frontline staff, including case managers, veteran service agents and call centre agents.
Participant recruitment:
Beta test sessions 1, 2, and 3 were conducted inside VAC Area Offices located in Ottawa, Halifax and Gatineau respectively. Beta test session 4 was conducted in Petawawa, Ontario, out of recreational and community centres.
Veterans, still-serving military members and their family members were intercepted at these locations and requested to volunteer to participate. People were approached only once they had completed the tasks they come to the area office for.
If the moderator received a person’s full consent to participate, they were guided into a comfortable, quiet room in the area office to begin their test session.
After completing the test, participants were thanked and directed to meet with a frontline staff member before they exited the premises. Consent forms and privacy documents were shared with them to take back.
References and presentations
Beta - insights and decision log
https://docs.google.com/presentation/d/1ljJZqCagcLw9ompscKNrfEWeGDr2K0E1lJkrTSIEKZU/edit
Product evaluation summary
https://drive.google.com/open?id=1dWsLARRpgehK-bnV2nLZcWOkSsrCjyz9ctV7RnWeraE
Beta - documentation for test session 1
https://drive.google.com/open?id=1z6KdHZQJm8hgL8h7QMFKqrY2bmo5GDmPVLr2UojJMec
Beta - documentation for test session 2
Beta - documentation for test session 3
https://docs.google.com/document/d/16Dz7oNM5ZESYClkPB6Jy6zal-xfOEInvrTlkHFuRGog/edit
Beta - documentation for test session 4
Other
Show & Tell - public showcase
https://docs.google.com/presentation/d/1JRtdR5ryn6FQmcMJr5CzWspuwZdNAV-LfAjH5VNYJSo/edit
Onboarding documents for the VAC UX Researcher - Emily Coffin
https://drive.google.com/open?id=1yTAzItKZfbZGP06NS2TMeKrw07HNwivirSXQ-UqJqL0