Research frequency self-assessment
Conduct research repeatedly through all phases of product development.
Research isn’t confined to one phase of product development. And each phase of product development should include more than one research study. For example:
- Discovery could include observations, interviews, secondary literature review or other background data gathering.
- Alpha could include prototype testing, usability testing or larger scale testing.
- Beta could include repeated usability testing, A/B testing or analytics analysis.
There’s no simple guideline for how often research should happen.
How often should you usability test in alpha and beta? How often should you conduct interviews in discovery? There is no simple answer to these questions. Instead, conduct research often enough that you score well on the following self-assement.
Use this self-assessment to evaluate your research pace:
Check each statement you agree with.
-
We make major delivery decisions based on research. We decide what to build, how to build it, and what order to build it in, based on research. Conduct research often enough that you can make those decisions based on research.
-
We identify substantial issues before we release to the public. Our research doesn’t just help us make decisions; it also identifies problems we didn’t know we had. Conduct research often enough that you can identify those problems before you release.
-
We test whether we actually fixed the issues we’ve found. When we try to fix issues, we can sometimes make them worse. Conduct research often enough that we can be sure our fixes have the intended effect.
-
We don’t just conduct research to support decisions we’ve already made. We think about how research informs design before plunging ahead into building more. We’re not just conducting research to test things we’ve already built. Conduct research slowly enough that you figure out what’s happening.
-
We have enough time to address major issues before testing again. We don’t retest and find the same issues over and over again. Conduct research at a pace that allows you to fix problems before you test again.
-
We include all the participants we should (including preferred language, assistive technology, bandwidth). Fast past research can come at the expense of groups that are harder to recruit or take longer to interview. We often forget to include people with disabilities. Or people with low bandwidth in our testing. Research isn’t complete without them. Conduct research at a pace that allows us to include and listen to everyone.
-
Our team and researcher isn’t burnt out. Research is intellectually, emotionally and physically consuming. Tired researchers don’t do their best work. Tired teams aren’t the most supportive. Conduct research at a speed that’s challenging, but not overwhelming.
Scoring the self assessment
- Did you check six or seven boxes? You’re doing well. Pat yourself on the back.
- Did you not check numbers 1-3? Consider ways to speed up your research. You’re likely conducting high quality research that isn’t as influential as it might be.
- Did you not check numbers 4-7? Consider ways to slow down your research. You may be conducting enough research, but could make better use of it.
- Missing checkmarks on both 1-3 and 4-7? Focus on getting 1-3 right, and then refining your process to meet 4-7.
- Last updated by Colin on May 27, 2019