Formative Usability Testing in Medical Device Development

Imagine a medical device development team has done preliminary analyses on who will be using the device, determined the environment in which the device will be used, and defined key device user interfaces. A use-related risk analysis was performed, which exposed possible UI design controls. These design controls were implemented into the design of the device and user interface. Now what? How do you know that your design controls have adequately prepared your medical device for summative usability testing, and when is UX/HFE ‘done’ on a project?


How do you know that your design controls have adequately prepared your
medical device for summative usability testing, and when is UX/HFE ‘done’ on
a project?

What is formative usability testing and why is it necessary?

Formative usability testing serves to validate that the chosen UI design controls are sufficient to eliminate all risk related to use safety and effectiveness to an acceptable level. Used as a detection method for UI design flaws and not simply as ‘another step’ that needs to be done to check all of the boxes enables the design team to iteratively improve the usability of their device above a threshold defined by risk.

Keys to Successful Formative Usability Testing

The first and most important aspect of successful formative usability testing is to test early and often. If formative usability testing perceived as a way to expedite HF validation by figuring out all of the use-related issues iteratively before summative testing, it should be obvious that testing early and often in a product development cycle is a way to give your product an advantage. Using even low fidelity or rapidly generated prototypes can expose most use errors and provide key design insights to incorporate into subsequent revisions. Testing early is key to capture initial design direction and decisions which may have significant usability implications. If caught early, these decisions can be amended with little schedule or budget risk; conversely, if design decisions are made without regard for usability the downstream consequences can be disastrous. Testing often ensures that design changes are not adding new use-related risks or having negative effects on existing use-related risks.

Next, remember that formative testing doesn’t require large participant sample sizes, and the format and number of iterations are flexible to each product development effort. Use related patterns can emerge after just three participants, and there are no formal requirements on how to conduct testing, so huge participant groups are not necessary. Best practice for a standard formative usability test is five to eight participants. Additionally, many use related issues can be identified by any participant not familiar with the device; while it is best to perform all testing with representative intended users, early formative usability testing can be done in house.

Finally, focusing on the user needs that the FDA cares about will help facilitate more successful formative usability testing. This means focusing testing efforts specifically on issues related to use safety and use effectiveness. Simply put, users must be able to operate the device without injury or negative clinical consequences to either the user or the patient and be able to operate the device successfully for the intended use. Design formative evaluations mindfully to expose these types of risks and you will be well on your way to a successful summative usability test.

Common Pitfalls of Formative Usability Testing

The biggest mistake that can be made in formative usability testing is obscuring use-related issues by conducting testing in poor conditions. Usually, this means overtraining participants during or before testing. Because formative evaluations are a tool to purposefully discover UX/HF issues, providing test participants advantages in simulated use conditions that they would not have in a real use scenario is shooting your product in the foot.

Also make sure to capture what worked well and what users found intuitive and easy in order to be able to take credit for that in your Human Factors Engineering and Usability Engineering (HFE/UE) Report that details the usability design process for your medical device. While you should note what didn’t work for the purpose of iterating the design and improving the issues, and demonstrate those changes result in a safe and effective final device in your HF Validation Report, you should also take credit for things that may have remained unchanged in the design but that improved usability during formative testing.

Formative usability testing is a detection method to uncover use related issues and iteratively improve on them until the residual risks related to use safety and effectiveness are acceptable. Testing early and often takes advantage of the full power or formative usability testing by incorporating UX/HFE into the whole design process.

Tensentric is a team of highly experienced engineers developing a wide range of medical devices and in vitro diagnostic systems. Tensentric has completed over 200 development projects for clients in the medical device and IVD space since the company’s inception in 2009 and is ISO 13485:2016 certified.