Enhancing CSV through Risk-Based Testing
There are aspects of each of our professional lives that draw out certain connotations simply by the nature of their presence in our daily routines. Those connotations can be positive and embraced with open arms or they can be negative and met with a reluctant embrace. With that in mind, let me throw out a phrase and I want you to do some free association. Ready?
Computer Systems Validation.
What comes to mind? Clunky? Cumbersome? Tedious? Time-consuming? Challenging? If none of those words come to mind, then kudos to you. You likely have a streamlined and efficient approach to CSV that other organizations should be working to replicate. However, if most of those words came flooding into your mind in a rush of panic and anxiety with faint hallucinations of an FDA auditor knocking on your door, you’re not alone.
The good news is that there are practical approaches to get much closer to the person/organization for whom those connotations don’t come to mind. While CSV definitely can be all of those things and is all of those things for many organizations, it doesn’t mean that it has to be or that it should be.
There are several ways of enhancing your CSV program to get away from more traditional adaptations of the compliance requirement, but one such way that I would like to focus on here is through the adoption of a Risk-Based Testing program for your GxP systems. It’s possible that this approach already exists in your non-validated application landscape, but it can and should be adopted in your validation testing efforts as well.
We're going to keep this all pretty high level, but let’s begin with some definitions…….
What is Risk-Based Testing?
Risk-Based Testing, or RBT (which you may see used from here on out since I’m a lazy typer - an attribute that is not very redeeming given that I write blogs for the company that signs my checks), is an approach to testing wherein a risk analysis is performed to establish risk profiles for applications or application areas. Those profiles then dictate the level/rigor/frequency at which said application or application area must be tested, as well as the subsequent documentation that must be captured.
Further, we can even begin to get more granular and assign risk profiles to things like individual functional requirements, or groups of functional requirements for a given application or application area.
While this is a widely adopted and well-known practice, many Life Sciences organizations have struggled, or been slow to embrace this practice for use in their computer systems validation programs, even if they already apply it to their non-regulated systems.
Why the Reluctance to Adopt Risk-Based Testing in Life Sciences?
While regulatory compliance is a very real factor in Life Sciences, many of the roadblocks that face Life Sciences teams quite often stem from comfort and familiarity. That’s not to say that all that is comfortable or familiar is ineffective or broken, but sometimes it can keep us on a path for longer than we should be. That is a bit of what we see going on at many organizations when it comes to applying RBT to a CSV program.
If it’s not broken, why fix it? If the way things are done gets you through an audit, why do anything differently?
The comfort of knowing that everything is tested to high levels of rigor with binders full of documentation (possibly metaphorically, but quite literally in some cases) feels like a safe approach. It feels like the safest option in the face of compliance requirements. If everything is blanket tested and documented all the time, then nothing can be missed, right?
Well…...we’ll get to that in a bit.
Still, it can feel uncomfortable to make the shift to a Risk-Based methodology. It does take a lot of trust to implement. Trust in your data. Trust in your processes. Trust in your people. Trust in your systems. Trust in your analysis. Trust in the justifications for your established risk profiles and subsequent tests.
It is not uncommon for us to work with an organization and for them to talk to us about risk, and how they think about risk, and how they have defined risk, but the question ultimately becomes, “now what are you doing with all that? What’s the action you take with that data?”.
Very often the reality is that not much is being done in applying those risk definitions or risk profiles in a practical way when it comes to the actual act of test planning, testing, and documentation. If the appropriate level of critical thinking and evaluation has been done up front, then that should be applied.
It does take a bit of a leap, but choosing a pilot area as a proving ground for a Risk-Based Testing program can be a great place to start moving away from legacy processes, and to start getting more organizational comfort with doing things like structuring test cycles according to risk, or setting up regression testing based on risk.
Starting with some small wins, proven success, and streamlined, yet reliable outcomes, can be a great way to initiate movement on the cultural shift that is sometimes required to implement a widely adopted Risk-Based Testing approach to a CSV program - and certainly for the transition to more of a CSA model, if that is something your organization is exploring.
What are the Benefits of a Risk-Based Testing Program?
By implementing an RBT program and having trust in the data that your team leveraged to arrive at a decision on your risk criteria and risk profiles, your organization is now on a great path to start streamlining your testing and CSV programs - And remember earlier when I said that I would get back to addressing the mindset that testing and documenting everything to the same level of rigor all the time is a safer way to ensure quality and compliance? The reality is often quite different.
By reducing efforts on testing and documenting everything, all the time, and to the same level of rigor, organizations are actually able to increase quality. They are able to reduce errors in their critical systems by applying appropriate effort and resources where they should be focused, rather than being spread too thin with efforts being applied equally across all systems, applications, requirements, etc.
Further, while there can be some upfront cost and effort associated with planning for RBT, once implemented, the savings in both cost and resource time begin to manifest quite substantially. By taking the time to put critical thinking activities upfront, and making the investment in doing proper risk assessments, and clearly defining intended use and critical features, organizations can really start intelligently reducing the scope of their overall testing and documentation efforts.
When taking into account all the different applications an enterprise owns, and all the different application types (off the shelf, configured, in-house developed, etc), and the lifespan of each of those applications, there will be a direct and measurable impact on cost associated with each when RBT is applied. In addition, if taking into account things like MSP models, or offshore testing models, resource attrition and utilization becomes increasingly important as costs can quickly start to balloon in these areas under a loosely defined, or blanket testing model. RBT is an extremely effective method for keeping those costs and resource models in check.
Remember, the FDA never said you have to test and document everything, or that you had to test and document everything to the same level of rigor. That was never the intent. The intent is to understand business processes and to understand intended use. An investment in a proper Risk-Based Testing program is a way for an organization to not only be able to understand those things but to be able to adequately justify their classifications and to be able to test and verify accordingly.
Closing Thoughts and Resources
While we have only touched on RBT at a high level, the benefits from a quality, cost, and effort standpoint can be extremely valuable. If your organization has not yet adopted this type of approach into your regulated system, I would highly encourage you to take a serious look at how your team can begin to pilot and incorporate it into your processes. Risk-Based Testing is a crucial component for moving further up the testing and validation maturity models we have discussed in previous blogs. Here is one example you can reference: Use Case - Applying Testing and Validation Maturity Models.
If you would like to discuss this in more detail, reach out to discuss how to incorporate Risk-Based Testing into an overall SDLC and CSV Modernization strategy. We would be happy to see how we can help your team adopt this approach.
We also recently did a webinar featuring our VP of Strategic Solutions, Dori Gonzalez-Acevedo, and Biogen’s Sr Manager (TCoE) - ALM Platform & PO&T IT, Ajay Pandey. They discussed how to leverage Tx3 VERA and Micro Focused ALM for Risk-Based Testing in CSV. Feel free to view it here:
Jason Secola manages content marketing and channel activities at Tx3 Services and has been with the company since 2016. Jason began working with the larger portion of the existing Tx3 team dating back to 2007 when he got his first start in the world of application testing which ultimately led to a focus on testing in a regulated environment. He currently resides near Sacramento, CA.
Jason Secola manages content marketing and channel activities at Tx3 Services and has been with the company since 2016. Jason began working with the larger portion of the existing Tx3 team dating back to 2007 when he got his first start in the world of application testing which ultimately led to a focus on testing in a regulated environment. He currently resides near Sacramento, CA.View all posts by Jason Secola