Ensuring Patient safety through caring and intelligence

Today’s modern healthcare systems are large, integrated electronic systems that incorporate thousands of cross-functional actions in unique environments. They require extensive testing and validation. This is to ensure they are operating as the manufacturer intended and providing accurate and error-free data processing within the healthcare provider’s organization.

The FDA has mandated that electronic blood banks undergo quality assurance testing and validation after every change to these systems. The FDA and AABB consider these systems 510(k) medical devices and issue updated testing and validation guidelines annually. However, outside of blood banks, the same crucial testing and attention to changes is just as important.

How much is enough testing to ensure a quality system that is safe for patients?

Click to View Full Size

To test these highly complex systems, healthcare organizations typically employ teams of trained IT professionals armed with automated software tools and some quantity of manual test scripts. All of these testing tools require individual scripts to be written, loaded and maintained continuously to operate correctly. These scripts are single-patient, dedicated functionality focused, end-to-end “patient journeys” that are unique to the system, and version, and very specific in nature. These scripts test a minute fraction of the testable system and, as such, do not address the large-volume, en mass testing required to effectively validate all the variables and conditions incurred in a “day in the life” of these systems. These highly integrated systems are simply too large and complex to effectively maintain manually.

Traditionally, the largest part of the hospital’s testing overhead expense has been to train and retain a talented resource pool to test the system. The problem with this approach is once a single change is made, those individual test scripts now cease to function. Subsequent workarounds to the test script itself is required to get a conditional “Pass”. The major drawback to this approach is the scripts themselves cannot self-adapt to changes that inevitably get introduced into the system regularly. Scripts must be manually debugged with a subsequent trial and error process in an attempt to get a script pass each and every time they are executed.

Take this exercise and multiply it times hundreds of thousands of times across every facility with every variable… and you start to understand the extreme scope and challenges associated with validation and why it’s so difficult. These systems are in a constant state of flux and manual approaches can provide only at best single digit percentage test coverage. Today’s modern systems are simply too large and complex with too many changes being introduced daily to effectively test by traditional methods. Clinical errors continue to manifest and proliferate over time that poses a very real risk to patients — and these become even more critical as these systems evolve in complexity and integration capabilities.

Taking test scripts and their headaches and limitations aside for the moment, we must address first the quality state of the system under test. With a faulty system build configuration, system testing is a complete waste of time. Unknown errors introduce themselves in the build configuration, causing once passed test scripts to fail. Critical risks to patient safety are introduced through routine use and maintenance updates and creep in over time and lurk in the build configurations. Unless you have absolute up-to-the-minute system build intelligence to uncover these system flaws and correct them upfront, they will continue to flourish undetected and create endless havoc for test teams.

The endless cycle of script-based testing has proven ineffective, costly and dangerous to patient safety. It’s clear a different approach is necessary to successfully test and validate both the operational and quality assurance requirements for the LIS/HIS.

Introducing Vedant Health’s TestStream 

TestStream is unlike any other technology or testing approach available. TestStream is not a test tool, but instead a proprietary, broad-spectrum, intelligence-based learning engine that intimately understands the system under test and applies this system knowledge to test the system just like a human would — through the front end graphical user interface (GUI). TestStream understands your unique build configuration through its own data collection and analysis processes. Once the system is learned and understood shortly after installation, TestStream utilizes our own proprietary visual recognition technology which has the uncanny ability to “see” the screen and understand what it’s “seeing”. Because TestStream is smart enough to test the system just like a human being, it can register the patient, order the medication, and schedule the surgery. By moving the mouse and typing the keyboard, TestStream has the ability to test your entire order catalog across every department and facility as it understands your own build infrastructure and site specific configurations, options, customizations and variables.

TestStream’s artificial intelligence approach to clinical testing incorporates these three major process elements:

• Data Collection
  TestStream understands system architecture from testing hundreds of hospitals for over 20 years. Upon installation at your site, TestStream reviews the installation and the build, analyzes your system, and detects fundamental flaws that can affect system operation, quality and integrity. This first stage is critical to ensure the vendor system is installed and configured correctly before testing. TestStream understands what a perfect Epic, Millennium and Sunquest system looks like. Once the system is fully analyzed and understood, it then offers suggestions on how to locate, repair and optimize the build configuration for your own unique environment.Another benefit of this intelligence is the ability to compare different environments. Because TestStream interrogates and is intimately familiar with all your system applications, environment and domain topography, it can compare two individual data collections (for example TEST versus PROD) to truly facilitate the standardization between sites, domains, HUBs, etc. This has proven invaluable for our larger enterprise clients who depend on TestStream to standardize the enterprise across sites and domains.
• Testing
  Once the system is understood by TestStream and all the build maintenance issues are corrected, TestStream is ready to test the system on its own. From the Data Collection learning exercise, it knows about every department, orderable, middleware and downstream systems currently utilized. After test setup, it has the ability to execute order and result validation test cases to test the entire system. Test setup options are presented and after about 5 minutes of configuration, TestStream runs on its own, without fail, over days and sends email notifications as to the status and outcome of each test executed. TestStream incorporates the Data Collection learning functionality for all the associated vendor solutions for Cerner Millennium, Epic Hyperspace and Sunquest Laboratory and Blood Bank.Because TestStream is a self-configuring and adaptive intelligence, it automatically maintains its own internal test cases itself to adjust to system changes and addresses the main reason test tool approaches fail: Lack of effective and efficient change control management with quality assurance oversight. You will never have to fix test scripts ever again because someone made a change somewhere in the system.
• Quality Assurance
  All tests performed in TestStream are analyzed and graded as to the risks to patient safety and presented in our TestStream HealthScore informational dashboard. You can monitor the risks to patient safety in snapshots of time. Finally, testing effectiveness can be accurately measured and charted to help organizations improve real systemic quality and integrity. With TestStream automated change control in place, system quality and ROI grow as it is being utilized. TestStream takes the guesswork out of system testing and maintenance and answers the age-old question: “Is all this testing activity getting us anywhere?”

With TestStream’s HealthScore, you can now know for certain where your system stands as it pertains to quality assurance and safety over time.

TestStream is available for the following leading healthcare system applications: