The Key Takeaway
Minnetronix’s OneBrain case study shows how AI and Traditional Machine Learning techniques can inform medical device requirements and design.
In the case of OneBrain, our robust data science selection and development process resulted in:
-
Added flexibility through identification and early correction of data issues hidden in large swathes of data.
-
Saved cost and timeline by analysis to determine required sampling rate and most important data features for our application.
-
Minimization of future risks by working with domain experts to ensure a meaningful and repeatable output.
Flexible Capabilities for Data Science
Data science is a critical yet often misunderstood aspect of medical device development. And because medical devices have strict regulatory requirements for data, there is little margin for error in effectively incorporating data science into a device development project. Specifically, you must consider two significant barriers: the cost of change and the cost of input data.
To address the cost of change, companies must carefully plan for the future at the outset. From a data science perspective, this means selecting the optimal data science approach for the device based on both current design and future plans in the product roadmap. A typical example is that many companies are developing a first-generation commercial device while also planning for artificial intelligence & machine learning to be integrated into future generations of the device. In these cases, starting on the right data science path early in the development cycle will be vital to ensure you can meet current and future plans, without incurring dreaded cost or schedule blow-ups.
To address the cost of input data, companies must find ways to make more with less. From a data science perspective, this means following a rigorous algorithm development process that incorporates the clinical context, knowledge of clinical applications, hardware design implications, and regulatory strategy to optimize the amount of insight that can be extracted from costly and often scarce input data.
Minnetronix offers full-scale data science capabilities as a part of our core services. We are a single partner that excels at the junction of medical device and data science, meaning that our clients don’t need to manage the algorithm development themselves or seek additional support from an outside data science vendor. When partnering with Minnetronix, our data science processes are built into the core medical device development program and handled by our team of data scientists. Instead of running siloed workstreams that merge at the end, this integrated approach ensures better overall device performance and fewer development headaches.
The best way to demonstrate our data science aptitude is with a case study. This project, called OneBrain, demonstrates the importance of integrating clinical context, medical device knowledge and data science at the outset and reveals that Minnetronix’s data science team can derive significant value for our partners in their medical device development program.
“Medical device designers, in particular, must consider two significant barriers: the cost of change and the cost of input data.”
OneBrain: Predicting Neuroworsening
Algorithms are often built to support medical devices. However, sometimes the opposite is true. Instead of building an algorithm to support a device, data scientists might create an algorithm with the ultimate intention of incorporating that algorithm into a new medical device or technology. The risk, of course, is that the algorithm you create requires technology or hardware that is impractical or impossible at the intended price point or call point. However, the potential benefits are huge; developing the algorithm ahead of the device de-risks what can often be a very costly development cycle. In this case, by developing the algorithm first, our data scientists and engineers were better able to define and translate algorithm inputs to device requirements, ensuring viable but not extraneous inputs are acquired, ultimately translating it to device requirements that are practical for the use environment. Through early planning, a company can seize an opportunity to flip the order of operations of normal device development. There is no better example of our capabilities in this regard than OneBrain.
Patients in the Neuro-ICU often decline cognitively. This phenomenon, known as neuroworsening, is complex for physicians to predict. A device that monitors neurological signs and predicts neuroworsening could help physicians provide earlier medical intervention.
Multiple monitoring devices currently collect data that is relevant to neuroworsening in the Neuro-ICU, but do not paint the whole picture on their own; and what’s more, these devices do not interact with one another. In the case of OneBrain, unifying these data sources provided critical insights that the sum of the isolated sources could not.
After the Minnetronix team identified this opportunity, work began toward building an algorithm that would ultimately inform physicians and prevent neuroworsening. But before making the algorithm, Minnetronix had to select the right data science approach.
“Beginning algorithm development ahead of the device de-risks what can often be a very costly development cycle.”
Selecting An Approach
One critical aspect of any medical device algorithm is selecting the data science approach that minimizes the high potential cost of change. Each approach introduces different trade-offs that will occur immediately and in later development stages.
Minnetronix has developed a framework to aid in the selection process and provide an approach to data that minimizes future risk. We generally consider four approaches to data science. Each provides benefits and drawbacks, according to six competing characteristics. To illustrate these characteristics and how they play into the selected approach to data science, we will consider the decisions made during the OneBrain project.
We understand the importance of defining requirements before the design of a medical device. This understanding led us to investigate the predictors of neuroworsening before creating our solution. Defining these requirements up-front will allow for a smoother design and manufacture of a device to predict neuroworsening.
With OneBrain, Minnetronix is designing a device to meet the algorithm’s specifications to predict neuroworsening without providing costly extraneous inputs to add to the device design and risk acquiring approval.
After our team assessed the unique needs for OneBrain, they chose the traditional ML/AI (ML/AI that isn’t deep learning) category as the best approach to meet the project’s needs. This choice ensured that the algorithm could meet performance requirements while providing enough cost and timeline certainty to the project. As a result, we felt comfortable that we could correctly identify the needed inputs for our algorithm.
Assessing the Needs for OneBrain
We chose the proper approach based on six competing variables and tweaked it to fit our desired goal.
The amount of available data informs which data science paths are feasible. In the case of OneBrain, the algorithm needed to aggregate and interpret a large amount of data. Therefore, we prioritized an approach with the capability to leverage large quantities of data.
Some algorithms are easier to interpret and translate to clinicians. For the sake of this project, the algorithm only needed to predict neuroworsening rather than provide additional clinical insights.
Larger data sets require more hardware resources. We assigned hardware utilization resources a medium priority since the cost, but not the feasibility of the device, was likely dependent on it.
This project was exploratory, so cost and timeline certainty were not a priority for the approach to data. Emphasis on cost and timeline are more common in cases where Minnetronix designs an algorithm to support a medical device.
For this project to be successful, our algorithm needed to signal neuroworsening better than any singular vital sign (HR, spO2, ECG) could currently do. As a result, the capability ceiling requirement for this project was high.
Post-market growth had a medium requirement. Because of the exploratory nature of this project, proving efficacy and potential for the algorithm to improve with more data held more importance.
Following the Minnetronix Process
After we mitigated the high cost of change barrier by selecting the right data science approach, only one barrier remained: the high cost of input data.
A standardized algorithm development process is vital to a device’s success. This importance exists because the cost of input data is high for medical devices. As a result, any partner needs to ensure they make the most out of that scarce data.
Minnetronix focuses on medical device development and manufacturing, meaning we have the relevant knowledge that applies to your data and know how to optimize inputs to minimize device requirements while maximizing performance.
Regardless of which approach to data science is best, every project follows Minnetronix’s trusted seven-step process. This process produces valuable data and viable products for our clients. In this instance there were, in particular, some steps that were most critical for the success of OneBrain.
This step involves identifying skillsets required to complete this project and finding necessary partners. Our engineers and data scientists realize the value of clinical context to achieve your medical device clearance and commercialization goals and have the experience and willingness to work with clinical experts to achieve these goals. In the case of OneBrain, much of this capability was in-house.
In the planning stages of the OneBrain project, we identified the following needs and resources to fill those needs:
- Data Science Expertise (in-house)
- Medical Device Design and Manufacturing (in-house)
- Clinical Consultation (in-house and partnership)
- Data Access and Hospital IT expertise (partnership)
Minnetronix is an expert in data science and device design and manufacturing, and we regularly identify and collaborate with partners to fill any gaps we identify in planning.
We leveraged relationships with an existing hospital and hospital IT experts to organize the collection of 20 TB of data from two thousand patients prospectively over 18 months. Data came in many forms, including vitals, EMR, CTs, and more.
This step also involved an agreement to extract data at 6-month milestones throughout the period. This practice allowed for determining data transfer capabilities early and correcting problems while our partners were still directly invested in the process. The agreement also allowed the architecting, organizing, and data analysis to begin in parallel with the data collection effort.
Python scripts were built and optimized to unpack nearly 20TB of waveform data and verify its accuracy within reasonable time constraints. verifying the data in the first export, we identified a host of problems. These led to short term fixes on our end and necessitated a fix from our technical partner for future exports. Our planning to receive multiple exports throughout the data collection paid dividends here, allowing us to correct the issues and avoid a timeline delay.
To store the data for analysis, we architected a database and connected it to a development environment that allowed for quick sorting and analysis of data. Further checks on the validity of the data were then performed and resulted in the identification and eventual resolution of several other issues
One of the critical components of the exploration and analysis was the decision on how to define neuroworsening. We leveraged our partnerships with thought-leading neurointensivists and neurosurgeons to evaluate options to represent neurological status. We ultimately chose neurological scales, since research has highlighted the significance of changes in these scales, and post-processing can improve their predictive power.
We ultimately investigated a bevy of relationships between predictors and neurological scales. The result was the identification of HRV, spO2, and labs as possible predictors of neuroworsening–which we measured via neurological scales.
We evaluated many combinations of algorithms and inputs and compared them using a common set of metrics. The best-performing combinations provided insight into the value of the inputs and what sort of inferences made algorithms successful. Ultimately, this stage allowed us to identify the most important inputs to our algorithm moving forward.
A key outcome of our analysis was the revelation that fewer and less densely sampled HRV metrics provided nearly the same predictive power as many HRV metrics in concert. Since algorithm input needs directly tie to device requirements, this allowed us to ease our device requirements, ultimately shortening the timeline and lessening the cost of the device. Sometimes, revelations such as these can fundamentally alter the financial viability of your product.
Currently, spO2, HRV, and lab results have all shown promise in predicting neuroworsening. Early testing has shown these results generalize to the new sets of data. The project is currently finalizing testing on the latest data and preparing to evaluate the current level of accuracy.
After testing and engagement, we will need to perform any changes to meet physician standards and begin the process of building a device around what the algorithm needs. This process is Minnetronix Medical’s specialty. Minnetronix provides whole product solutions that facilitate viable medical devices–including the OneBrain project.
After building a device, we’ll need to seek regulatory approval and commercialize it. Minnetronix provides an unparalleled level of experience in this regard. Our team has facilitated the approval and commercialization of many devices while incorporating the needs of our customers.
Partnering with Minnetronix
The OneBrain case study displays Minnetronix’s flexibility and expertise in incorporating data science into medical device development. Not only is our team able to use data to create viable medical technologies, but it is also able to utilize existing data sets to inform new products.
Few medical device partners can provide multiple approaches to data science in tandem with traditional device development services. This lack of expertise can necessitate working with an outside data expert or interpreting the data yourself. Working with an external data service has many disadvantages, like a lack of integration with your hardware, an absence of medical or regulatory knowledge, and an inability to choose the right approach for your overall business constraints. With Minnetronix, however, our customers can be confident in our ability to meet your needs for data science.
Even though the cost of change and input data is high, Minnetronix was able to facilitate a successful development program with OneBrain. Our expert team accomplished its goals using a robust selection and development process.
In the case of OneBrain, these processes resulted in
-
Added flexibility through identification and early correction of data issues hidden in large swathes of data.
-
Saved cost and timeline by analysis to determine required sampling rate and most important data features for our application.
-
Minimization of future risks by working with domain experts to ensure a meaningful and repeatable output.
With Minnetronix as a partner during development, you can make the most out of your data by utilizing our proven process, knowledge of clincial applications, and data scientists.
Interested in learning more? Click here to contact us!