Validation and verification framework and data integration of biosensors and in vitro diagnostic devices: a position statement of the IFCC Committee on Mobile Health and Bioengineering in Laboratory Medicine (C-MBHLM) and the IFCC Scientific Division

a. Explicit context of use

1. Each biosensor device must define its intended use and eventually its role in the clinical care pathway, if applicable (measurement of blood pressure, heart rate, glucose concentration). We can roughly define biosensors for the purpose of use into two main groups: biosensors that are used for medical (monitoring chronic patients) or non-medical purposes (well-being purposes, monitoring the state of individuals during their activities i.e. athletes).

2. According to this definition, various necessary instructions for the user should be described in the instructions for use. As an example of medical use, are glucose meters that are intended to monitor the glucose concentration over a longer period, for the purpose of glucose regulation and therapy of diabetics. In the case of medical use, it is necessary to clearly define that their use does not replace but complements/supports the monitoring of their disease, while/next to monitoring other parameters (HbA_1c), which are determined in clinical laboratories upon request of the treating medical doctors.

b. User interface/user experience

1. Each biosensor must have a defined user interface/software that enables the connection of the device/application with other devices (computers) intended for monitoring and analyzing (follow-up the trend) the measured parameter.

2. Instructions for use must be clearly given, which must be adapted according to the different (minimum) previous experiences of the user.

3. It is necessary to define acceptable limits of performance of the measured biosensor-derived test result considering the intended use of the test; also, inclusion of warning signs or sounds in case of deviations must be considered.

4. Since the biosensors are intended for wider use by untrained individuals, it is also desirable that each software has the possibility of easy display of results (in the form of curves, arrows) and their interpretation in the form of different colors (e.g. green-acceptable; yellow-warning; red-unacceptable, necessary action) or emotions (☺, ☹)

c. Device specification

1. Each biosensor must have defined dimensions (width, height, depth), device weight. It is necessary to indicate the place on the body where the device is also installed, with clear instructions on how to install it correctly, to ensure adequate measurements.

2. It is also important for the manufacturer to state the expected lifetime of the device, within which it guarantees the adequacy of its operation.

3. It is important to define possible cases of improper installation/use or to clearly state other restrictions/influences that would prevent improper use or affect the correctness of the measurements.

d. Measured digital biomarker

1. Each manufacturer of biosensors must clearly define the measurand intended to be measured, and should describe its molecular characteristics (clinical importance),

2. It should be defined whether it is possible to co-measure other parameters of interest with this device, and if so, how they are dependently/independently related to each other.

3. Insofar as the measurand can also be measured in other biological materials (e.g. blood glucose), with other standardized measurement systems (in the laboratory), it is recommended to define the relationship/ratio of concentrations for reasons of data consistency, and support it with evidence (references of the conducted research)

4. It is also important to clearly define the measurement system and its possible limitations (interferences, selectivity, analytical sensitivity, clinical sensitivity and specificity, special conditions for appropriate use)

e. Technical data

1. For appropriate use, biosensor performance and interpretation of measured values/obtained results, it is important to know the technical data of the measuring system and the determination method.

2. It is necessary to clearly define the site of collection (location on a specific part of the body, skin, under the skin, on the muscle), the required (minimum or required) volume of the sample, if it is a collection.

3. The manufacturer must provide information on the measurement time; how long does it take to get the result and what is the maximum time range for monitoring the parameter on the device (data storage size) or is it necessary to transfer the data to another computer, for which the exact data must be specified for the necessary connection/transfer.

4. It is necessary to define and specify the analytical performance characteristics of the biosensor device (detection limit, linearity, measurement range) which enable appropriate use and interpretation of the results. It is also important to note that post market surveillance is demanded during the entire life cycle of an IVD type of biosensor.

Accuracy

Biosensors and wearable devices monitor biometric data, like blood pressure, heart rate and physical activity, as well as analytes such as glucose and electrolytes. They are factory calibrated by the manufacturer and the user cannot change the calibration. These sensors test body fluids other than blood. So, metrological traceability to a blood-based standard may not apply to fluid matrices, and a fluid-based standard has not been established for these devices. For example, continuous glucose monitors (CGM) measure glucose in interstitial fluid, while glucose meters measure capillary blood collected from fingersticks. Blood gas glucose sensors utilize arterial or venous samples, while central laboratory instrumentation requires venous plasma or serum for glucose testing. Laboratory glucose methods are metrologically traceable to a plasma-based standard with glucose analyzed by mass spectrometry. Whole blood measured by blood gas glucose sensors can be linked to the same plasma standard by comparison to traceable laboratory glucose methods using the same sample set (whole blood analyzed by the blood gas instrument, centrifuged to separate plasma and analyzing plasma by the traceable laboratory method). However, CGM measures interstitial fluid and collecting sufficient interstitial fluid for laboratory analysis is not feasible. Hence, a head-to-head comparison using a matrix derived from the same specimens is not possible. CGM traceability is also challenged by the delay in equilibrium between glucose in the vascular system and interstitial spaces after a meal.

The US Food and Drug Administration recommends comparison of a venous sample analyzed by a traceable laboratory method to establish CGM accuracy [30]. However, the ease of capillary fingerstick collections and rapid analysis by glucose meters has also been utilized since CGM is a clinical replacement for glucose meters and current diabetes treatment protocols have been established using glucose meters. Alternatively whole blood analyzed by a YSI 2300 or YSI 2900 analyzer that has metrological traceability is also an option for establishing CGM accuracy. So, establishing traceability and accuracy of mobile health devices can be challenging. In the EU and in the IVDR metrological traceability of biosensor test results to higher order reference materials, methods and systems is demanded, if available. Calibration hierarchies that describe how to enable metrological traceability are described in ISO 17511:2020. Note that the metrology approach in the EU is different from the USA FDA approach where comparison to predicate devices is the norm.

Measurement uncertainty

A separate issue is defining allowable measurement uncertainty, according to ISO 17511. For conventional biomarkers, the Milan hierarchy describes how to deduce analytical performance specifications [31]. For CGM, until recently, minimum accuracy requirements have not been defined [32]. Allowable error limits have not been established for many mobile devices, and total allowable error limits may be different for biosensors compared to laboratory methods. Sensors not only generate data at a single point in time, but they also make frequent measurements, every several minutes, and can provide continuous data trends such as rate of change (rise or fall) over time. This allows for the ability to predict future events and alarm the wearer before the event occurs. CGM, for example, can warn the wearer of impending hypoglycemia and alarm before the patient experiences a hypoglycemic event. This allows for intervention preventing potential harm. This ability of mobile devices to trend and alarm the wearer may allow a greater tolerance for bias compared to laboratory methods. Total allowable error limits for accuracy of biosensors will have to consider these factors for clinical outcome.

Precision

There are two considerations for defining precision of mobile devices and wearable sensors – within sensor variability (on the same individual) over the life of the sensor and between sensor variability (between 2 different sensors worn by an individual) over time. These two sensors may come from the same manufacturing lot or different lots and present with different precision. Variability can be measured by wearing two sensors (one on each arm for example) and determining the difference between the sensors over time. Because the sensors average results over time, some of the variability (and bias as well) may be smoothed as the device software calculates data trends. Comparison of biosensor results with a comparative method, like CGM to a glucose meter, can be confounding by variability in both methods. So, establishing true device variability is challenging and the wearer may be better defining how the results will be used in clinical decision-making to change treatment or lifestyle factors rather than comparing bias or variability to laboratory methods.

Quality control and EQA

Since biosensors are wearable devices, analyzing a control sample or testing a proficiency material is not possible by the end-user. Quality control and traceability of sensors is more the responsibility of the manufacturer and challenging to establish by the wearer of the device.

Data interpretation, transfer, security, and privacy

Sensors involves physical, network, and application layers [33]. Implementing biosensors in health systems requires understanding complex security challenges [6]. Data obtained by the sensors and devices can only be transferred to health records with patient’s consent. In the data-transfer, the application programming interface (API), which makes the parts of software and application available to the outsides and can share the data with the other software and application developed by third parties [34, 35]. Under the security of systems and protection of patient’s privacy, the data can be shared among such record systems (i.e., electronic health records [EHR] and personal health records [PHR]). The data should be easily interpretable and useful for health/medical care in the individual patient, facility, community and global situation. For the data-sharing systems, there are currently standardized formats such as DICOM (images) and HL7 FHIR (for instance, it is used in laboratory information systems) [34, 35]. The data for the sharing systems require assurance of compatibility and exchangeability among the measured data.

Validation of the sensor data collection system

In modern healthcare systems, sensor technologies can prove to be an integral part of processes such as improving efficacy and lowering the cost. For clinics and patients’ day-to-day experiences, emerging biosensors can help prove to be potential help resulting in major improvements in care delivery system. In general, the appropriateness of using a particular product, can be measured by a term “validation” (e.g., validation of wristwatch measuring heart rate and temperature). Further, “GxP” is a generalized abbreviation for “good practice” and includes compliance to a set of codified quality guidelines, regulations, quality management systems, controls which have been developed into the various specializations for stakeholders such as manufacturers, industries, clinical trials (e.g., clinical trialists, manufacturers, laboratories). Validation is evaluating analytical and clinical performance claims mentioned in the Instructions for Use based on raw and derived measurement data of the sensor and algorithms.

The V3 qualification framework encompasses the following three steps of connected sensor technology: verification of the sensor technology, and validation of the analytical & clinical performance and the fitness-for-clinical-purpose of the biosensor. This three-stage process of verification, analytical and clinical validation (V3) proposed by Goldsack, Coravos, Bakker et al. accounts for the unique hardware, software, and algorithmic properties of connected biometric monitoring technologies (BioMeTs). However, it is always challenging to evaluate a connected sensor technology’s data supply chain, the data flow and data provenance for information generated from hardware, sensors, software, and algorithms (31, 32).

Connectivity (wifi, bluetooth, NFC…)

Wireless technologies have made significant progress, and they are now being integrated into many mainstream applications and medical devices. In particular, Bluetooth technology has now been utilised in a variety of medical applications ranging from home-healthcare devices to room equipment. Bluetooth well suited role for cable replacement and mobile connectivity. It is a relatively low-cost technology which also provides excellent security and reliability and coexists well with other wireless technologies. It also uses virtually no power and, because it doesn’t travel far, are theoretically more secure than wireless networks. The first mobile device that incorporated both communication and computing features was the Blackberry, which was introduced in 2002. Subsequently, Apple and other smartphones based on the Google Android operating system were introduced for better functioning and desired outcome. Many developers, solution providers and government agencies utilised Bluetooth technology especially during COVID19 era to implement innovative solutions that helped managing the spread, accelerate reopening efforts, with safer treatment of patients during the disease outbreaks. Bluetooth enabled proximity warnings through phone app based technology to remind citizens and visitors to maintain safe distances has helped managing to reduce the risk of viral transmission. Thus, flexibility of Bluetooth technology and easy availability in our phones and devices is helped us to minimize the spread and enabled safe reopening [36, 37].

Interoperability

Recently, BLE (Bluetooth Low Energy) such as iBeacons or Beacons has become a recent alternative for Wi-Fi, especially in IoT devices. BLE are used to send data over short distances and found more suitable for transmitting small amounts of data at 1 Mbps, like sensor readings of temperature, acceleration details, GPS coordinates. However, BLE is not suited for sending data in real-time to a server. LE signals can be picked up by any Bluetooth 4.0 enabled devices. For Android devices, Version 4.3 or later would be perfect, while for Apple devices, the technology runs on Version 4S or later. Wi-Fi works on WLAN 802.11 a/b/g/n/ac devices. For Android devices, Version 4.0 or later is the preferred choice for Wi-Fi connectivity. Bluetooth devices have lower power consumption, but Wi-Fi devices consume more power, and for that reason broadcast range of both technologies is different. Both the technologies emit a signal with a frequency of 2.4 GHz. The iBeacon signals travels for about 30 feet, however Wi-Fi signals can travel over 10 times more distance. Thus, Wi-Fi requires 10 times more power than BLE, even if performing a similar task. Wi-Fi devices need considerable power, about 500 µW for ten messages per day, while BLE consumes only 50 µW [38], [39], [40] (Tables 1 and 2).

Security – cybersecurity

The user authentication is the process to keep a user’s personal information confidential in digital devices. The management systems by identity are classified as the one-factor authentication (user knows), two-factor authentication (user must be) using one-time passwords (tokens), and three-factor authentication (user status) using biometrics such as iris scanners and voice recognition [34, 35]. There are currently several ways to guarantee data security. First, a single sign-on allows using the multiple systems with a single user authentication. It streamlines the access to EHR and other applications by eliminating the need to repeatedly enter the user’s names and passwords manually [34, 35]. Second, a gateway security is a specialized device and/or software that protects the systems in a network from unauthorized intrusion, viruses, and other threats [34, 35]. Third, a firewall, which allows or denies the network communications, includes the basic style of ‘packet filtering firewall’ that can restrict access to specific Internet Protocol addresses (IP addresses) within an organization as well as the more protective style of ‘stateful inspection firewall’ that can check the complex correlations between internal and external IP address connections [35, 41], [42], [43]. Forth, a blockchain uses cryptography (e.g., digital signatures) to control data in a decentralized manner. The data are distributed among participants in blockchain, and since many participants keep a copy of everyone’s transaction history, it is impossible to tamper with (even the service provider cannot tamper with or erase the data). Furthermore, there is the advantage that, even if some computers go down, the entire system does not go down.

Sensor design (small-size, lightweight, wearable, non-/light-invasive)

The Internet of Things (IoT) is an information carrier based on the Internet, traditional telecommunications networks. The information interaction between people and things is the core of the IoT. Radio frequency identification (RFID) is a significant and widespread application scheme for IoT systems. RFID is not only a communication for two-way between two objects, but also has a unique identifier and can be equipped with various sensing functions. The near-field communication (NFC) is developed based on RFID technology combined with wireless interconnection technology. It is also considered as a subset of RFID. NFC uses the 13.56 MHz frequency band for fast communication of short-distance devices.

Sensors are classified into location, physiological, image, inertial, environmental, binary and tags. Indoor location sensors include wearable and ambient sensors. Parylene-C has been widely used as a surface coating for various materials due to its excellent physical and chemical properties, such as superior flexibility, high transparency, easy preparation. It can be used as new wearable materials due to their ultra-thin, highly flexible, and easy-to-peel properties. A conductive polymer is an additional type of wearable material, which has been widely used in wearable sensors. Conductive polymers are different from conventional organic polymers, as they have high electrical conductivity, electron affinity and redox activity. A hydrogel is another type of wearable material. Like ionic liquids, hydrogels are typically conducted through ionic motion in a hydrated polymer matrix. The modulus of hydrogels may closely match human skin.

User engagement (convenient, stable, non-training, clearly aimed, easily interpreted data, secure, inexpensive)

User engagement means the strength of relationship between a user and a service as sensor and device systems [35, 41], [42], [43]. As an example, the user engagement with a smartphone application is suggested as a user’s emotional, cognitive, and behavioral experience. The engagement might be associated with clearly aimed, less trained, convenient, stable, easily interpreted, secure, and inexpensive characteristics of sensor and device systems.

Communication (data-sharing, interactive)

The wearable devices with wireless communication data transmission provide new capabilities for real-time monitoring and dynamic responses. Wireless communication technologies, such as RFID, NFC and Bluetooth, have been widely used for information transmission in wearable electronic devices. Peer-to-peer (P2P) communication is possible between NFC tags and smartphones. Wireless communication technology is required to seamlessly stream important information to the users.

Ultra-high-frequency RFID (UHF RFID) has a long transmission range, but the readers are very expensive (USD 1000–2000). UHF RFID is also more susceptible to environmental impacts, leading to losses.

Low-frequency RFID (LF RFID) has been commercially available for a long time with wide range of applications. The wireless communication of Bluetooth (3–200 m) has a greater read range and requires less power than NFC. NFC are the better protocols for use in transmitting data from wearable sensors due to their low cost, low power consumption and portability. Thus, the goal of wireless communication technology is to transmit relevant and appropriate information data to the user or clinician in cost effective manners with no data loss.

Medical safety (alarming function, direction of emergency, attentive watch)

Sensor and device systems available to health/medical care should guarantee the user’s medical safety. For instance, the systems with some functions of an alarm for adverse phenomena, a direction of emergency and an attentive monitoring seems to be applicable to the medical safety.