Methods for analytical validation of novel digital clinical measures: A simulation study
Simon Turner, Chen Chen, Rolando Acosta, Rachell Chon, Eric J. Daza, Lysbeth Floden, Joss Langford, Leif Simmatis, Berend Terluin, Benjamin Vandendriessche, Piper Fromy
Abstract
Analytical validation is a crucial step in the evaluation of algorithms that process data from sensor-based digital health technologies (sDHTs). Analytical validation of novel digital measures can be complicated when reference measures with directly comparable units are not available. To address this, we conducted a simulation study. Data was simulated assuming a latent physical ability trait, indirectly accessed through an sDHT-derived target measure collecting step count data, and the items of a clinical outcome assessment (COA) measuring self-reported physical activity.
Introduction
For a sensor-based digital health technology (sDHT) to aid scientific and clinical decision-making, its sensors must be verified against a technical specification, algorithms analytically validated, usability validated, and clinical validation conducted to ensure relevance in specific contexts, as outlined in the V3 + framework by the Digital Medicine Society (DiMe) [1,2].
Methods
We evaluated statistical methods for analytically validating a digital measure (target) against reference measures of clinical outcome assessments (COAs). We simulated total daily step count data from a fictive sDHT as a measure of physical activity. The associated latent trait is physical ability which fluctuates daily. Additionally, we simulated COAs that measure patient or clinician-reported physical activity.
Results
Primary aim of the study
Table 2 depicts a summary of the mean empirical bias and mean empSE for the PCC and CFA methods under the influence of different simulation conditions, as well as an overall summary across all conditions.
Discussion
Summary of results and how they help in practice
Our simulation study highlights the challenges in validating novel digital endpoints against reference measures with non directly comparable units. While sDHTs offer the potential for increasingly more objective and continuous methods of measurement, rigorous analytical validation is critical to support the continued development and implementation of such novel digital clinical measures.
Acknowledgments
The authors gratefully acknowledge the contributions of the following experts through participation in the statistical advisory committee and async review of the simulation protocol and results: Chakib Battoui, Jakob Bjørner, Yiorgos Christakis, Valentin Hamy, Andrew Potter, Bohdana Ratitch, David Reasner, Colleen Russell, Sachin Shah, Berend Terluin, Andrew Trigg, Kevin Weinfurt, Robert Wright. In addition, the authors gratefully acknowledge the contributions of DiMe members for their support: Samantha McClenahan and Bethanie McCrary.
Citation: Turner S, Chen C, Acosta R, Chon R, Daza EJ, Floden L, et al. (2026) Methods for analytical validation of novel digital clinical measures: A simulation study. PLoS One 21(5): e0308190. https://doi.org/10.1371/journal.pone.0308190
Editor: Ambiga Natesan, Sri Akilandeswari Women’s College, INDIA
Received: July 18, 2024; Accepted: March 16, 2026; Published: May 13, 2026
Copyright: © 2026 Turner et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All simulation code, simulated data, full results, the data generating mechanism, and guidance of how to adapt these materials to different digital measure scenarios, are available open-access through the website of the Digital Medicine Society (https://datacc.dimesociety.org/validating-novel-digital-clinical-measures). The code and simulated data is housed on GitHub (https://github.com/Digital-Medicine-Society/VNDCM-Simulation-Toolkit).
Funding: This work was supported by Arnold Ventures (grant number 23-08673). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests: ST is a contractual employee of the Digital Medicine Society. CC reports employment and equity ownership in Verily Life Sciences. PG is a contractual employee of the Digital Medicine Society and President of SeeingTheta. The Digital Medicine Society received a grant from Arnold Ventures (URL: https://www.arnoldventures.org) to support this work (grant number 23-08673). Arnold Ventures had no involvement in the design of the simulation study, analysis and interpretation of the simulated data, or writing of the report; nor do they require any restrictions regarding the submission of the report for publication.
