Currently, there are frameworks that systematically assess the use of health-related technologies, such as the Health Technology Assessment Core Model (Lampe et al., 2009; EUnetHTA, 2016), frameworks on digital health (without specifically considering public health), and frameworks on public health. Given the rapidly increasing digital technologies designed for public health interventions, a framework for digital public health interventions is deemed necessary. The current DigiPHrame aims to be comprehensive. Therefore, users need not draw on multiple frameworks for their assessments. Among others, it considers technical aspects, implementation, ethics, and data security.

Each criterion in the framework is framed as a question. Users are encouraged to apply these questions to the digital public health intervention or health technology they want to develop and assess. Not all questions will be equally important or pertinent for all interventions, and some may not be applicable. For some, there may be no robust evidence available to answer the question.

The framework does not provide methodologies related to the questions. For some questions, common sense may be enough; for others, specialist expertise may be required. For an intervention under development, a first orientation might be enough to determine whether it is worth continuing along the determined path or whether adjustments are necessary.

In summary, the framework’s application is primarily user-led. At a minimum, it can serve as a checklist to help avoid overlooking key issues relevant to the intervention’s performance.

The proposed framework is the result of a four-step process. First, a scoping review was conducted to identify existing Public Health and Digital Health frameworks for developing and evaluating health interventions in primary prevention and health promotion, along with their assessment criteria (OSF scoping review protocol registration). Second, all assessment criteria collected from the scoping review were analysed and mapped into domains based on the structure of the Health Technology Assessment (HTA) Core Model (Lampe et al. 2009; EUnetHTA, 2016). Third, multidisciplinary Digital Public Health experts from the Leibniz Science Campus Digital Public Health (LSC) were invited to a scientific consensus meeting to discuss and validate each framework domain. These steps formed the first version of the DigiPHrame, which is regularly updated in step four based on expert consultations, case study assessments, emerging literature, and intervention requirements driven by technical advancements.

The information sources for the scoping review included articles published in scientific journals. Journal articles were identified using the electronic literature search function of international bibliographic databases, along with a manual search of relevant reviews’ reference lists. The electronic bibliographic databases used for the literature search were MEDLINE (via PubMed), Scopus, IEEE, CINAHL (via EBSCO), and PsycINFO (via Ovid). The search syntax in the bibliographic databases included the following basic keywords, specific search fields, and Boolean operators: (“Public Health” [Title/Abstract] OR “Digital Health” [Title/Abstract]) AND Evaluation [Title] AND Framework [Title]. The search syntax included synonyms and the most relevant subject terms of our primary keywords in each concept. A truncated term with the wildcard character was used when appropriate to maximize sensitivity while striving for reasonable precision. The final search was executed on 12 April 2022 with no language or publication date limitations.

The inclusion criteria included English-written journal articles, reports, or theses that depicted a descriptive framework. The aim of the frameworks was to develop, monitor, validate, evaluate, or implement interventions in public or digital health, focusing on primary prevention or health promotion at the population level. A total of 9,011 articles were identified after database searches. After deduplication, 4,830 titles and abstracts were independently screened by two researchers. A total of 433 full texts were assessed by two independent researchers for potential extraction. Disagreements between the two researchers at both stages were discussed. If no agreement was achieved, a third researcher was involved in making a conclusive decision. After the full-text screening, 68 articles were included for data extraction.

Data from all 68 articles were extracted, specifically including the framework criteria and sub-criteria. These were inductively analysed as a first step into clustered domains suggested by the EUnetHTA Health Technology Assessment (HTA) Core Model (Lampe et al. 2009, EUnetHTA 2016). The multidisciplinary experts from the LSC were assigned to domains they conferred and restructured based on the specific characteristics of digital public health interventions. Where necessary, additional literature was consulted.

Version 1.1 of the framework was sent to a multidisciplinary expert panel consisting of 105 members of the LSC. They were invited to give written feedback and to participate in an online scientific consensus meeting to discuss and validate each domain of the framework. A total of 25 members participated in the meeting on 19 July 2022.

As the field of Digital Public Health rapidly evolves, the framework has been designed as a living document. In the coming months, it will be applied and tested on a heterogeneous set of digital public health interventions. As this occurs, additional literature will be integrated, and experts beyond the LSC will be consulted.

If you find items missing, questions difficult to understand or operationalize, or if you have applied the framework and would like to share your experiences, we are very keen to hear from you. Also, if you are interested in participating as an expert in the external consensus meeting, contact us with your expertise and a brief overview of your area of interest.

In these cases, or if there is other relevant feedback, please send an e-mail to:
framework(@)lsc-digital-public-health.de