https://orcid.org/0000-0003-2318-1783
Figures
Abstract
Introduction
Mobile health (mHealth) applications may improve timely access to health services and improve patient-provider communication, but the upfront costs of implementation may be prohibitive, especially in resource-limited settings.
Methods
We measured the costs of developing and implementing an mHealth-facilitated, home-based strategy for tuberculosis (TB) contact investigation in Kampala, Uganda, between February 2014 and July 2017. We compared routine implementation involving community health workers (CHWs) screening and referring household contacts to clinics for TB evaluation to home-based HIV testing and sputum collection and transport with test results delivered by automated short messaging services (SMS). We carried out key informant interviews with CHWs and asked them to complete time-and-motion surveys. We estimated program costs from the perspective of the Ugandan health system, using top-down and bottom-up (components-based) approaches. We estimated total costs per contact investigated and per TB-positive contact identified in 2018 US dollars, one and five years after program implementation.
Results
The total top-down cost was $472,327, including $358,504 (76%) for program development and $108,584 (24%) for program implementation. This corresponded to $320-$348 per household contact investigated and $8,873-$9,652 per contact diagnosed with active TB over a 5-year period. CHW time was spent primarily evaluating household contacts who returned to the clinic for evaluation (median 30 minutes per contact investigated, interquartile range [IQR]: 30–70), collecting sputum samples (median 29 minutes, IQR: 25–30) and offering HIV testing services (median 28 minutes, IQR: 17–43). Cost estimates were sensitive to infrastructural capacity needs, program reach, and the epidemiological yield of contact investigation.
Conclusion
Over 75% of all costs of the mHealth-facilitated TB contact investigation strategy were dedicated to establishing mHealth infrastructure and capacity. Implementing the mHealth strategy at scale and maintaining it over a longer time horizon could help decrease development costs as a proportion of total costs.
Citation: Turimumahoro P, Tucker A, Gupta AJ, Tampi RP, Babirye D, Ochom E, et al. (2022) A cost analysis of implementing mobile health facilitated tuberculosis contact investigation in a low-income setting. PLoS ONE 17(4): e0265033. https://doi.org/10.1371/journal.pone.0265033
Editor: Limakatso Lebina, University of the Witwatersrand, SOUTH AFRICA
Received: January 13, 2021; Accepted: February 22, 2022; Published: April 1, 2022
Copyright: © 2022 Turimumahoro et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data are held in the Dryad online repository, available at https://datadryad.org/stash/share/02ECnaLd-vQstsxpf3_L8tWynB6DG8ILjKcAHqqdK3g.
Funding: This study was supported by National Institutes of Health through grants awarded to JLD (R01AI104824 and D43TW009607). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Tuberculosis (TB) is among the leading causes of death due to an infectious disease worldwide, with approximately 7 million new TB cases diagnosed in 2020 [1]. Low TB case detection rates represent a major gap in the TB care cascade in high burden countries, and more than 30% of estimated incident TB cases continue to go undiagnosed and/or unreported [1–3]. This problem has impeded progress toward the global End TB targets [4]. Patient-centered interventions that can facilitate early case detection and reduce barriers to TB care are thus an important public health and global health priority.
Contacts of TB patients have a substantial risk of developing active TB within the first one to two years after exposure [5]. and in high-income settings, interventions focusing on contacts of index TB patients (e.g., contact investigation) have become an important priority for TB control and elimination [6]. TB contact investigation involves teams of health care workers visiting households or workplaces of people diagnosed with active TB to identify and refer those with TB-specific symptoms or key risk factors for further clinical and bacteriologic TB evaluation. Recent evidence suggests that contact investigation has the potential to improve TB case detection in high prevalence settings relative to passive case-detection services in health facilities [7]. However, barriers to acceptance and completion, operational complexities, and resource constraints have limited wide adoption of contact investigation in low-and middle-income countries[2,8,9].
To address these challenges, we developed a home-based, mHealth-facilitated household contact investigation strategy and evaluated it in a pragmatic, prospective, household randomized trial [10]. Compared to routine contact investigation delivered by community health workers (CHWs), the mHealth-facilitated contact investigation intervention included home-based HIV testing and TB evaluation, collection and transport of sputum samples, and follow-up communications using automated short messaging services (SMS). The strategy was feasible and acceptable but not more effective than routine contact investigation because of implementation challenges [11,12]. Nonetheless, another area of uncertainty in the mHealth field is the limited and heterogeneous evidence on the costs and cost effectiveness of mHealth strategies [13] including some evidence of high up-front costs [14], which may in turn act as a barrier to ongoing research and innovation. Therefore, to characterize the resource implications of mobile health interventions more fully, we conducted a comprehensive assessment of the costs of development, implementation, and maintenance of home-based, mHealth-facilitated TB contact investigation in Kampala, Uganda.
Materials and methods
Study design and setting
We estimated the costs of developing, implementing, and maintaining a home-based, mHealth-facilitated household TB contact investigation intervention in Kampala, Uganda, from the health system perspective, using both a “top-down” and “bottom-up” (components-based) approach. In this setting, TB contact investigation involved CHWs visiting the homes of TB patients, screening all contacts for TB symptoms, and recording their findings using a customized electronic survey application (CommCare, Dimagi, Boston, USA). The application employed decision-support logic to identify contacts requiring evaluation for TB and prompted CHWs to collect a sputum sample and offer HIV testing to eligible household members. The application also delivered personalized, automated text messages to each participant providing follow-up instructions, clinic visit reminders, and TB test results. In the routine care arm, automated text messages were not sent, and all contacts needing TB evaluation were referred to the clinic. The home-based strategy sought to increase the proportion of contacts fully evaluated for TB by reducing the need for contacts to travel to clinics.
To comprehensively evaluate the costs of the mHealth-facilitated intervention, we divided the program into two phases and evaluated the costs accrued in each phase. The development phase, which lasted 30 months (February 2014-July 2016), consisted of formative research, software customization, and pilot testing. Activities during this phase included, but were not limited to, development of decision-support logic, integration of fingerprint identification technology and automated short messaging service (SMS) technology; pilot testing; and optimization of technological components. This was followed by the implementation phase, which occurred over 12 months (July 2016 –July 2017) in the context of a cluster-randomized trial. A total of 919 contacts were randomized at the household level, including 471 contacts in the intervention arm who are the focus of this cost analysis. The trial observed a marginal probability of completing TB evaluation of 14% (95% CI 8–20) in intervention households and 15% (95% CI 9–21) in routine care households, representing a difference of -1% (95% CI -9% to 7%, p = 0.81) [10]. Fig 1 shows a conceptual outline of project phases, activities, and timelines.
Estimation of program development costs
To estimate the cost of developing, implementing, and maintaining the mHealth-facilitated contact investigation intervention, we retrospectively collected programmatic costs of each phase of development from the health system perspective using health facility and study budget estimates. All cost and volume estimates extracted from study and health center financial records, and key informant interviews were performed with study staff and health facility administrators to confirm which budgetary items mapped to specific expenditures for program development. Cost components were appropriately mapped to specific thematic expenditure categories: human resource costs, capital costs, recurrent costs, overhead costs, and building space costs. Human resource costs included salaries of a coordinator, data manager, laboratory manager, IT officer and CHWs. Capital costs included investment in hardware and software for mHealth, a vehicle, and cost to adapt the intervention to the local setting. Recurrent costs included expenditure on consumables such as laboratory supplies, internet, and text messages. Overhead costs included operational costs such as those for supervision teams and patient care at the clinic. Building space included the space occupied by the supervision teams and patient rooms. Since budget allocations were similar across facilities with similar clinical capacity, patient visit volumes, and scope of service delivery, we compiled cost data from four of the seven participating health centers and extrapolated these costs for the remaining three health facilities. Full details on cost components, according to study phase, are summarized in the Supporting Information, S1 Table.
Estimation of program implementation costs
We assessed the cost of program implementation using both a top-down and bottom-up approach. Top-down costing of program implementation was performed similarly to the top-down estimation of program development costs described above. Bottom-up costing was performed using an activity-based approach. Specifically, we conducted a time-and-motion (TAM) study on consecutive days between March and August 2017 asking CHWs to record start and end times for each discrete contact investigation activity, as defined in the Supporting Information, S2 Table. Clinic-based activities included: TB index patient recruitment, waiting time, contact evaluation, and other activities. Community-based activities included: travel to and from the household, TB education and counseling, household contact screening for TB symptoms, HIV testing, sputum collection, HIV testing, and confirmation of phone contacts for SMS messaging. Using the top-down approach, we summed unit costs in five primary categories: human resources, capital investments, building space, overhead costs, and recurrent costs. Human resource costs were estimated by enumerating staffing levels for the entire development phase and multiplying monthly staff salaries by the percent effort contributed to program development (50%), based on opinions from key informants. Capital investments included software, hardware, and formative work. The cost of building space utilized for patient services was approximated as 5% of the cost of the entire building and operational costs for the program as 6–7% of clinic operational costs. The cost estimate for one square meter of building space was based on local rates suggested by clinic administrators and multiplied by the measured area of the building. Recurrent costs accrued during the development of the program were summarized as software, lab consumables, and program evaluation costs for routine quality assurance. The total cost of program development was then calculated as the total of all five categories over the entirety of the development phase. The cost of program implementation was calculated in a similar fashion, and total program costs were estimated as the sum of program development plus program implementation costs.
Total program costs: Bottom-up
For bottom-up costing, program development costs were assessed as annual costs based on corresponding estimates of useful life years for each component (between 5 and 30 years, based on key informant interviews). These were converted to an estimated cost per minute-use, based on the estimated total number of operational minutes per year: 8 hours per day, 5 days per week, 46 weeks per year). The bottom-up costs of program implementation were estimated by multiplying the median time estimates to perform each activity (estimated using TAM studies as above) by the cost per minute for each resource type. Resource use was categorized as 1) direct human resources; 2) capital equipment; 3) program overhead (operational costs and recurrent costs); and 4) building space costs. We summed the unit costs for each category to calculate the total activity-based cost per household contact investigated. Some activities (e.g., TB index patient recruitment) were associated with a specific index case, not a specific household contact. For these activities, we divided these unit time estimates by the average number of household contacts observed per index patient to get the time estimate for each activity per household contact investigated.
Analysis
We assumed five useful life years for all program development costs except for building space and vehicles, for which we assumed an expected useful life of 30 years. All capital costs were depreciated linearly using a 3% discount rate. We divided the total program costs by the total number of patients enrolled and the total number of new TB diagnoses made to estimate the cost per household contact investigated and the cost per TB diagnosis made. All costs are reported in 2018 US dollars. All costs measured in Ugandan Shillings were updated to 2018 using the Ugandan GDP deflator [15] and converted to US dollars at the average annual exchange rate for 2018 [16].
Sensitivity analysis
This program required a large initial investment in technology development, infrastructure, and equipment with a low marginal cost for including additional patients during program implementation. As such, we performed a sensitivity analysis to estimate costs under “continued implementation”, under the assumption that the one-year intervention could be continued for four additional years at the same volume without any additional development costs (i.e., continued implementation costs only). To perform this sensitivity analysis, we used top-down cost estimates and allocated costs incurred at the program level, clinic level, and contact level. We then multiplied costs at the clinic level by the number of clinics included in the program and costs at the contact level by the number of contacts screened per clinic. To explore differences in clinic capacity over different years, we performed a three-way sensitivity analysis in which we simultaneously varied the numbers of clinics that might be covered by the mHealth program (in increments of 10, from 10 to 50), the mean annual contacts evaluated per clinic (in increments of 50, from 100 to 350), and the average annual contact positivity rate (at four levels, from 0.025 to 0.042). The maximum number of clinics that could be covered was based on the expert opinion of research staff and implementers, and the annual number of contacts who would be screened per clinic was estimated using the observed number of patients per facility (plus or minus 2 standard deviations from the mean as minimum and maximum values).
Results
In the 12 months of program implementation, 190 index TB patients with 471 household contacts were randomized to receive the intervention. The TB case notification rate at the seven contributing health centers ranged from 15 to 67 TB cases per month. Of the 471 contacts, 106 (23%) had TB symptoms, ranging from 14% to 33% across health centers (Table 1).
Using a top-down approach, the total cost of the mHealth TB contact investigation intervention was estimated at $472,327, of which program development accounted for $358,504 (76%) and program implementation for $113,823 (24%). Human resource costs accounted for $178,542 (38%), capital assets for $156,091 (33%), and recurrent costs for $74,965 (16%), overhead costs for $51,202, and building space for $11,525 (2%) (Table 2A).
The mHealth TB contact investigation intervention was estimated to cost $1,003 per household contact investigated or $27,748 per positive contact found. Assuming that the intervention was continued for an additional four years at similar capacity, the program was estimated to cost $348 per household contact investigated, or $9,652 per positive contact diagnosed (Table 2B).
a. Granular unit cost estimates for each phase of implementation of an mHealth-facilitated TB contact investigation program. b. Top-down cost estimates for household contact investigation of tuberculosis in Uganda.
In the time and motion survey, we observed a total of 12,100 person-minutes from 11 discrete activities across seven trial clinics. A total of 5,496 (45%) person-minutes of clinic-based activities and 6,604 (55%) person-minutes of community-based activities were observed. In the clinic, CHWs spent approximately 1.2 hours per household contact investigated, with the most time spent evaluating contacts returning to the clinic (median 30 person-minutes per contact evaluated enrolled, interquartile range [IQR]: 30–70). In the community, CHWs spent approximately 3.5 hours per contact investigated, with sputum collection (median 29 person-minutes per contact investigated, IQR 25–30) and offering HIV testing services (median 28 minutes per contacts investigated, IQR: 17–43) found to be the most time-consuming activities (Supporting Information, S3 Table). After enumerating all costs and activity times, our bottom-up cost estimate of program costs was $320 per household contact investigated or $8,873 per TB positive contact diagnosed, with component cost estimates as summarized in Table 3.
Program capacity (both the number of participating facilities and the number of household contacts investigated per facility) had a large effect on the estimated cost per positive contact diagnosed (Fig 2). At the observed average annual contact positivity rate (0.036), over half of the capacity scenarios projected a cost per positive contact diagnosed of less than $600 –substantially lower than the estimates of $8,873 (bottom-up) and $9,652 (top-down) at the observed capacity of 7 clinics and 67 contacts screened per facility. Under the highest capacity projected (50 facilities with an average of 300 contacts investigated per facility per year), and the average annual contact positivity rate (0.036), we estimated that this program could be developed and implemented at a total cost of $459 per contact diagnosed with TB. Higher TB prevalence among contacts resulted in even lower costs per person diagnosed.
Each graph represents the cost of an mHealth-facilitated TB contact investigation program as the average annual contact positivity rate (number of contacts diagnosed with TB divided by the number of contacts evaluated) varies from 0.0252 to 0.0307 to 0.0361 to 0.0415 among household contacts of TB patients. The y-axis represents the number of clinics covered by the program and the x-axis the total annual contacts at each clinic. The orange hue gradient represents the cost gradient associated with different program coverage capacities, with the darkest shade representing scenarios with the lowest programmatic costs and the lightest shade representing scenarios with the highest programmatic costs. Abbreviations: CHW, Community health worker; RCT, randomized-controlled trial; SMS, short-messaging service; TB, tuberculosis.
Discussion
In resource-limited settings, mHealth technologies are being widely implemented, not only for TB contact investigation [17–20], but also for a wide array of other health-related interventions. More broadly, eHealth is being rapidly adopted worldwide and has been endorsed by the WHO [18]. In this economic evaluation we provide context on the adaptation of mHealth for a home-based TB contact investigation intervention in a low-income setting. Specifically, we found that 76% of the total cost of the program was incurred during development, before the recruitment of a single participant. The total program cost was therefore estimated at $320-$348 per contact investigated and $8,873-$9,652 per new diagnosis of TB–a high cost relative to many other programs [13,21,22]. Importantly, these costs could be reduced to under $600 per new TB diagnosis simply by expanding capacity (extending from one to five years, increasing the number of clinics participating, and optimizing the volume of household contacts evaluated in each clinic). These findings illustrate that, when implementing mHealth and other interventions with substantial development costs in resource-limited settings, the feasible scope and duration of the program, as well as the expected yield, must be considered to evaluate whether a meaningful return on investment is likely.
Assuming that the maximum capacity of participants could be achieved, our projected costs of mHealth-facilitated contact investigation are comparable to the estimated costs of contact investigation (without mHealth) in other low-income settings, with one economic evaluation of contact investigation in Uganda estimating a cost of US$878 per new TB case identified [23,24]. This finding suggests that mHealth implementation may be economically viable in this context if sufficient patient volumes can be achieved. The drivers of cost in the development phase of this project included capital investments in technology (42%) and human resources–including highly trained IT staff and implementers (36%). During the implementation phase, human resource costs for supervision became more prominent (46%), reflecting the relatively small number of participants engaged relative to high-level supervisory staff. Again, this finding highlights the need to achieve economies of scale to make such programs more cost-effective.
More broadly, the cost implications of digital health interventions in low- and middle-income countries vary depending on their scope and purpose. For example, a digital adherence program in Brazil applying two-way SMS cited a cost of $65 ($53–$105) per person enrolled [25], while an mHealth support intervention for HIV in Uganda cited an annual cost of $2.35 per patient enrolled [22]. A study in India on an mHealth-facilitated intervention to improve CHW counseling skills in maternal and newborn health suggested that start-up costs represented only 9% of total costs; this program was able to operate at a cost of $20 per woman registered [21]. Taken as a whole, these findings illustrate the heterogeneous economic implications of mHealth-facilitated health interventions designed for resource-limited settings, as well as the importance of considering what is feasible in terms of program scope and the costs of program development before embarking on large investments in mHealth infrastructure and capacity.
Our analysis adheres to transparency recommendations on scope and accuracy, and captures unit cost estimates of all the key program components [26]. While we do not report granular expenditures (e.g., number of sputum cups purchased), all macro cost calculations were derived from itemized unit costs and quantities. Furthermore, our method of micro-cost estimation also provides the recommended highest level of accuracy, estimating the cost of each discrete implementation activity based on actual consumption. The observed breakdown of costs for program development versus implementation may serve as a reference for future cost-effectiveness analyses [27], and help potential implementers concisely project the budgetary implications of such a program over time. Our cost estimates using a top-down and bottom-up approach were reasonably similar, suggesting that our estimates are robust to the precise method of costing employed [28]. The inputs for this analysis were derived from real-world implementation; while the trial did not find a significant improvement in TB case detection [10], it identified important process-related challenges that, if addressed, could improve program adoption, implementation, and maintenance [2,11,29]. A follow-up trial using human-centered design and communities of practice to overcome these barriers is currently underway [30,31], and the cost implications of the revised design and implementation processes will also be important to consider.
This study had several limitations. First, volume-based costs were collected retrospectively from budget estimates and financial records. This may have led to underestimation of the absolute unit cost estimates of mHealth TB contact investigation due to missing records or poor recall by implementing staff. This retrospective data collection may also have compromised the delineation between programmatic and research-related costs. Second, clinic estimates of overhead costs were not readily available because of variations and inconsistency of funding at the clinic, with many of the inputs based on verbal estimates by clinic administrators. These limitations could have led to overestimation or underestimation of the cost of overheads due to recall bias. Third, we conducted a self-reported time-and-motion study by CHWs; these results are therefore subject to potential social desirability bias. Self-reported time-and-motion forms were collected daily, and data quality checks were done immediately to minimize this bias, but future evaluations could attempt to use more automated approaches to track activity times. Fourth, as a focused estimate of the cost of contact investigation, broader costs (such as patients’ cost of accessing TB diagnostic and treatment services if found to be positive for TB or recommended for TB preventive therapy) that are of relevance to society were not included. Finally, scenarios for sensitivity analysis were constructed based on expert opinion and may not fully represent the potential scope of the program if scaled up more broadly.
Conclusions
In summary, this investigation of the costs of mHealth-facilitated contact investigation for TB revealed high up-front costs for design and development of the mHealth infrastructure and capacity; these costs amounted to three-fourths of total program costs after one year of program implementation. In deciding whether to invest in similar interventions in Uganda and other resource-limited settings, careful consideration must be made as to the feasible duration and scope of program implementation to ensure a favorable return on investment.
Supporting information
S1 Table. Detailed list of cost components considered for the mHealth-facilitated contact investigation program.
https://doi.org/10.1371/journal.pone.0265033.s001
(DOCX)
S2 Table. Community health worker activities identified during time-and-motion surveys.
https://doi.org/10.1371/journal.pone.0265033.s002
(DOCX)
S3 Table. Distribution of community health worker activity time by health center per household TB contact investigated.
https://doi.org/10.1371/journal.pone.0265033.s003
(DOCX)
Acknowledgments
The authors thank all of the community health workers on the mHealth Contact Investigation Study; AIDS Information Centre staff; all administrative, TB unit, and laboratory staff at the participating Kampala City Council Authority clinics; staff at the Department of Health at the Kampala City Council Authority; National TB and Leprosy Program staff at the Uganda Ministry of Health, and all participants of the mHealth Contact Investigation Study.
References
- 1.
World Health Organization. COVID-19 and Tuberculosis. Global Tuberculosis Report 2021. Geneva Switzerland, 2021.
- 2. Armstrong-Hough M, Turimumahoro P, Meyer AJ, Ochom E, Babirye D, Ayakaka I, et al. Drop-out from the tuberculosis contact investigation cascade in a routine public health setting in urban Uganda: A prospective, multi-center study. PLoS One. 2017;12(11).
- 3. Subbaraman R, Nathavitharana RR, Mayer KH, Satyanarayana S, Chadha VK, Arinaminpathy N, et al. Constructing care cascades for active tuberculosis: A strategy for program monitoring and identifying gaps in quality of care. PLoS Med. 2019. pmid:30811385
- 4.
Ghebreyesus T. Global Tuberculosis Report 2019. World Health Organization. 2019.
- 5. Velen K, Shingde RV, Ho J, Fox GJ. The effectiveness of contact investigation among contacts of tuberculosis patients: a systematic review and meta-analysis. Eur Respir J. 2021. pmid:34016621
- 6.
World Health Organization. WHO consolidated guidelines on tuberculosis. WHO Press. 2020.
- 7. Fox GJ, Nhung N V., Sy DN, Hoa NLP, Anh LTN, Anh NT, et al. Household-Contact Investigation for Detection of Tuberculosis in Vietnam. N Engl J Med. 2018.
- 8. Ayakaka I, Ackerman S, Ggita JM, Kajubi P, Dowdy D, Haberer JE, et al. Identifying barriers to and facilitators of tuberculosis contact investigation in Kampala, Uganda: A behavioral approach. Implement Sci. 2017. pmid:28274245
- 9. Ngamvithayapong-Yanai J, Luangjina S, Thawthong S, Bupachat S, Imsangaun W. Stigma against tuberculosis may hinder non-household contact investigation: a qualitative study in Thailand. Public Health Action. 2019. pmid:30963038
- 10. Davis JL, Turimumahoro P, Meyer AJ, Ayakaka I, Ochom E, Ggita J, et al. Home-based tuberculosis contact investigation in Uganda: a household randomised trial. ERJ Open Res. 2019. pmid:31367636
- 11. Meyer AJ, Babirye D, Armstrong-Hough M, Mark D, Ayakaka I, Katamba A, et al. Text messages sent to household tuberculosis contacts in Kampala, Uganda: Process evaluation. JMIR mHealth uHealth. 2018. pmid:30459147
- 12. Ggita JM, Katahoire A, Meyer AJ, Nansubuga E, Nalugwa T, Turimumahoro P, et al. Experiences and intentions of Ugandan household tuberculosis contacts receiving test results via text message: An exploratory study. BMC Public Health. 2020;20(1).
- 13. Iribarren SJ, Cato K, Falzon L, Stone PW. What is the economic evidence for mHealth? A systematic review of economic evaluations of mHealth solutions. PLoS One. 2017. pmid:28152012
- 14. Larsen-Cooper E, Bancroft E, Rajagopal S, O’Toole M, Levin A. Scale matters: A cost-outcome analysis of an m-health intervention in Malawi. Telemed e-Health. 2016. pmid:26348994
- 15.
The World Bank. GDP deflator (base year varies by country). World Development Indicators. 2015.
- 16.
World Bank. GDP (current US$). World Development Indicators. 2014.
- 17. Sacks JA, Zehe E, Redick C, Bah A, Cowger K, Camara M, et al. Introduction of mobile health tools to support Ebola surveillance and contact tracing in Guinea. Glob Heal Sci Pract. 2015. pmid:26681710
- 18.
World Health Organization. mHealth: New horizons for health through mobile technologies. Observatory. 2011.
- 19. Braun R, Catalani C, Wimbush J, Israelski D. Community Health Workers and Mobile Technology: A Systematic Review of the Literature. PLoS One. 2013. pmid:23776544
- 20. Yasaka TM, Lehrich BM, Sahyouni R. Peer-to-peer contact tracing: Development of a privacy-preserving smartphone app. JMIR mHealth uHealth. 2020. pmid:32240973
- 21. Prinja S, Bahuguna P, Gupta A, Nimesh R, Gupta M, Thakur JS. Cost effectiveness of mHealth intervention by community health workers for reducing maternal and newborn mortality in rural Uttar Pradesh, India. Cost Eff Resour Alloc. 2018. pmid:29983645
- 22. Chang LW, Kagaayi J, Nakigozi G, Serwada D, Quinn TC, Gray RH, et al. Cost analyses of peer health worker and mHealth support interventions for improving AIDS care in Rakai, Uganda. AIDS Care—Psychol Socio-Medical Asp AIDS/HIV. 2013. pmid:22971113
- 23. Kakinda M, Matovu JKB. A yield and cost comparison of tuberculosis contact investigation and intensified case finding in Uganda. PLoS One. 2020. pmid:32511264
- 24. Jo Y, Mirzoeva F, Chry M, Qin ZZ, Codlin A, Bobokhojaev O, et al. Standardized framework for evaluating costs of active case-finding programs: An analysis of two programs in Cambodia and Tajikistan. PLoS One. 2020. pmid:31986183
- 25. Nsengiyumva NP, Mappin-Kasirer B, Oxlade O, Bastos M, Trajman A, Falzon D, et al. Evaluating the potential costs and impact of digital health technologies for tuberculosis treatment support. Eur Respir J. 2018. pmid:30166325
- 26. Fukuda H, Imanaka Y. Assessment of transparency of cost estimates in economic evaluations of patient safety programmes. J Eval Clin Pract. 2009;15(3):451–9. pmid:19366392
- 27. Sohn H, Tucker A, Ferguson O, Gomes I, Dowdy D. Costing the implementation of public health interventions in resource-limited settings: A conceptual framework. Implementation Science. 2020. pmid:32993713
- 28. Chapko MK, Liu CF, Perkins M, Li YF, Fortney JC, Maciejewski ML. Equivalence of two healthcare costing methods: Bottom-up and top-down. Health Econ. 2009. pmid:19097041
- 29. Armstrong-Hough M, Ggita J, Turimumahoro P, Meyer AJ, Ochom E, Dowdy D, et al. “Something so hard”: A mixed-methods study of home sputum collection for tuberculosis contact investigation in Uganda. Int J Tuberc Lung Dis. 2018. pmid:30236182
- 30.
IDEO. IDEO Design Thinking. IDEO. 2020.
- 31. Catalani C, Green E, Owiti P, Keny A, Diero L, Yeung A, et al. A clinical decision support system for integrating tuberculosis and HIV care in Kenya: A human-centered design approach. PLoS One. 2014. pmid:25170939