Abstract
Background: Although the field of digital health is rapidly growing, there is scanty information on the impact of these interventions on the overall performance of health projects.
Objectives: We assessed the influence of utilisation of four types of digital health interventions (DHIs) and application of monitoring and evaluation (M&E) practices on performance of human immunodeficiency virus (HIV) projects.
Method: This was a cross-sectional survey across eight public health facilities providing care to HIV patients and where all the four types of DHIs were being implemented in Kisumu County, Kenya. A total of 191 service providers who were at their stations of work on the day of data collection were recruited into the study. Aspects of utilisation of the DHIs, application of M&E practices and performance of the HIV projects were measured using standardised 12 statements on a 5-point Likert scale.
Results: Using a multi-linear regression model, we established that the four DHIs could potentially explain 22% (R2 = 0.22; p-value < 0.001 at 95% confidence interval) of variation in performance of HIV projects. Application of best M&E practices could further explain the variation of the relationship between utilisation of DHIs and performance of HIV or AIDS projects up to 33.2% (R2 = 0.332; p-value < 0.001 at 95% confidence interval).
Conclusion: Optimal utilisation of DHIs improves performance of HIV projects.
Contribution: This study provides evidence on the importance of utilising digital health in managing health projects. Further, it augments the central role of monitoring and evaluation in project performance.
Keywords: digital health; monitoring; evaluation; performance; project management.
Introduction
The world is quickly moving towards digitalisation of processes and activities in many sectors. Countries in Africa including Kenya, are at the forefront of innovating and adopting various digital solutions to make work more efficient and effective. In the healthcare sector, the World Health Organization (WHO) provides guidelines on utilisation of digital health interventions (DHIs) by classifying DHIs into four classes, namely DHIs for clients, DHIs for healthcare providers, DHIs for health system managers and DHIs for data services (WHO 2018). Digital health interventions have shown great potential to influence health outcomes such as adherence to treatment and retention of care (Sherman et al. 2020). Studies have demonstrated that DHIs can increase knowledge and awareness among community members thus influencing how they adhere to appointment and treatment plans (Chandrasekhar & Ghosh 2001). In addition, there is evidence that DHIs have the potential to influence how service providers adhere to guidelines in service provision (Kuriyan et al. 2014). Furthermore, DHIs have shown the capability to increase efficiencies in service provision by cutting down on costs and time needed to access services among clients (Mehran et al. 2012).
There are, however, some challenges associated with the utilisation of DHIs in many low-to-middle-income countries such as Kenya. For instance, the high initial cost of purchasing devices and installing systems, a lack of knowledge, interest or ability to adopt utilisation of DHIs among targeted users, poor infrastructure, poor connectivity to the internet and power and organisational culture grounded in manual processes (Leon, Scheneider & Daviaud 2012; Minichiello et al. 2013; Vesel et al. 2015). Most DHIs are not followed up with robust monitoring and evaluation (M&E). Monitoring and evaluation of DHIs is critical in order to ascertain the potential for success or failure in good time. Results from M&E can inform new strategies aimed at achieving desired objectives. These challenges hinder full-blown implementation of DHIs, thus most digital health projects end at the pilot or trial phases.
There have been many efforts in financing and implementing new and innovative strategies towards ending the AIDS epidemic in Kenya. In the recent past, focus has been put on the use of DHIs within and beyond the HIV programme in Kenya. There have also been great efforts to ensure that the legal and institutional framework is supportive enough for the implementation of DHIs in Kenya. In terms of policy and legal framework, the government of Kenya has developed an eHealth policy (Ministry of Health Kenya 2016) and guidelines for implementation of mHealth interventions (Ministry of Health Republic of Kenya 2017). The national government has also developed a Digital Health Bill (2023), which is aimed at digitalisation of all health services through what has been dubbed as ‘digital health highway’.
However, little is known about the relationship between utilisation of DHIs and overall performance of health projects in Kenya. We mapped four types of DHIs being utilised within the HIV projects in public health facilities in Kisumu county, Kenya. Kisumu county is one of the with the highest prevalence of AIDS in Kenya. It was estimated that the county had an AIDS prevalence of 17.5% against a national prevalence of 4.9% among adults (Ministry of Health and NASCOP 2020). The DHIs identified were classified into (1) digital health interventions for appointment adherence, (2) digital health interventions for transmission of laboratory results, (3) digital health interventions for data management and (4) digital health interventions for antiretroviral drug management. This study aimed at assessing the association between utilising the four types of DHIs and performance of HIV projects based on perceptions of service providers. Furthermore, we assessed the moderating influence of applying M&E practices on the association between utilisation of DHIs and performance of HIV projects.
Utilisation of digital health interventions
The term digital health is the broader term used to denote the use of technological innovations to improve healthcare outcomes. Digital health incorporates eHealth, which is defined as ‘the use of information and communications technology in support of health and health-related fields’ and mobile health (mHealth) which is a subset of eHealth and is defined as ‘the use of mobile wireless technologies for health’. There are several components of the digital health ecosystem including big data and artificial intelligence, considered to be emerging areas, which are aimed at addressing more complex challenges (Fatehi, Samadbeik & Kazemi 2020; WHO 2019).
Digital health interventions have the ability to ameliorate challenges in health at all levels. At community level, DHIs have been used to improve health-related knowledge and information thus improving access to and utilisation of health services. In addition, interventions have been developed to enable community health promoters or volunteers to collect data digitally. At the facility level, DHIs have been used for data management, building capacity among service providers and ensuring that guidelines are followed. Moreover, DHIs such as telemedicine reduce the time and costs of physical visits to health facilities (Haleem et al. 2021).
In this study, utilisation of DHIs was conceptualised as the use of four types of digital solutions being used to enhance services in AIDS comprehensive care centres in eight selected public health facilities in Kisumu County, Kenya. The four DHIs were DHI for management of appointments, DHIs for healthcare workers to receive laboratory results, DHIs for management of antiretroviral drugs and DHI for data management (electronic medical records). The DHI for the management of appointments was both a web-based and mobile-based electronic diary. The solution was being used to send scheduling of reminders, sending of appointment messages and tracking of defaulters. The DHI for healthcare workers to receive laboratory results was aimed at reducing the turnaround time from the release of early infant diagnosis and viral load results from reference testing laboratories to health facilities. This was meant to ensure that clinical decisions are made in a timely manner in terms of linking patients to the appropriate treatment. The DHI for antiretroviral drug management was being used to dispense and account for the AIDS-related drugs at the pharmacies. Lastly, the DHI for data management was aimed at digitalising the management of records at the comprehensive care centres for easy record keeping and reporting.
Performance of HIV projects
The definition of performance measurement that we adopted for this study was evaluating how well digital health projects were being managed in the provision of HIV care and treatment services to clients. Performance of a project is gauged on how well the resources including time and budget are utilised to achieve quality results within the scope. Project performance measurement may include measures based on various aspects such as financial, customer satisfaction, project or process measures and growth measures. In many cases, performance may include the goods and services that are generated during project implementation. There are two key factors of any project performance, namely effectiveness and efficiency (Moullin 2007). While effectiveness refers to the attainment of key objectives, efficiency goes further to assess the inputs put into the project against the outputs and outcomes (Molaei, Bosch-Rekveldt & Bakker 2019).
Based on the Joint United Nations Programme on HIV/AIDS (UNAIDS) (2019) programme targets, performance of HIV projects refers to how well a project is able to achieve the three key targets, namely (1) 95% of people who are living with HIV to be aware of their status, (2) 95 of those who are aware of their positive status to be linked to care and (3) 95 of those linked to care be virally suppressed by year 2030. These targets are aimed at ending the AIDS epidemic by 2030 (UNAIDS 2019). Given the comprehensive nature of this study in which we examined perceptions on utilisation of four different types of DHIs, we broadened the concept of HIV project performance to also capture perceptions on both programme-level outcomes such as reporting, project expenditure and patient outcomes such as adherence to care and linkage to care. Mostly, previous studies only focused on one aspect of a particular DHI and one outcome for example use of phone calls to improve adherence to appointments.
Monitoring and evaluation of digital health interventions
Monitoring of projects is critical in progressively assessing the performance of a given project, identifying gaps, challenges, best interventions, achievements and milestones (WHO 2016a). Evaluation, on the other hand, is the periodic assessment either at baseline, mid-line or end-line to assess project performance or status at a given point. Applying the best M&E practices can enhance the performance of a project by identifying high-impact strategies that can be sustained and, on the other hand, identifying low-impact strategies that require re-strategising (WHO 2016a).
With the increasing use of DHIs to enhance performance of health programmes, including HIV projects, there is need to monitor and evaluate the impact (Greenhalgh & Rusell 2010). The WHO has developed guidelines on monitoring and evaluating DHIs in which there is emphasis to apply M&E during the entire life cycle of DHIs (WHO 2016b). The guidelines call for concurrent M&E activities to be planned, often in parallel and supporting each other. Moreover, there is a need to ensure that as the intervention matures, the M&E needs should evolve such that there is transition from monitoring the system’s technical functionality and stability, towards continuous, real-time monitoring of its consistency in producing the expected outputs at a predefined level of quality. The evaluation of the DHI over time should move from exploring how users interact with the technology to determining how outcomes link to the technology (WHO 2016b). In this study, application of M&E practices involved assessing perceptions among participants on planning for M&E, availability of resources, data collection, data analysis, dissemination and use of findings during implementation of DHIs.
Theoretical framework
This study was pegged on two theories namely Diffusion of Innovation Theory and Unified Theory of Acceptance and Use of Technology (UTAUT). The Diffusion of Innovation Theory as postulated by Rogers (1995) provides a robust theoretical framework for researching on acceptance and adoption of technological innovations. Critical tenets of the theory are concerned with how innovations are communicated by the designers or implementers. As the innovation is communicated, it diffuses, and the targeted users are expected to accept and adopt it. However, the theory holds that before adoption, the process of diffusing an innovation should create understanding among targeted users; the targeted users have to be persuaded that the innovation will be useful; the innovation has to be implemented appropriately and there has to be confirmation from both the implementers and users that indeed the innovation is working. Based on the innovation characteristics and how the process of diffusion has been undertaken, the theory suggests that there are five categories of users on the adoption curve namely late majority, innovators, early majority, early adopters and laggards. Along the adoption curve, innovators pick the innovation instantly while the laggards weigh options and see how the innovation is working for other people before adopting it.
The diffusion of innovation theory was critical in understanding the varying levels of utilisation of the four different types of DHIs in this study. It is critical to note, while interpreting the results that the composite mean scores and standard deviations vary across the four DHIs in this study. As such, the theory is useful in explaining why users agree at a high level on utilisation of certain DHIs compared to others. Secondly, the diffusion of innovation theory was critical in associating the utilisation of the specific DHIs with the performance of the HIV projects. The differences in statistical significance between utilisation of various DHIs and performance of HIV can be explained based on the varying levels of importance users place on specific DHIs. The theory states that before users adopt an innovation, they have to be convinced that the innovation is useful. How users perceived the usefulness of the specific DHIs was critical in understanding the varying levels of utilisation.
The second theory used in this study was developed by Venkatesh and others in 2003. The UTAUT is based on four key tenets of expected performance, expected effort, influence within the social networks and conditions that facilitate acceptance and use of technology. The theory aims to explain the intentions among users to accept technological innovations and subsequently the actual use and related behaviour. According to the theory, several factors influence the users’ intentions to use a technological innovation. Key factors include perceived technological effectiveness and the required effort to be invested in the social environmental surroundings of an individual. Similar to Ajzen’s and Andersen’s theories, Venkatesh hypothesised that technology adoption would most likely be determined by user intentions to employ a technology and the availability of facilitating or enabling factors (such as the price value of the new technology).
The first key tenet of the theory, expected performance, clearly underscores the purpose of this study. The tenet of expected performance provides grounding for the interpretation of the results on the association between utilisation of the different DHIs and performance of HIV projects. UTAUT is also critical for consideration in explaining factors that influence utilisation of DHIs. The theory provides the basis for understanding the moderating influence of M&E practices.
Conceptual framework
This study examined the relationship between three main variables, namely utilisation of four types of DHIs (independent variable), M&E practices (moderating variable) and performance of HIV projects (dependent variable). Figure 1 presents a summary of the conceptual framework of the study illustrating the relationship between the variables of the study. We conceptualised utilisation of DHIs based on four categories that were broadly informed by the WHO’s classification and discussed in the introduction section of this article (WHO 2018). Performance of HIV was conceptualised based on indicators that could be assessed by the healthcare providers at the comprehensive care centres through perception (Mudogo et al. 2023). Indicators such as viral load of the patients were excluded at the design stage of the study because they would have required actual testing of the patients or review of records which were out of scope of the study.
Research methods and design
We conducted a cross-sectional descriptive survey across eight high-volume public health facilities in Kisumu County of Kenya providing care and treatment to people living with AIDS at the comprehensive care centres. The selected health facilities included six sub-county hospitals and two county-level hospitals. These were facilities where all the four types of DHIs were being implemented. The study targeted service providers at health facilities, facility in-charges and managers at sub-county level. Specific cadres included clinical staff (nurses and clinical officers), laboratory staff, pharmacy staff, records and information officers, data officers, M&E officers, facility in-charges and managers at sub-county levels. The≈study recruited a total of 191 health service providers who were at their stations of work during the 21-day data collection period in January 2022. All the participants needed to confirm that they were aware of or had used at least one or more of the specific DHIs being utilised within the HIV project at the health facilities for at least 1 year prior to the study. Data were collected using an online questionnaire based on standardised 12 statements per variable and built on a 5-point Likert Scale where: 1 = strongly disagree, 2 = Disagree, 3 = Neutral, 4 = Agree and 5 = Strongly agree. Results were interpreted at 95% confidence interval and specific descriptive results have been published elsewhere (Mudogo et al. 2023). We used the Pearson correlation coefficient and linear regression models to demonstrate the association between the study variables, namely utilisation of DHIs (independent variable); M&E practices (moderating variable and performance of HIV projects (dependent variable). Moksony in an article entitled ‘Small Is Beautiful. The Use And Interpretation Of R2 In Social Research’ (Moksony 1990) argues that the coefficient of determination (R-squared value) should be used for prediction purposes while the regression coefficient (β) value should be used when the aim is to test a theory. In addition, he argues that depending on the aim of a study a small or large R2-value may still be useful. In this article, we have presented both the coefficient of determination as R2 values and the regression coefficient as the standardised β values as key measures of association between the variables.
Ethical considerations
The study was approved by the Kenyatta Hospital/University of Nairobi Ethics and Research Committee, approval number P477/05/2021. A research permit was acquired from the Kenya National Commission of Science, Technology and Innovation, permit number NACOSTI/ P/21/14786. In addition, the researchers received approval from Kisumu County Department of Health to conduct the study. All the participants consented to be part of the study.
Results
Descriptive statistics
Table 1 presents the mean scores and standard deviations on levels of utilisation of the various types of DHIs, application of M&E practices and performance of HIV projects based on the Likert scale scores. Additional results of scores on specific variables have been published elsewhere in separate journal articles (Mudogo, Mulwa & Kyalo 2023; Mudogo et al. 2023).
| TABLE 1: Mean scores and standard deviations on study variables based on the Likert scale. |
The findings show that there was relatively high level of utilisation of three of the four DHIs, which had mean scores above the composite mean score of 4.11 for the four types of DHIs. These were DHIs for clients’ appointment adherence (mean score = 4.28, standard deviation [SD] = 0.77); DHI for ARV drug management (mean score = 4.42, SD = 0.67) and DHI for data management (mean score = 4.41, SD = 0.66). However, scores on utilisation of DHI for healthcare workers to receive laboratory results were lower with a mean score = 3.94 (SD = 0.82). Overall the four DHIs had a mean score of 4.11 (SD = 0.73) implying high level of use and adoption of DHIs within the HIV projects across the eight health facilities. Application of M&E practices had a mean score of 4.24, SD = 0.69, while performance of HIV projects had a mean score of 4.47, SD = 0.57.
Correlation between variables
The second level of analysis was to test the following null hypothesis to determine whether there existed a statistically significant association between the study variables:
- H0: There was no statistically significant association between utilisation of the four types of digital health interventions, monitoring and evaluation practices and performance of HIV projects.
A Pearson Correlation Coefficient was used to establish whether there was a statistically significant correlation between the variables. Results are presented in Table 2.
| TABLE 2: Correlation between combined utilisation of digital health interventions and performance of HIV projects Z-score: Performance of HIV projects (N = 191). |
The results of the Pearson correlation coefficient show that there was a statistically significant positive correlation between utilisation of the combined (four) DHIs and performance of HIV projects (r = 0.462; p< 0.001).
The study sought to establish individual correlations between specific DHIs and performance of HIV projects. Pearson correlation coefficients were used to assess the correlations between the various aspects of the independent variable (utilisation of different DHIs) and the dependent variable (performance of HIV projects). Results are presented in Table 3.
| TABLE 3: Correlation between aspects of the independent variable and performance of HIV projects (N = 188). |
Pearson correlation coefficients for each of the independent variables revealed that three types of the DHIs had statistically significant correlations with performance of HIV projects. Utilisation of DHI for appointment adherence 0.349, p < 0.01; utilisation of DHI for management of ARV drugs 0.507, p < 0.01 and utilisation of DHI data management 0.531, p < 0.01. However, utilisation of the DHI for HCWs to receive laboratory results did not have statistically significant correlation with performance of HIV projects 0.084, p = 0.288.
Association between independent aspects of utilisation of digital health interventions and performance of HIV projects
To determine the association between utilisation of DHIs and performance of HIV projects as well as the moderating influence of M&E practices, the following moderated multi-linear regression model was used (Equation 1):
Where:
y = performance of HIV projects
β0 = the constant
β1, β2, β3, β4 = coefficients for independent variables
X1 = utilisation of digital health intervention for clients’ appointment adherence
X2 = utilisation of digital health intervention for healthcare workers to receive laboratory results
X3 = utilisation of digital health intervention for drug management
X4 = utilisation of digital health intervention for data management
m = moderated effect of monitoring and evaluation practices
ε = random error
An analysis of the association between each independent variable and performance of HIV projects was determined using R-squared and β values. Results are shown in Table 4.
| TABLE 4: Model summaries of independent aspects of utilisation of digital health interventions and performance of HIV projects. |
Three categories of the DHIs had statistically significant association with performance of HIV projects as follows: (1) utilisation of DHIs for clients’ appointments adherence (R2 = 0.28, p < 0.001). This indicates that utilisation of the DHI for appointments’ adherence could explain up to 28% of variation in the performance of HIV projects; (2) utilisation of DHIs for management of ARV drug (R2 = 0.332, p < 0.001). This indicates that utilisation of DHI for the management of ARV drugs could explain up to 33.2% of variation in the performance of HIV projects; (3) utilisation of DHIs for data management (R2 = 0.34, p < 0.001). This indicates that utilisation of DHI for data management could explain up to 34% of variation in the performance of HIV projects. Utilisation of DHIs for healthcare workers to receive laboratory results did not have a statistically significant association with the performance of HIV projects (R2 = 0.121, p= 0.288). However, it could explain 12% of variation in the performance of HIV projects.
Association between combined utilisation of digital health interventions and performance of HIV projects
The association between the combined utilisation of DHIs and the performance of HIV projects was assessed using a multilinear regression model. Results are shown in Table 5.
| TABLE 5: Model summary between combined utilisation of digital health interventions and performance of HIV projects. |
The results show an R2 = 0.220 which implies that utilisation of the combined four DHIs (DHI for appointment adherence, DHI for service providers to receive laboratory results, DHI for ARV drug management and DHI for data management) could potentially explain 22.0% of the variation in the performance of HIV projects. With F(1,187) = 52.604; p < 0.000 as shown in Table 5, the results further demonstrate that the combined utilisation of the four DHIs had a statistically significant association with variation of performance of HIV projects.
Correlation between monitoring and evaluation practices and performance of HIV projects
A Pearson correlation coefficient test was conducted to establish the relationship between M&E practices and the performance of the HIV projects. Table 6 presents a summary of the results.
| TABLE 6: Correlation between monitoring and evaluation practices and performance of HIV projects (N = 189). |
The results show a Pearson correlation coefficient of 0.533 at p < 0.001 indicating that there was a statistically significant and moderately positive correlation between M&E practices and the performance of HIV projects.
Moderating influence of monitoring and evaluation on the association between utilisation of digital health interventions and performance of HIV projects
The final step of the inferential analysis was to introduce M&E practices into the overall model between utilisation of all the DHIs and the performance of HIV projects as a moderating variable. Table 7 presents the findings based on the moderated multilinear regression model.
| TABLE 7: Model summary of combined utilisation of digital health interventions, monitoring and evaluation practices and performance of HIV projects. |
With an R2 = 0.332, the model shows that M&E practices could potentially moderate the association between utilisation of the combined DHIs and the performance of HIV from 22.0% in Table 5 up to 33.2% in Table 7.
The moderating influence was statistically significant at F(2,185) = 45.946; p < 0.001 as shown in Table 7. Analysis of the coefficients in the model shows that both combined utilisation of DHIs, at p < 0.001 and M&E practices at p < 0.001 had statistically significant associations with the performance of the HIV projects. From the model, the study established that a unit increase in utilisation of combined DHIs could explain variation in the performance of HIV projects by up to 24.8%. In addition, a unit increase in application of M&E best practices could explain up to 39.4% of variation in the performance of HIV projects.
Discussion
The study established high levels of utilisation as well as a statistically significant association between the three types of DHIs, namely DHI for appointment adherence, DHIs for management of ARV drugs and DHIs for data management and performance of HIV projects. However, utilisation of DHIs for healthcare workers to receive laboratory results did not have a statistically significant association with the performance of HIV projects. It was established that during the period of the study there was limited testing of viral loads and early infant diagnosis because of the lack of laboratory reagents. This resulted in suboptimal utilisation of the DHIs for service providers to receive laboratory results, hence the insignificant association.
The role of M&E, in project management is undisputed. Monitoring and evaluation have the critical and central role of risk management by identifying gaps and challenges early enough and strategising how to overcome them while on the other hand, identifying high impact interventions and activities that can be sustained. Results in this study have shown that application of M&E best practices could potentially enhance the relationship between utilisation of DHIs and the performance of HIV projects. This is a call to digital health implementors to ensure that utilisation of DHIs within their projects is followed by robust M&E practices. Monitoring and Evaluation ensure that an intervention is delivered within the time schedule, scope and budget.
Theoretically, this study was pegged on the Diffusion of Innovation Theory and the UTAUT (Kaplan 2001; Venkatesh, Thong & Xu 2016). The Diffusion of Innovation Theory is a proposition that outlines how and why adoption of innovation in societies may spread over a long period of time. The theory was critical in understanding varied patterns in utilisation of the four types of DHIs that were studied. As per the mean scores presented in the results section, the DHIs for the management of ARV drugs were highly scored in terms of utilisation, followed by the DHIs for data management, then the DHIs for client’s appointment adherence and lastly the DHI for healthcare workers to receive laboratory results. Various factors could be responsible for the varying levels of utilisation. We hypothesise that various factors including the length of the use of the DHI, the implementing partners, approaches used and resources available to support use of a particular DHI could have influenced the different levels of use. It is also critical to note that the targeted users of an innovation are an important factor that influences the adoption and use of technological innovation. According to the theory of diffusion of innovation, the key actors during the implementation of an innovation can be categorised into innovators, early adopters, early majority, late majority and laggards.
On the other hand, the second theory upon which this study was grounded is the UTAUT. The theory’s four key tenets of expected performance, expected effort, influence within the social networks and conditions that facilitate acceptance and use of technology were critical in one, designing this study to focus on the perceptions of the healthcare providers, users of the technologies and managers. In addition, the UTAUT provided insights into understanding the influence of the M&E practices as part of the conditions that either facilitate or challenge utilisation of the technological innovations.
In terms of the choice of the targeted participants, the techniques employed in this study were in line with another study (Muinga et al. 2020), which focussed on health records and information officers as users of health information systems to understand the level of system adoption. There is usually an attempt to focus such studies on narrow patient-level outcomes. Whereas examining the effect of digital health at the patient level is critical, it is important to understand that it is the service providers who recruit the clients into these DHIs and encourage them to continue using the systems. The patients trust the service providers. In case the service providers do not seem to understand the importance of the systems then they won’t recruit the patients and implementation of the digital interventions is likely to fail. This study examined the broader picture of project performance. In this study, we conceptualised performance broadly to include perceptions on patient level and organisational outcomes such as reporting and expenditure. The results demonstrate that utilisation of DHIs could potentially impact both patient-level outcomes and organisational performance as a whole and from a broader perspective. This study proposes a new methodological approach that can enable researchers to holistically assess the performance of projects as a result of utilising a combination of DHIs.
This study calls for sustainable implementation of the DHIs within the HIV projects. Focus should be put on attaining interoperability as these systems are servicing the same patients. This could lead to high levels of efficiency and effectiveness in service delivery as a result of seamless data sharing (Bhartiya, Mehrotra & Girdhar 2016; Measure Evaluation 2015; Mwangi, Mudogo & Maghanga 2022). Consequently, given the evidence generated from this study on the impact of DHIs on HIV projects’ performance, we suggest that similar technologies could be employed in other health programme areas that serve clients living with chronic diseases for a long-term period. The coronavirus disease 2019 (COVID-19) pandemic demonstrated that certain diseases could actually limit physical interaction between patients and healthcare workers. Digital health interventions could be useful in bridging such gaps in service delivery.
Limitations of the study
Digital health interventions may not necessarily replace human beings or even manual processes in enhancing project performance. However, DHIs are critical in complementing human resources and manual processes. This study did not confirm the actual figures in terms of cause and effect of the DHIs on the indicators of HIV project performance. For example, we did not confirm whether there were actual improvements in patients’ adherence and retention rates or actual reductions in project expenditure and improvements in project reporting rates, timelines or quality of reports. The results in this study are purely based on the perceptions of the healthcare providers, users and managers scored using the Likert scale. The study was limited to descriptively establishing the association between utilisation of DHIs, M&E practices and performance of HIV projects based on perceptions of the service providers. The assumption of the study was that the perceptions of the service providers could determine uptake, use and influence of digital health projects on HIV health projects. We acknowledge that there are many other contextual factors including funding, human resources, infrastructure, implementing partners and patient-level socio-cultural and economic factors that affect performance of HIV projects. These factors were out of scope of this study. This study provides basis for further robust studies on the cause-and-effect analysis of DHIs on the overall performance of health projects.
Conclusion
The study highlights the critical role DHIs could play in enhancing the performance of HIV projects based on perceptions of the healthcare providers, users and managers. Furthermore, the study has demonstrated that application of M&E best practices could significantly increase the association between utilising DHIs and the performance of HIV projects. This study calls for the continued use of DHIs and support for application of robust M&E practices to optimise performance of HIV projects.
Acknowledgements
C.M.M., A.M. and D.K. would like to acknowledge the research assistants who were critical in data collection, the leadership of Kisumu County Government and the participants.
Competing interests
The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.
Authors’ contributions
C.M.M. conceptualised, carried out the study, analysed the data and wrote the article. A.M. and D.K. helped in reviewing and shaping the study. All authors reviewed the article before submission for publication.
Funding information
This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Data availability
The data that support the findings of this study are available from the corresponding author, C.M.M. upon reasonable request.
Disclaimer
The views and opinions expressed in this article are those of the authors and are the product of professional research. It does not necessarily reflect the official policy or position of any affiliated institution, funder, agency or that of the publisher. The authors are responsible for this article’s results, findings and content.
References
Bhartiya, S., Mehrotra, D. & Girdhar, A., 2016, ‘Issues in achieving complete interoperability while sharing electronic health records’, Procedia – Procedia Computer Science 78(December 2015), 192–198. https://doi.org/10.1016/j.procs.2016.02.033
Cathy, M., Collins, M.M. & Christine, M., 2022, ‘Assessing the Interoperability of mLab and Ushauri mHealth Systems to Enhance Care for HIV/AIDS Patients in Kenya’, Journal of Intellectual Property and Information Technology Law 2(1), 83–116. https://doi.org/10.52907/jipit.v2i1.209
Chandrasekhar, C. & Ghosh, J., 2001, ‘Information and communication technologies and health in low income countries: The potential and the constraints’, Bulletin of the World Health Organization 79(9), 850–855.
Fatehi, F., Samadbeik, M. & Kazemi, A., 2020, ‘What is digital health? Review of definitions’, Studies in Health Technology and Informatics 275, 67–71. https://doi.org/10.3233/SHTI200696
Greenhalgh, T. & Russell, J., 2010, ‘Why do evaluations of eHealth programs fail? An alternative set of guiding principles’, PLOS Medicine 7(11), 1–5. https://doi.org/10.1371/journal.pmed.1000360
Joint United Nations Programme on HIV/AIDS (UNAIDS), 2019, Global HIV statistics, viewed 18 September 2024, from https://www.unaids.org/en/resources/fact-sheet.
Haleem, A., Javaid, M., Singh, R.P. & Suman, R., 2021, ‘Telemedicine for healthcare: Capability, feature, barrier, and applications’, Sensors International 2(100117), 1–13. https://doi.org/10.1016/j.sintl.2021.100117
Kaplan, B., 2001, ‘Evaluating informatics applications – Some alternative approaches: Theory, social interactionism, and call for methodological pluralism’, International Journal of Medical Informatics 64(1), 39–56. https://doi.org/10.1016/S1386-5056(01)00184-8
Kuriyan, R., Griffiths, K.J., Finkelstein, J.L., Thomas, T., Raj, T., Bosch, J.R. et al., 2014, ‘Innovations in nutrition education and global health: The Bangalore Boston nutrition collaborative’, BMC Medical Education 14(1), 5. https://doi.org/10.1186/1472-6920-14-5
Leon, N., Scheneider, H. & Daviaud, E., 2012, ‘Applying a framework for assessing the health system challenges to scaling up mHealth in South Africa’, BMC Medical Informatics and Decision Making 12(1), 123. https://doi.org/10.1186/1472-6947-12-123
Measure Evaluation, 2015, Interoperability considerations in the design, development, and implementation of mhealth projects measure evaluation strategic information for South Africa, pp. 2013–2016, mHealth/e-Health Technical Brief (September 2015), USAID, Cape Town.
Mehran, L., Nazeri, P., Delshad, H., Mirmiran, P., Mehrabi, Y. & Azizi, F., 2012, ‘Does a text messaging intervention improve knowledge, attitudes and practice regarding iodine deficiency and iodized salt consumption?’, Public Health Nutrition 15(12), 2320–2325. https://doi.org/10.1017/S1368980012000869
Minichiello, V., Rahman, S., Dune, T., Scott, J. & Dowsett, G., 2013, ‘E-health: Potential benefits and challenges in providing and accessing sexual health services’, BMC Public Health 13(1), 1. https://doi.org/10.1186/1471-2458-13-790
Ministry of Health Kenya, 2016, Kenya national eHealth policy 2016–2030, Policy Document, Nairobi.
Ministry of Health and NASCOP, 2020, Kenya population-based HIV impact assessment (KENPHIA), KENPHIA, Nairobi.
Ministry of Health Republic of Kenya, 2017, Kenya standards and guidelines for mHealth systems, Report, pp. 1–42, viewed 10 October 2024, from https://mhealthkenya.org/wp-content/uploads/2017/06/Kenya-Standards-and-Guidelines-for-mHealth-Systems-April-2017.pdf.
Molaei, M., Bosch-Rekveldt, M. & Bakker, H., 2019, ‘Extending the view on project performance’, Administrative Sciences 9(3), 65. https://doi.org/10.3390/admsci9030065
Moksony, R.H., 1990, ‘Small is beautiful. The use and interpretation of R2 in social research’, Szociologiai Szemle Special issue, 130–138.
Moullin, M., 2007, ‘Performance measurement definitions: Linking performance measurement and organisational excellence’, International Journal of Health Care Quality Assurance 20(3), 181–183. https://doi.org/10.1108/09526860710743327
Mudogo, C., Mulwa, A. & Kyalo, D., 2023, ‘Antiretroviral drug management tool and performance of HIV/AIDS projects in Kisumu County, Kenya: A digital heath solution approach’, European Journal of Theoretical and Applied Sciences 1(4), 608–617. https://doi.org/10.59324/ejtas.2023.1(4).56
Mudogo, C., Sabina, M.A., Kyalo, D.N. & Mwangi, C., 2023, ‘Enhancing performance of HIV/AIDS projects through digital health interventions for client appointments in Kisumu County, Kenya’, European Journal of Theoretical and Applied Sciences 1(4), 618–629. https://doi.org/10.59324/ejtas.2023.1(4).57
Muinga, N., Magare, S., Monda, J., English, M., Fraser, H., Powell, J. et al., 2020, ‘Digital health systems in Kenyan public hospitals: A mixed-methods survey’, BMC Medical Informatics and Decision Making 20(1), 21–14. https://doi.org/10.1186/s12911-019-1005-7
Rogers, E., 1995, Diffusion of innovations, viewed 05 October 2024, from https://teddykw2.wordpress.com/wp-content/uploads/2012/07/everett-m-rogers-diffusion-of-innovations.pdf.
Sherman, E., Niu, J., Elrod, S., Clauson, K.A., Alkhateeb, F. & Eckardt, P., 2020, ‘Effect of mobile text messages on antiretroviral medication adherence and patient retention in early HIV care: An open – Label, randomized, single center study in South Florida’, AIDS Research and Therapy 17, 16. https://doi.org/10.1186/s12981-020-00275-2
Silva, H.M.D., 2013, Performance measurement in project management, viewed 10 October 2024, from https:file:///C:/Users/MOH%208/Downloads/Performance_Measurement_in_Project_Manag.pdf.
The National Parliament of Kenya, 2023, The Digital Health Bill, viewed 01 November 2024, from https://www.health.go.ke/sites/default/files/2024-01/Digital%20Health%20Bill%20Final_0.pdf.
Venkatesh, V., Thong, J. & Xu, X., 2016, ‘Unified theory of acceptance and use of technology: A synthesis and the road ahead’, Journal of the Association of Information Systems 17(5), 328–376. https://doi.org/10.17705/1jais.00428
Vesel, L., Hipgrave, D., Dowden, J. & Kariuki, W., 2015, ‘Application of mHealth to improve service delivery and health outcomes: Opportunities and challenges’, African Population Studies Special Edition 29(1), 1683. https://doi.org/10.11564/29-1-718
World Health Organization (WHO), 2016a, Monitoring and evaluating digital health interventions: A practical guide to conducting research and assessment, WHO.
World Health Organization (WHO), 2016b, WHO | Monitoring and evaluating digital health interventions, WHO, viewed 10 November 2024, from https://www.who.int/reproductivehealth/publications/mhealth/digital-health-interventions/en/.
World Health Organization (WHO), 2018, Classification of digital health interventions, World Health Organization, Geneva, pp. 1–20, WHO/RHR/1, viewed 05 November 2025, from https://www.who.int/publications/i/item/WHO-RHR-18.06.
World Health Organization (WHO), 2019, WHO guideline: Recommendations on digital interventions for health system strengthening, World Health Organisation.
|
Hide
Show all
Email this article (Login required)
