Machine Learning for Personalized Care: Our Journey

We believe that technology can transform the primary healthcare experience in low and middle income countries (LMICs), improve coordination and the standard of care delivered by health workers, and ultimately making care more accessible, personalized and tailored. Last year, we wrote about the start of our journey to explore the use of machine learning for personalized care. In this blog, Tracey Li, Senior Data Lead at D-tree, writes about what we have learned so far, and how we see the future.

A year ago, we started partnering with researchers from N/LAB at the University of Nottingham, to develop a machine learning model to identify which pregnant women from Zanzibar’s national community health program were at risk of their child dying during or after birth. For those women identified as high-risk, the digital app at the core of the program would automatically prompt Community Health Workers (CHWs) to deliver a special service package to the women to decrease the risk. A year on, we have completed the implementation of this pilot project and are using our experience to guide our long-term plans in this area.

The gap between research and practice

Machine learning undoubtedly has the potential to make transformative improvements to healthcare delivery. When integrated with digital applications for uses such as disease diagnosis and patient risk stratification, machine learning algorithms can guide healthcare workers as they are delivering care, resulting in more patients receiving the right treatment at the right time. 

But although the potential is huge, getting there is challenging. We are currently seeing a large gap between the research environments in which machine learning models are typically developed and the real world environments in which the models will be operationalized. This is especially true when operating in environments where resources are limited, such as Zanzibar. It is crucial that we understand how real life implementation differs from these theoretically ideal scenarios in order to be able to bridge the gap. Our early work in this area in Zanzibar has enabled us to do exactly that, and we have identified 2 key learnings for future development.  

It quickly became apparent that the accuracy of the machine learning model was not going to reach a ‘high precision’ level. D-tree works with CHWs in Zanzibar who collect data during routine visits, but this data is not sufficiently detailed to enable highly accurate predictions. We need to obtain more data, such as data collected at health facilities, to improve the quality of the prediction. We also need to think about combining the machine learning output with other sources of information, such as input from human experts, to make the machine learning model one component of a broader client assessment system rather than it being the only component. 

Secondly, we found that although the CHWs in our programs have proven successful at increasing the health-seeking behaviors of clients, such as attending a health facility to give birth, we can’t assume that clients will immediately respond to all the advice from a CHW. We also know that reducing the risk of neonatal death requires clinical interventions, and not only behavioral changes. We therefore need to invest more in developing the interventions that a CHW delivers, to ensure that they result in the behavior change we are looking for, as well as think about how best to support health facility staff to be able to deliver the necessary clinical interventions.

We were pleased to see that the project was well received and that the Zanzibar Ministry of Health supports the exploration of technological innovations to improve healthcare. CHWs also responded positively to the project as they felt they were better positioned to identify which women needed additional healthcare. They also demonstrated that they have the time and willingness to provide extra services to high-risk clients. From a technology perspective, we reached the significant milestone of deploying and testing a user-friendly risk assessment tool, powered by the machine learning model, that several hundred CHWs were able to use effectively, even when offline. 

“We reached the significant milestone of deploying and testing a user-friendly risk assessment tool, powered by the machine learning model, that several hundred CHWs were able to use effectively, even when offline” 

Because of the promising learnings and interest from the Zanzibar Ministry of Health, we are looking at ways to improve the quality of the machine learning risk classification, taking into account the data that will be available. This includes investigating the development of hybrid human/machine systems, where we believe the combination of human expertise and machine learning can produce better results than either one alone.

What’s next for Zanzibar?

Our commitment lies in ensuring that everyone, no matter where, has access to the healthcare they need to live longer, happier lives. People, not technological innovations, are our focus. We know that a machine learning model can inform us about who needs to receive a particular intervention, but it is the intervention itself that will directly affect a client’s health outcome. We therefore plan to focus as much on the development of interventions as on the development of machine learning models. 

Over the medium term, rather than focusing only on directly predicting physical health outcomes, our plan is to work with partners at Harvard Medical School to deploy a machine learning model that instead predicts a behavior – which pregnant women will give birth at home rather than at a health facility, even after receiving the standard package of CHW services (which includes promoting the importance of health facility delivery). We have found that this model does perform relatively well and we are confident that we can develop an effective CHW intervention to mitigate the risk. Since delivering at a health facility is known to be one of the most effective ways of ensuring that mother and baby remain healthy throughout birthing, this behavior can have a huge impact on health outcomes. We plan to evaluate the size of this impact and the cost-effectiveness of the setup, to inform our long term plans. Whilst doing so, we will continue to explore avenues for directly predicting and mitigating adverse outcomes such as neonatal death. Our long term ambition is to develop a system that can predict multiple risks, and that enables multiple intervention pathways that meet each client’s unique combination of health needs.

We are grateful to Enabel, the Belgian Development Agency, for financially supporting the implementation of this project through the Wehubit program, and to our partners at N/LAB at the University of Nottingham, and our partners at Harvard Medical School, for their collaboration.

Photo credit: Brain technology photo created by rawpixel.com