Background

This evaluation looks at a project led by On Cue, a small company based in Cape Town, which sends Short Message Service (SMS) messages to patients via mobile telephones, reminding them to take their TB medication at pre-determined times. The Compliance Service aims to provide an affordable solution to improve patient adherence to TB treatment and reduce the associated costs of the DOTS system for both patients and clinics. This evaluation report presents the lessons learned so far to inform decision making about future rollout of this system, as well as other uses of cellular technology in the healthcare sector.

The evaluation set out to (1) determine the effect that the use of the Compliance Service had on TB cure rates and treatment completion rates; (2) identify and describe any related social and economic impacts that may result from the use of the technology in this context; and (3) conduct an assessment of the Compliance Service in terms of whether, and how, best practice principles for project management have been implemented. The evaluation involved three groups of key stakeholders: patients receiving the Service, clinic staff, and TB experts and managers at the City of Cape Town Health Directorate. The project evaluation combined quantitative and qualitative data collection. Information was collected from patient records, background documents and reports, clinic visits, and structured interviews of patients and staff through the use of questionnaires.

Findings

The project management issues are so inherently intertwined with the technology that it is difficult to separate them. Project implementation clearly limited the effectiveness of the Compliance Service, but it is not a reflection on the usefulness of the technology itself. To the contrary, the Service has potential to provide more choice in the care of TB and greater convenience for the patient. However, the problems encountered underline the limitations of the Service and imply there are important conditions for its success.

A number of obstacles to widespread rollout exist. Monitoring for treatment adherence is a problem where patients are not seen daily (as they are with DOTS). An overall lack of ownership of the project at the clinic limits the proactive participation of the staff, and no one on-site takes responsibility for ensuring the Service is implemented effectively. A lack of regular feedback and interaction between the City, On Cue, and the clinic creates a “disconnect” that hinders success in a number of ways. A number of practical implementation issues limited the effectiveness of the pilot. Clinic staff schedules are tight and many staff members feel that they are over-worked. City and clinic bureaucracy limits the add-on functionality that would expand the usefulness of the Compliance Service. Issues of privacy, data protection, and security will affect the widespread use of technology in healthcare in Africa over the long-term.

In our view, the Compliance Service pilot should be re-implemented and re-evaluated, leveraging on the lessons learned in this initial evaluation. The renewed pilot should be conducted according to clear, written procedure for running the Service, and recording data derived from it. The criteria for patient selection must be clearly defined. Patients must be educated such that the healthcare worker is satisfied they will remain adherent if selected for the Compliance Service. And to improve adherence levels, a purposeful effort is required to monitor adherence on the few opportunities healthcare workers have to see patients on the Compliance Service.

Conclusion

The technology works and it is effective. And on face value, it also provides a more cost-effective treatment option, both for the health service and patient. The convenience of TB treatment for the patient is also greatly improved. But, the Compliance Service is only a viable option if adherence levels are at least those of clinic-based DOTS. The evaluation has shown that implementing the Compliance Service involves a trade-off between the gains made on cost and convenience and the losses from having to put extra efforts into getting to know, and monitor self-supervised patients. This technology is not a silver bullet to solve the problem of patient adherence: it is all down to the way in which it is implemented.

View the report