Tag Archives: Learning

mHealth Framework WebEx

In our efforts to increase capacity across the region for the improved management of Technology for Development initiatives, the Regional ICT team is hosting a series of WebEx sessions on relevant T4D topics including: tools and technology solutions, useful innovations for Programme challenges, and methodologies for project management.

For our first session, we were very pleased to have Erica Kochi from UNICEF’s Global Innovations unit in New York join us to explain the mHealth Framework.  She was joined by a number of key partners who were involved in the mHealth Framework from its inception. The mHealth Framework has been developed in collaboration with Johns Hopkins University, the World Health Organization, and frog Design. The mHealth Framework is a tool for helping governments, donors, implementing partners or other stakeholders understand how to determine appropriate technology solutions for health interventions, and how mobile technologies can improve health outcomes.  The framework aids practitioners in conceptualizing the larger “health system”, so that challenges, constraints, and key actors are better incorporated into project design and implementation. For more background on the mHealth Framework, please see this paper.

See below for the mHealth Framework.

Erica Kochi gave the introduction to the session, and discussed some of the opportunities in East Africa for integrating technology and innovation in design into health programming.  mHealth aims to make connections and bridge gaps across the continuum of care, and the mHealth framework was developed, “somewhat organically,” she says, to guide practitioners in navigating this evolving space.

Peter Benjamin, who is Director of mHelp, the Capacity Building unit of the mHealth Alliance, gave us some background on mHealth and how it has evolved in the last few years.  While mHealth has come a long way, there are still many challenges in health systems that mHealth, as a tool, is not yet able to solve.  The challenges for mHealth solutions include improving interoperability, determining financially sustainable business models for scaling mHealth solutions, and improving the evidence to determine its tangible benefits. When it comes to scaling mHealth solutions, Peter emphasized that it is important to keep in mind the end-user, to plan for scale from the beginning, and to invest in evaluation so that lessons learned can be fed into future projects.

Alain Labrique is Director of mHealth Initiatives at Johns Hopkins University and focuses on Health Systems in Asia and East Africa.  He discussed how mHealth technologies improve coverage of over-stretched health systems. Johns Hopkins and WHO have been working to develop a mHealth Taxonomy in order to standardize the language around mHealth interventions.  Alain highlighted how using a common language will help practitioners identify complementary efforts and existing gaps. The mHealth Framework takes this one step further, by identifying and visualizing common constraints faced by actors in the system.

Garret Mehl conducts research on reproductive health and innovations for strengthening health systems at WHO.  He took us through the different components of the mHealth Framework and gave examples of how it can be applied.  The framework acts as a planning and communication tool, to help illustrate health projects to stakeholders and governments. It consists of validated interventions along the continuum of care (for now it focuses solely on Reproductive, Maternal, Newborn and Child Health), constraints and challenges, and possible mHealth applications (clustered by their different utilities). Garrett explained that the Framework allows for the “when, what, how and why a mHealth strategy is being deployed.”

Finally, Sean Blaschke (Health Systems Specialist, UNICEF Uganda) and Nick Oliphant (Health Specialist, UNICEF HQ) discussed their work on health systems strengthening at the Country Office level. Sean discussed the importance of working with government partners to implement national policies or strategies around the use of technology, and to focus on building the capacity of the Ministry of Health to manage and maintain a national health information system. He noted that one of the challenges to scaling mHealth initiatives can often be related to the enabling environment including legislative or regulatory frameworks.    Nick has been spearheading work with the University of Oslo to prototype the new features of the DHIS2, or District Health Information System. DHIS2 is being rapidly adopted in over 40 countries, 20 of which are national deployments, and can easily be mapped onto the mHealth Framework.

Thank you to those of you who joined this session on the mHealth Framework.  Once again, we’d like to extend huge appreciation to our presenters and to Erica Kochi for leading the call.  In addition, all of the documents and presentations referenced during the session can be downloaded here:

https://www.dropbox.com/sh/kq914f9lz2mqusn/R-1A3uxviY

Below are the two components of the framework from a blog post written by our friends at Global Innovation for UNICEF Stories a few months back. The visualizations and descriptions provide a helpful conceptualization of how the mHealth framework can be applied to strengthen health systems, and ultimately, improve access to health services.

Source: http://unicefstories.org/2013/08/08/mhealth-innovations-as-health-system-strengthening-tools-12-common-applications-and-a-visual-framework/

  1. A place to depict the specifics of the mHealth intervention, described as one or more common mHealth or information and communications technology (ICT) applications used to target specific health system challenges or constraints within specific areas of the RMNCH continuum of care.

Image

2. A visual depiction of mHealth implementation through the concept of ‘‘touch points,’’ or points of contact, which describe the specific mHealth interactions across health system actors (for example, clients, providers), locations (such as clinics or hospitals), and timings of interactions and data exchange.

GHSP-13-00031-Mehl_Figure 2

 

In the coming weeks we will be sharing an edited version of the session for those who missed it. And stay tuned for more WebEx sessions in the future!

The Beginnings of Data Collection – The First Baby Steps in a Marathon

Data. Information. Data collection. Good data. Big data. Data-for-development.

More and more, data is becoming an important tool within the humanitarian world to aid project design and define programme objectives. Data allows for real-time monitoring and evaluation, adapting projects to changing circumstances, and ultimately making projects and initiatives more effective. Everyday it seems a new tool or platform is developed to collect, analyze, and visualize data. But, for those of us that aren’t as experienced as the techies developing these (cool) tools, navigating the data collection waters can be overwhelming. Decisions need to be made about what data to collect, how to collect it, and  how it will be used and shared.

Below is quick, three-step process to begin thinking about data collection. The questions are not intended to provide a comprehensive “how to guide,” but should instead begin to stimulate thinking about the data collection process.

1) Deciding what data to collect is the first step when thinking about data collection. It’s true that a myriad of tool exists to collect data from FrontlineSMS to Formhub to Data Winners to TextIt to RapidSMS, and it is difficult to know which tool to choose. However, choosing the right technology is not the first step. Instead, knowing what information is desired and how it will be used is the first, and moreover, most important step.

Questions to think about when deciding what data to collect?

  • Is a specific piece of information desired? (Structured data collection)
  • Do you want to conduct an interactive poll that modifies questions based on previous answers? This is known as the skip logic method.
  • Who will benefit from this information?
  • Are partners involved?

2) Figure out the logistics of the project. Like all project design, context matters and solutions need to be designed around the situation on the ground. Many times understanding the contextual environment is the difference between a successful initiative and a so-called “failure.” For example, designing a Smartphone app for Community Health Workers and then learning that continual electricity doesn’t exist nor an Internet connection, could have been easily avoided if research had been completed about the end-users. Thinking about the following questions will not only contribute to the project design but will also help decide the appropriate technology to choose later on in the process.

Logistical and context questions to ponder:

  • Do you want continual data collection? At what interval and frequency?
  • Who will be doing the data collection?
  • Small-scale? Large scale?
  • Is there infrastructure to support SMS? Internet connections? Electricity to charge phones?

3) Choosing the best technology for the situation. The final step when thinking about data collection is the technology selection, and this happens only after the project designer knows what data is wanted and understands the working environment. Choosing technology can be a slightly daunting task as many tools and platforms exist. Answering a few questions will help guide the selection process.

Questions to help think about when making tech choices:

  • SMS or smartphone? Web? Interactive voice response (IVR)? Excel? (A useful list for SMS pros and cons is below.)
  • Which vendor to choose?
  • If a partner is involved will platforms be compatible?
  • Is an interactive feedback loop desired?
  • Do you want to visualize data and conduct analysis?
  • Would a cloud based system be suitable?

Pros and Cons to SMS outlined by Matt Berg:

Pros:

  • Any phone
  • Any network (no data required)
  • Messages queued (don’t drop messages when out of network or phone is dead)
  • Toll-free (defray cost of end-user but difficult to set-up)
  • Effective for monitoring. Users can trained to effectively use structured SMS
  • East to broadcast. Good for reminders/alerts

Cons:

  • Data quality / User error
  • Best effort delivery (message occasionally drop)
  • Not ideal for larger surveys
  • Literacy
  • Cost (high vs. data on wifi)
  • Data security

A multitude of data collection tools exist, and in order to choose the correct one understanding what is desired from the data collection is the first step before researching the tech landscape to make informed decision.

At the end of the day, data is supposed to enhance decision making by providing information that was previously unknown. Despite all of the various tools and methods for data collection, we should remember to strive for common data, and ultimately common knowledge. Data collection is a very powerful tool, and creating common awareness around the basics of the collection process is crucial if data is to have maximum impact in the humanitarian world.

Source:

“Getting to Common Awareness.” Matt Berg. Presentation given for T4D Capacity Building Workshop in Nairobi, Kenya. October 2013.

Coming to Terms with Failure in T4D and Innovation

“Ever tried. Ever failed. No matter. Try again. Fail again. Fail better.” Samuel Beckett

“Fail fast and fail forward. You learn and iterate. You document what you learn, share it with the world and look for insights wherever you find them.” Aleem Walji, World Bank Institute Director of Innovation Labs

These two quotes symbolize the increasingly popular modus operandi within the T4D and Innovation community. As T4D and Innovation mature, they are increasingly breaking away from the traditional operating paradigm that success is the be all and end all. Instead of hiding from our failures, we are attempting to learn from them and also accept that failure is an important step in the innovation process. The truth is that failures happen, but the crucial questions to ask are what do we learn from the experience, and moreover, how are the lessons learned applied to future initiatives.

Richard Heeks, Professor of Development Informatics at the University of Manchester and T4D thinker, outlined five broad reasons for why ICT4D projects fail:

  1. Failure to involve beneficiaries and users: those who can ensure that project designs are well matched to local realities.
  2. Rigidity in project delivery: following a pre-planned approach such as that mandated by methods.
  3. Failure to learn: not incorporating lessons from experience that arises either before or during the ICT4D project.
  4. Ignoring local institutional capacities: not making use of good local institutions where they already exist or not strengthening those which could form a viable support base.
  5. Ineffective project leadership: inability to direct and control the ICT4D projects.

Plenty of other reasons exist for why projects fail, but for all intensive purposes this list can be applied to most if not all. During the T4D Capacity Building Workshop in Nairobi during the last week of October, we learned about two UNICEF projects that failed. We applauded these the designers of these two projects for coming forward about the missteps taken, and more importantly we learned a lesson or two about what to avoid when planning future projects.

Case 1 – eNutrition and mHealth

This project aimed to design decision support applications to aid frontline health workers in child nutrition (eNutrition) and maternal health (eMNH) initiatives. An Adroid application was developed coupled with OpenMRS (an open source medical record system platform, more info here) was developed in partnership with a local NGO for two purposes: correctly identify, register, examine and treat malnourished children and increase adherence to national standards of care for antenatal care.

The project was piloted in six facilities with four facilities trialling the eNutrition platform and two the eMNH. Project implementers were initially hopeful about the pilot for three reasons. First, the project had relevancy since it was designed around the national Ministry of Health guidelines. Second, it was considered effective as the facilities where it was introduced reported improved and comprehensiveness of treatment. And third, the projects had impact as clients noted positive interactions with health workers using the eMNH and eNutrition platforms compared to comparator tools. However, despite these initial positive signs, the pilot revealed some serious flaws with the project design.

What happened?

Lack of Sustainability:

  • A comprehensive eHealth and eNutrition strategy was lacking as well as government leadership and coordination mechanisms.
  • There was a missing link with HMIS (Health Management Information System), causing parallel databases with the national health information system, meaning that the two databases couldn’t communicate with one another.
  • Health officials, the Ministry of Health, UNICEF and other partners in the nutrition and maternal health fields did not use the information that was gathered through the eMNH and eNutrition initiatives.

Scalability Issues:

  • The pilot was expensive
  • It became incomprehensible to think of scaling the project to 25,000 CHWs or 6,000 + Health Facilities.
  • No ICT staff was involved from the UNICEF side to advise.

Take Away?

  • Involve ICT from the beginning.
  • Complete an evaluation of the project during the pilot phase to know when or when not to scale.
  • Need to have government on-board with projects in order to fulfill the UNICEF mandate to support governments.
    • Think about ultimate ownership of the project from the beginning.
    • Not focus on just pilot – beginning thinking about scale-up early.
    • Think long and hard about the technology choice to ensure it fits the project needs and context.

Case 2- Connecting Classrooms, Communities, and Youth for Biodiversity Conservation

This project sought to design an open source communication platform allowing students from around the world to learn about and discuss global issues. Connecting Classrooms was developed in the developed world with the intention to be implemented in the developing world. The project aimed to put technology in students’ hands with the hope of empowering and connecting them to other students learning about similar issues. The diversity and remoteness of the host country environment is extremely challenging. Ultimately, the project creators were unable to negotiate the environmental challenges in the project design, which led to significant setbacks.

What happened?

Equipment:

  • Due to the challenging operating environment equipment needed to be rugged enough to survive, and this proved to be very difficult find.
  • The project team, therefore, decided to outsource equipment selection and testing to an NGO. However, UNICEF experienced a supply nightmare after the NGO didn’t have the funds to buy the equipment upfront to then have UNCIEF buy them from the NGO.
  • When the NGO designed and tested computers were no longer an option, the project team felt rushed and bought whatever equipment was available. However, the purchased equipment was ultimately inadequate for the environment.

Take Away?

  • Failure to field-test negatively impact projects. Knowing the environment and conditions of end-users is critical when designing projects.
  • Need to be prepared for the supply process and the rigid structures and bureaucracy of large organizations.

Even though the two projects highlighted above did not scale and qualify as “successful,” we should not discount the lessons learned. It is important not to be discouraged by failing, and at no times should the fear or cost of failure stop us from trying to innovate. Moreover, we should strive to be resilient to failure and not only learn from past mistakes. Failures happen, and as the two quotes at the beginning of this post stress, we should fail fast, recognize the mistakes, and then move forward.

 

For further reading on failures in ICT4D please see:

http://www.ictworks.org/2011/08/17/great-success-world-bank-has-70-failure-rate-ict4d-projects-increase-universal-acces/ (Encouragement to break with the status quo and not only publicize failures but embrace them.)

http://wayan.com/ict4d/10-levels-of-failure.html (Something to chuckle at.)

Sources:

“Can a Process Approach Improve ICT4D Project Success?” Richard Heeks. http://ict4dblog.wordpress.com/?s=failure