Assessing the impact of #ict4d interventions: Going beyond access and infrastructure indicators

One the many challenges of a development practitioner is to assess the impact of development interventions. When you compound this with also figuring out how an ICT4D component has helped or hindered development and progress, this may become a bit challenging.

While the development community has comprehensive set of indicators for rural development and agriculture-related interventions, we are lagging a bit behind vis-a-vis ICT4D indicators.

This said, our ITU colleagues have compiled a set of core ICT indicators covering access and infrastructure.  While this is commendable, these two set of indicators are not sufficient to tell the full development story. For one thing, for example, the access indicators are limited to the physical access to ICTs and do not take into account aspects such as literacy. As such coming up with a comprehensive set of ICT4D indicators is up for grabs.

So here is what I've been thinking about, and would like to know if we were to complement access and infrastructure indicators with  appropriateness of ICTs for the target population and how these are used and the extent to which they contribute to transformation at social and economic level, would this be a good starting point to come up with a comprehensive set of ICT4D indicators?

Another domain where we require indicators is that of national policies so that we can assess whether or not these are conducive both for the target population and potential investors, whether policies allows open and transparent competition.

Last but not least, the sustainability of the intervention and its potential for scaling up could constitute another domain.

We know that development interventions have their own set of indicators. I am now totally convinced that the only we can assess the impact of an ICT4D intervention for all different perspectives and angles is to embed the specific ICT indicators as part of the overall development project, as opposed to having standalone indicators. This will allow us to have  a better grasp as to how and if the ICT4D intervention has contributed to the overall socio-economic development impact. 

In terms of infrastructure and access,  the ITU indicators provide statistics as to how and whether individuals, households and businesses have access to landline, mobile phone, extent of mobile phone penetration and use, number of computers, availability and use of broadband, etc. 

Moving now to the proposed domains - in terms of appropriateness, how can we assess whether a technology is appropriate? How can we assess if a service delivered thanks to a technology is appropriate? Could we say that if a household is willing to spend x% of its disposable income on an ICT service, that makes it appropriate? Can affordability be a parameter? Could we say that if a community has owned the technology that makes it appropriate? What about the cultural appropriateness of a technology?

I would say definitely locally relevant content is something that we should take into account, along with how technology has contributed and provides for  social and economic opportunities for progress. 

On the transformational side, one indicator to consider is whether the introduction of ICTs has led to the community acquiring new skills and whether there was any type of capacity development both at individual and/or institution level. This could be anything from improved negotiation skills, to acquiring technical knowledge on the use of the technology, to automation of manual tasks, leading to transition from semi-skilled to skilled labour. 

Another indicator could be whether the introduction of ICTs has contributed or enhanced social inclusion and interactions.

Taking this further, we could  examine whether the timely access to information has led to better decision making and whether the introduction of ICTs has been an impetus for increased and improved local content creation leading to the demise of information gatekeepers.

Last but not least, in this category perhaps another indicator could be the extent to which ICTs were equally available to women and young people and how and if this has led to their empowerment and positioning them on an equal footing with other members of the community.

As far as the economic indicators are concerned some obvious ones are how and if ICTs have:

• created new employment opportunities and if so has this been in the formal or informal labor market, off-farm or on-farm; whether new businesses were formed and how has it contributed to enhancing bargaining power of the beneficiaries. 
• led to creating a vibrant rural environment which has helped curtail the migration from rural to urban areas 
• contributed to increase in income and what is the percentage of increase in GDP thanks to deployment of ICTs. Taking this further, I wonder if we can go as far as being able to give figures of people lifted  out of poverty thanks to a specific ICT or thanks to a specific ICT4D intervention
• led to an increased expenditure in this sector at household level. Can we assume that if there is an increase in expenditure  it is because the household finds the technology appropriate and the content it is delivering appropriate?

Moving on to the policy level, here is a menu of option:

• are ICTs part of sectoral national policies. For example, is the agriculture, health or education national policy ICT enabled
• does the country have a national technology policy and if so does it advocate for universal access and in what form
• are the national policies conducive for creating the right environment for public-private-people partnership
• do national policies encourage public and private sector to invest in ICTs
• do national policies  foster competition and transparency
• are the ICT policies gender and youth sensitive - do they ensure equitable access 

Last but not least on the domain of scalability and sustainability, I guess we should be assessing the degree to which the ICT4D intervention responded to and met the needs of the local communities and assess the sustainability of the intervention once the funding is over. This could be in terms of knowledge transfer to maintain and operate the technology; the sustainability of the business model in the case that the ICT4D intervention led to creation of a business and assessing the prospects of expansion.

In terms of scalability we would need to assess the replicability of the intervention. Here I am not talking about a cookie cutter approach, as this never works. I am talking about understanding and assessing the context and evaluating the feasibility of replicating an experience in a similar environment and/or  assessing what modifications need to be made so that it can be replicated in a different context. We know that 9 times out of 10, this would require tweaking and adaptation to meet the needs of  the local population and respond to local reality.

To conclude, I am putting on the table some of my thoughts and I would like to seek your views and guidance on what could potentially be a sound set of indicators to assess the social, economic, political impact of ICT4D interventions?

And lastly what do you think are or should be the ingredients of a "successful" sustainable and scalable ICT4D intervention?