MW4D Impact Analysis

Synopsis

Impact analysis is a critical dimension in all ICT4D projects. How do you measure a specific project impact, what are the factors to consider, and appropriate methodologies to use are key questions. While this topic is wider than just mobile applications, there is no well established methodology to measure the social impact of a specific service. This document will analyze the current methodologies used in ICT4D projects to assess impact and results, evaluate their applicability in the mobile space, identify the key factors to take into account to measure social impact, and how to measure them.

Resources

[1] ITU. (2010) Measuring the Information Society 2010. Geneva, Switzerland.

[2] Purwandari, B., Hall, W. and De Roure, D. (2010) The Impact of the Mobile Web in Developing Countries. In: Web Science Conference 2010, 26-27 April, 2010, Raleigh, NC, USA.

[3] Chaudhuri, A. (2010) Mobiles, Income and Utility: A Methodological Assessment of m4D. National Forum on Mobile Applications for Inclusive Growth and Sustainable Development, New Delhi, India

[4] Duncombe, R. (2009) Impact Assessment of Mobile Phones on Development: Concepts, Methods and Lessons for Practice. Centre for Development Informatics Institute for Development Policy and Management, SED, University of Manchester, Manchester, UK.

[5] Boyera, S. (2009) Mobile Web for Social Development Roadmap. W3C

[6] Bhavnani, A., Chiu, R.W., Janakiram, S. and Silarszky, P. (2008) The Role of Mobile Phones in Sustainable Rural Poverty Reduction. ICT Policy Division, Global Information and Communications Department.

Contributors

• BettyP
• MiraS
• RenjishK
• StephaneB
• ShwetankD

Content

A methodology to measure social impact of the Mobile Web for development is started by developing a model, which demonstrates Mobile Web uptake. From each component within the model, we can make lists of indicators to be measured. Then we define how to measure the indicators, as well as relationships among them.

Model of Mobile Web Uptake for Social Development

In order to measure social impact of the Mobile Web, we need to understand processes involved in Mobile Web uptake for development. These processes are illustrated in a model that is adapted from previous related work. The first model is the three-stage model towards information society from the International Telecommunication Union (ITU) [1]. The second one is an impact value chain model of mobile-development (m-development), which was produced by researchers at the Centre for Development Informatics, Institute for Development Policy and Management, University of Manchester [4]. The last part of this section proposes a Model of Mobile Web Uptake (MMWU) that will lead to indicators to be measured by the impact analysis methodology [2].

1. The ITU Three-Stage Model [1]

The Mobile Web for Social Development is part of Information and Communication Technology (ICT). Therefore we can learn from the ITU three-stage model of ICT development process towards an information society, which is shown in Figure 1 [1]. It consists of 3 stages: ICT readiness (infrastructure, access), ICT use (intensity) and ICT impact (outcomes). ICT capability (skills) is another component that significantly determines ICT effective use.

Figure 1 The ITU three-stage model of information society [1]

This model emphasizes on calculation of the ICT Development Index (IDI) from indicators of ICT Readiness, ICT Use and ICT Capability. As a result, there is a thick circle surrounding the ICT Readiness, Use, and Capability, but the ICT impact is intentionally put outside the circle.

2. A Model of Mobile-for-Development (M-Development) Impact Value Chain [2]

Researchers at the Centre for Development Informatics, University of Manchester proposed a model, which they call mobile-for-development (m-development) impact value chain [4]. They do not focus on the interplay between the Mobile Web and its stakeholders. However, they concentrate on Impact Assessment (IA) to evaluate how effective and efficient m-development funds are being used.

Figure 2 The model of mobile development (m-development) impact value chain [4]

Outputs are defined as micro level changes in behaviour or practices, which are associated with mobile phone use. Outcomes are defined as measurable differences in cost and benefit related to mobile phone intervention. Development impacts are defined as the contribution of mobile phone intervention to broader development goals, such as the growth or decline in socio-economic indicators.

3. Proposed Mobile Web Impact Indicators [2]

We need to modify the ITU three-stage model and the model of m-development impact value chain, as they deal with ICT in general or mobile phones, and not the Mobile Web. A proposed Model of Mobile Web Uptake (MMWU) is shown in Figure 3 [2]. The components and sub components of the MMWU are adapted from previous models as well as from Mobile Web challenges in the Mobile Web for Social Development Roadmap [5].

Figure 3 The Model of Mobile Web Uptake (MMWU) [2]

The success of Mobile Web uptake is often dependent on Mobile Web Content and Applications. Therefore Mobile Web Readiness (stage 1) does not only cover Mobile Web Infrastructure, but it also covers Mobile Web Content and Applications. On the contrary, content and applications are not involved in ICT Readiness [1]. They are also less considered in m-Development Readiness [2].

Other crucial issues are feedbacks from Mobile Web Impact (stage 3) to previous stages. Successful Mobile Web uptake can inspire Mobile Web application developers to improve their programming skill and deliver better applications. It can also stimulate people to better use the Mobile Web. Consequently, those can influence Mobile Web Use (stage 2). The impact can also trigger Mobile Web Supporting Enablers (Standards, Policy, Funding and Collaboration) to enhance Mobile Web Readiness (stage 1). It is important to emphasize the intertwining of the Mobile Web and its stakeholders as illustrated in Figure 4. It means that the Mobile Web has impacts on its stakeholders, and then they reshape Mobile Web technology and infrastructure. Those happen continuously as a life cycle.

Figure 4 The intertwining between the Mobile Web and its stakeholders, adapted from [5]

Indicators to be Measured

From each model discussed above, we list indicators that can be measured to achieve objectives of the models.

1. ICT Development Index Indicators [1]

ITU measures eleven indicators of each country, which are used to calculate a composite index called the ICT Development Index (IDI). It is considered to represent information society development within that country.

ICT Impacts
ITU does not have indicators to measure ICT impacts. However, it is written in ITU report in 2010 that ICTs have different economic effects. They can directly and indirectly increase welfare and facilitate social and economic development. Furthermore, the report says that better educational performance is statistically positively associated with greater household Internet access. There was also statistical association between households with Internet access and female labour force participation.

2. Indicators of Mobile-for-Development (M-Development) Impact Value Chain [2]

3. Proposed Mobile Web Impact Indicators [2]

The Model of Mobile Web Uptake (MMWU) in Figure 3 has five components, from where lists of indicators are derived. Some sub components and proposed indicators are adopted from [5] and [6].