“Non-Profits and the Big Data Revolution” – Simone Cheung

May 19, 2014

As the amount of data in the world increases exponentially every day, business models have adapted to becoming highly data driven in all  aspects, from marketing to the provision of services.  This article examines how non-profit organizations can also take advantage of the multitude of data available to them.

Simone Cheung

It is estimated that we create 2.5 quintillion bytes of data every day. New technologies have meant that 90 percent of the data in the world today has been created in the last two years. Every time something is posted on Facebook, Instagram or Twitter, a purchase is made with a credit card, or the GPS feature on a mobile phone is used, the Big Data revolution expands.

Big Data has already been gaining traction in the business world for some time, with companies and scientists becoming highly data driven in every aspect of their business – from market research to providing personalized services and gaining new customers. In light of this, can we use Big Data for social good? And how can the not-for-profit sector benefit from it?

Why should non-profit organisations care about Big Data?

There are two ways in which non-profit organizations can benefit from the Big Data revolution. On one end of the spectrum, they can use Big Data to develop new programs and initiatives as well as to better target donors and recipients of aid.

This notion has been tested in the area of disaster relief. For example, mobile phone position data from the largest mobile phone company in Haiti (Digicel) was used to estimate the magnitude and trends of population movements following the Haiti 2010 earthquake and subsequent cholera outbreak.ii This led to significant improvements in the allocation of relief supplies and the quality of needs assessments following the occurrence of natural disasters, as well as during infectious disease outbreaks.

On the other end of the spectrum, Big Data can assist non-profit organizations to answer one of the biggest questions they are interested in: how do we know that what we are doing is making a difference? With the increasing pressure on the not-for-profit sector to become more transparent and accountable, monitoring and evaluation is becoming more and more important to demonstrate that programs and initiatives are achieving their objectives. Having an evidence base to better direct funding from donors is unquestionably beneficial for the not-for-profit sector.

Proving the correlation between programs or initiatives and improved outcomes is straightforward. It can usually be done even in the absence of appropriate data or rigorous methodological expertise in data analysis.  Policy-makers and donors are increasingly wanting to know more about the notion of causation. That is, they want to know whether changes in outcomes (whether they are positive or negative, or whether they are for an individual or a community or a country) can be directly attributed to a specific program or initiative implemented by non-profit organisations.

The availability of Big Data makes it much easier to establish causation , rather than mere correlation. However, it is important for non-profit organizations to be equipped with the skills to define the ‘question’ they are trying to solve, or the ‘objectives’ they are trying to achieve before launching into data analysis. Too often, organizations ‘mine’ available data to find the solution to a problem that does not yet exist.  Monitoring and evaluation should begin even before a program or project has been implemented, from collecting baseline data to tracking changes before and after implementation.

More does not necessarily mean better

The Big Data revolution is inescapable, but the risk is that the scale of data creation is now growing exponentially faster than our ability to analyze it. The sheer volume of data available can be overwhelming for most non-profit organizations, especially the smaller ones. Most non-profit organisations do not have the capacity nor the funding to make large technology investments to store, process and derive valuable insights from large datasets.

The key to enabling non-profit organizations to benefit from Big Data is to realize that more does not necessarily mean better; volume does not ensure veracity. What is needed is to identify ways to use currently available data more effectively.

There are five manners in which Big Data can create value in the social sector:

  • Making Big Data more readily available can increase the speed and accuracy with which social interventions can be deployed.
  • Big Data can enable better monitoring and evaluation of programs and policies, and allow better identification of improvements to programs and policies.
  • Big Data allows more targeted and tailored interventions and initiatives to ensure maximum impact.
  • Analysis of Big Data can also improve decision making, for instance, by making risk assessments faster and more accurate.
  • Analysis of Big Data can enable the creation of new products, services and business models to serve disadvantaged population groups.

What are the challenges?

While Big Data may be a valuable tool for non-profit organisations in the new era, there are a number of barriers they need to overcome to take full advantage of the Big Data revolution.

One of the biggest concerns with the copious amounts of data available is confidentiality. With the tremendous amount of data being constantly collected and analyzed by statistical agencies and organizations, privacy has become a key concern. This is particularly the case for social organizations as they are more likely to deal with data on vulnerable population groups and sensitive topics such as child welfare, domestic violence, and drug and alcohol misuse. As a result, non-profit organisations often find it difficult to gain access to data despite the vast amount of information being collected.

While there are a number of statistical methods to address privacy and confidentiality concerns, such as suppression, aggregation and adding random noise, it still remains a primary consideration for non-profit organisations, particularly as more and more data become publicly available.

Perhaps a greater challenge associated with the Big Data revolution is the shortage of talent and resources available to fully utilise the data. This ranges from structural constraints such as computing power and software tools required to integrate, analyse and visualise Big Data, to specialised data scientist skills in integrating datasets in a meaningful and valuable way.

Despite organisations, such as DataKind, which match scientists and statisticians with non-profit organisations, to facilitate pro-bono data work, shortages in these roles still pose major challenges for the not-for-profit sector.

Skilled statisticians and data scientists are particularly important when dealing with poor quality data, which is common in many developing countries. Poor quality data requires sophisticated methods to analyse and draw meaningful conclusions from.

The future of Big Data for non-profit organizations

In the past, data collection was often thought of as merely a means of supporting financial accountability for non-profit organisations, and served as a requirement for donor funding. However, there must be a culture shift to recognize the true importance of measuring the impact and outcomes that the not-for-profit sector is achieving. This will not only ensure limited resources are directed to those most in need, but  will also allow non-profit organisations to devise more targeted programs that have greater social impact.

There is no doubt that Big Data has the potential to achieve worthy social goals and bring about lasting social change. This is particularly the case in areas such as disease surveillance, micro-finance, monitoring and evaluation, and disaster relief.

The Big Data revolution is here to stay. Data could hold the key not only to attracting and targeting donors, but also to ensuring non-profit organizations operate more efficiently and effectively to achieve truly positive outcomes. We only need to make sure that we find effective methods of maining the data to avoid drowning in it.



i ‘Bringing Big Data to the Enterprise’ IBM  <http://www-01.ibm.com/software/au/data/bigdata>

ii Linus Bengtsson, Xin Lu, Anna Thorson, Richard Garfield and Johan von Schreeb, ‘Improved Response to Disasters and Outbreaks by Tracking Population Movements with Mobile Phone Network Data: A Post-Earthquake Geospatial Study in Haiti’ (2011)  PLOS Medicine, 8(8) 1.