Data Deluge and Public Policy: Promises and Perils

Session Report
Pranav Vijay Sonar

An online International Summer School Program on “Data, Monitoring and Evaluation” is a two-month immersive online hands-on certificate training course organized by IMPRI Impact and Policy Research Institute, New Delhi. The second week of the program on June 10th, started with a session on ‘Data Deluge & Public Policy: Promises and Perils’ by Dr. Soumyadip Chattopadhyay.  This session gave an insightful view of enhancing application of numerical literacy in Public Policy in India. The session was opened with introductory and welcoming remarks from Fiza Mahajan and was further moderated by her.

The session was opened by Fiza Mahajan, Visiting Researcher at IMPRI. She starts with details about the Program and provides a thematic view of the speakers who will be guiding during the program.  She says that there will be numerous curated expert sessions on issues to learn the insights of theory, methods, tools, techniques, and applications. Then she welcomes the guest lecturer Dr. Soumyadip Chattopadhyay, Associate professor of Economics, Visva Bharati ShantiNiketan, and visiting senior fellow IMPRI.

Dr. Chattopadhyay talks about the huge amount of data that we are generating in recent times and the implications of it in public policy decisions and choices. He explains that his main focus for the session is on the issues that are relevant to India. As an economist, he feels that the exponential increase in the volume of published data has led to what he calls the  ‘data revolution’, which is creating new challenges for data analysis. And to conquer these challenges we would need new theories, methods, and tools for data integration and visualization.

He further explains that we are flooded with data which is raw. And what we require is the good quality of data. In this aspect, he provides what are called the five V’s of data. And managing these five qualities, namely Volume, Variety, Value, Velocity, and Veracity helps understand the nature of data which is later used to produce good quality data which can be used further to add value to public policy decisions.

What is Public Policy?

He defines public policy as a set of actions that affect the solution of a policy problem and the capacity to create public value.  And these public policies have the capacity to solve individual problems and improve the quality of life at a macro level. With the help of data, this quality can be severely improved.

He explains that due to the digital economy, the cost of storage and disseminating data has fallen. While the efficiency of data gathering and processing has increased, which is leading to decreasing marginal cost of data. And along with these changes, the marginal benefit of data has increased. Because, it is promoting evidence-based policy, accountability in public services, and product innovation. He gives an economist’s view to look at the lower marginal cost and higher marginal benefit which explains how much this data revolution is benefitting. He states that this optimal quantum of data that society should consume is much higher than before.

Dr. Chattopadhyay is of the opinion that there is a need for a Government-driven data revolution. The government keeps track of administrative data which includes birth and death records, pensions, tax records, etc, and survey data like census and national sample survey data. This data is comprehensive and robustly maintained. He feels that the Government of India is trying to improve innovative data generation of transactions data and institutional data which includes public school data or public hospital data of patients.

He further explains what ways are implemented for an integrated data system. There are four steps to working with data, namely gathering, storing, processing, and disseminating the data. The recently launched digitized India is a kind of ingenious solution for converting paper-based data into a digital form and initiating digital data collection at source.

For the storage of data, there is a need to reduce the time lag between collection of data and data entry. Although records of data cannot be accessed by all the stakeholders, so for data to serve its purpose, it tends to be transmitted into a digital machine-readable form that can be downloaded very instantly. Thus there is a need to create scheme dashboards and open district-level dashboards to the public for dissemination of the data.

Then he dives into the process of putting all these benefits of huge data to form policy for people. To explain this he introduces the three I concepts namely, Integrate, Insights, and Impact. He gives an example of a huge amount of healthcare data that are kept in electronic medical records. One can integrate this data and march that with the health of students and different administrative databases. If it can be integrated in this fashion, then it would provide some useful insights which will be used for informed decision-making. This would be further helpful for us to identify the unmet needs and demands of people and monitor the feedback of citizens to cater to the impact on the people.

He concludes the session by raising some of the newer challenges that are ahead of policymakers. He feels that the mere availability and accessibility of data do not make it ethical. And there is limited access to data which creates a new digital divide. And one more issue which he feels should be addressed is the accountability and responsibility of public policymakers. He ends the presentation by stating the interesting paradox between data and public policy, which is that this data paradigm and these transitions towards huge data need to be accompanied by some proactive public policy options and some targeted development that will serve the basic purpose or basic utility of data, only then the data will be appreciated.

Acknowledgment: Pranav Vijay Sonar is a research intern at IMPRI.

Read more session reports on web policy learning events conducted by IMPRI: India’s Digital Public Diplomacy and Brand Building



    IMPRI, a startup research think tank, is a platform for pro-active, independent, non-partisan and policy-based research. It contributes to debates and deliberations for action-based solutions to a host of strategic issues. IMPRI is committed to democracy, mobilization and community building.

  • Bhanvi