Data Deluge & Public Policy: Promises & Perils

Session Report
Vamsi Gokaraju


The online course on “Data Analytics for Policy Research”, aims to equip participants with advanced analytical skills. This program is organized by IMPRI Gen Alpha Data Centre. The session equipped policymakers, researchers, and data enthusiasts with cutting-edge analytical skills. The students went beyond theory and learned with hands-on training in data, analytics, and techniques, empowering participants to derive meaningful insights from complex data sets.

The session titled “Data Deluge and Public Policy: Promises and Perils” by Dr Soumyadip Chattopadhyay, Associate Professor of Economics, Vishwabarati Shantha Nikitin and also a Visiting Senior Fellow at IMPRI. His session will mainly deal with regression with time series analysis and forecasting. Regression analysis with qualitative variables, and categorical, dependent, and variable regression were undertaken in this session.

The session dealt with the pros and cons of the data deluge in public policymaking with a special reference to the Indian context.

Data and its Need

The importance of data in research and the amount of data generated and researched in real life is explained in this session. According to Sir, various factors within the public policy scenario such as economic, institutional, environmental, and technological factors are taken into consideration when policies are assessed or implemented. In this context, awareness of the importance of data and its analysis is considered to be very crucial.

Concepts such as digital footprint were explained which helped in understanding how the technological progress in data collection has taken place over the years. Limitations for the same were also mentioned.

Data Revolution

The concept of Data Revolution is explored which helped the student understand the new kinds of challenges that data analysis faces. It was established that the need for newer kinds of theories, methods, and tools are required. The HKMBT pipeline was taken up in brief. 

5 Frameworks of Data

The 5 Frameworks of data, Volume, Variety, Velocity, Veracity, and Value are explained in detail. A better understanding of the framework along with the interlinkages between the two are also provided.

Quality of Data

The importance of the quality of data in research along with an explanation of the characteristics of the quality (Accuracy, Completeness, Timeliness, Consistency, and Uniqueness) of data is also provided. The interlinkage between the characteristics is also provided.

Relation between Framework and Quality of Data with Public Policy

An understanding of how the 5 frameworks and the 5 characteristics of data are to be utilized in public policy analysis was explained. The combination of these topics gave rise to three different unique characteristics. They are:

-> Capacity to solve individual problems.

-> Capacity to improve the quantity and quality of life at a macro level

-> readiness for change.

Additional three implications that increase the importance of policy design and implementation were taught. They were:

-> Capacity to provide a personalized service.

-> Involvement of final users in the process of designing and producing policies.

-> Capacity to change the characteristics of the service over time.

The inference that Data can improve the quality of public policies acting on these characteristics is also conveyed to the students. This was followed by a Cost Benefit Framework of Data.

Cost Benefit Framework of Data

An explainer of the cost-benefit framework of data and why analysts are currently experiencing decreasing marginal costs to data is explained in detail. They were attributed to:

-> Falling prices of data storage

-> Increasing skills and resources to process data rapidly.

-> Increasing efficiency of data gathering.

-> Falling marginal cost of data

-> Nearly costless ways to disseminate data.

Various examples for each characteristic were also mentioned to better understand the concept.

Economies of Scale and Scope of Data

The topic of Economies of Scale and Scope of Data was then undertaken with various examples ranging from the agricultural to private markets.

Structure of Government Data

Then the session went on to explain the basic structure of data available in various government portals. This included Administrative, Survey, Transactions, and Institutional Data. Along with this, the main characteristics of the data present in the government portals are also explained in detail.

The concept of an Integrated Data System and its importance is explained in detail in the current scenario where public policy is of utmost importance to better the Indian Economy.

In the end, the Integrated Command and Control Centre (ICCC) in Smart Cities was explained in brief to understand the progress of data collection and monitoring in India. The process of how data from ICCC is generated, compiled, and used for policy-making from various stakeholders, sectors, and industries is also explained.

National Policy on Official Statistics (NPOS)

Another important aspect named the Revised National Policy on Official Statistics (NPOS) was also explained since it is relevant to the concept of Data Deluge. The NPOS is India’s strategic blueprint for a future informed by data. The four characteristics and drawbacks of the NPOS are also mentioned in brief. The concept of Digital India concerning NPOS is also explained. Its significant challenges, visions, and goals were also taken up.


The session focused on discussing the impact of data deluge on public policymaking in India and the various factors, characteristics, and progress that happened with the public policy data sector in India. The session also went into great detail about the importance of data analysis in public policy and the places where India is lacking with respect to data generation and compilation. The session ended with a mention of the coming challenges that were present within the public policy data world that are yet to be tackled.

Question and Answer Session After the session, Dr. Soumyadip Chattopadhyay took a short Q&A session to make sure that the students were not confused with any concepts that were taught in the session. Various questions paved the way for a healthy discussion between the instructor and the students. Students asked doubts which helped them understand the concept in a better way.

Vamsi Gokaraju is a Research Intern at IMPRI.

Disclaimer: All views expressed in the article belong solely to the author and not necessarily to the organization.

Read more session reports:

Participatory Action Research and Policy-Influenced Research Methodology

Indian Feminism: Legacy and Future

Posted by Vamsi Gokaraju, Research Intern at IMPRI.


  • IMPRI Desk

    IMPRI, a startup research think tank, is a platform for pro-active, independent, non-partisan and policy-based research. It contributes to debates and deliberations for action-based solutions to a host of strategic issues. IMPRI is committed to democracy, mobilization and community building.

    View all posts