Potential Role of Big Data in Economic Policy
Over the past two decades, there has been a significant proliferation of big data, leading to the emergence of new challenges and opportunities in economic policy formulation. The use of big data, with its three defining characteristics (volume, velocity, and variety), poses questions about the future relevance of traditional data collection methods by National Statistical Organizations. The United Nations has recognized the importance of big data in official statistics, leading to the establishment of the UN Committee of Experts on Big Data and Data Science. The emphasis is now shifting towards practical use cases of big data to address societal challenges and transform abundant data into meaningful insights for policymaking.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
Presentation Transcript
Potential Role of Big Data in Economic Policy 6thMarch, NIBAF Islamabad By Dr Naeem uz Zafar
Proliferation of Big Data Last two decades have seen exponential growth in data generation. The rate at which data has been produced has been so fast, its size so big, and its nature so diverse that this data was named as big data using the 3 Vs to define its nature. One major reason for using 3 Vs was not to let it get mixed up with conventional survey and census data collected by National Statistical Organizations (NSO) or the administrative data at several levels. It was thought, and often discussed, that big data will soon threaten the existence of NSOs by developing information similar to one created by surveys and census of NSO in very cost effective ways and in almost real-time.
United Nation Statistics Division created the UN Committee of Experts on Big Data and Data Science for Official Statistics (UNCEBD) in 2014 to provide guidance on how to use big data to create indicators and statistics in more efficient and effective ways. The UNCEBD recently met in side meetings of 55thannual meeting of UNSD and reviewed 10 years of use of big data and data science.
Ten years back the original mandate included only Big Data and not Data Science and Alternate Data and there was not emphasis on Use Cases . As can be seen in the meeting that took place in Feb 2024, conducting use cases has become the priority for UNCEBD. Before diving into what use cases means here, let s consider one difference between big data and conventional data which will put this discussion in better context.
Unlike conventional data, Big Data is not structured . At times, it was thought that big data is proliferating so rapidly that it will create data, statistics, and information, hitherto created by conventional surveys on all conceivable socioeconomic aspects. In this situation NSO won t be able to put structure on big data as NSOs, alongwith their data, would be dwarfed by Big Data. Naturally, this line of thinking will be a doomsayer for NSOs, if structure is not needed, surveys and census lose to big data, and so what s the point? Will big data be the only go-to data source?
Lets see other insights from 10 years review of big data, these are based on responses from 93 NSOs on big data. Most of them responded that focus has not shifted to societal challenges-the challenge lies in transforming abundant data into useful societal information . They also responded that The term Big Data has become normalized, shifting from a revolutionary concept to just another data source This clearly shows that NSOs have started putting big data into structures they already had, there might have been some adjustments but there was no disruption.
It seems that NSOs have handled the big data and this February they talked about tasking themselves with more Use Cases thus putting more Structure on big data. Increasing the Use Cases or establishing more Structure is still evolving. There can be many ways to put structure. It can be growth and equity It can be political, social, and economic
One way to think about establishing structure can be more General Equilibrium Model, something economists are wedded to in a Roman Catholic way. In general equilibrium model households, firms, and government are linked through transactions of goods, services, and taxes. Households need goods and services for which they make labor leisure trade-offs The labor supplied by households is used by firms to produce goods and services for private and public consumption, and The government taxes households and firms to provide public goods.
Given that their decisions on labor supply, labor demand, and taxes are optimal, the information flow between these entities must also be optimal. Let s now talk specific, Use Cases . I will talk about some insights and possibilities highlighted from digital census recently concluded by PBS,
Glimpses of Census 2023 1070 Training Venue 50TB Databas e 4000 Trainers 220,000 Maps 126000 Tablets 40 38.3 million Househo lds www.pbs.gov.pk 241.4 million Population 250,000 Operatio nal Staff 8 million Economic unit million Structur es Pakistan Bureau of Statistics
Glimpses of Census 2023 20k 70k 146k Hotels 611k Mosque / Worship 4886 Police Stations 18k Banks 7510 Hostels Madrassa s Hospitals 190k Product Shops 268k Schools 9442 Colleges 2435 Universiti es 3.1 345k Wholesal e Shops 1.5 million Retail Shops million Service Shops 194k 20k Semi- public organizati 49k Public organizati ons Factories/ Industries Pakistan Bureau of Statistics www.pbs.gov.pk
Lets talk about retailers of commodities, say pulses, wheat flour, sugar, and rice. Now there must be buying function by big commodity buyers in grain markets of rural towns The commodities must be transported to storage for further distribution From storage, the commodities are either supplied to retailers or to wholesalers.
The steps in this supply chain involve costs at every step. If someone is doing business of supplying specific commodity, say, rice to given market(s) then business would like to know the demand of rice in the market. The distributor, wholesaler, and retailer, if doing optimal business, must be using best information flow to efficiently gauge demand.
Now, suppose PBS can do a survey to estimate rice sales at granular level across Pakistan The supply chain of rice finds that this information can help them in reducing ordering, logistics, inventory carrying, and distribution costs and this information is used to create surplus. Then questions arises, how quickly and frequently this information can be created? Something which can be answered by probing into POS data from medium and big retailers or using scanner/tablets to plot this data on digital map of retail outlets and aggregating the demand at lower level.
Two points are becoming obvious from here, scanner POS data definitely have virtues, but these were not obvious until we imposed on them the cost structure of rice s supply chain. This integration of conventional and big data become more meaningful than either could be separately. As imposition of structure made cost saving obvious, there was a natural demand from system to create this information. Sequel to this is what we saw in redefined mandate of UNCEBD in which they also recommended integration of geospatial data and statistics
Lets consider another Use Case In the same census PBS has geotagged all structures, residential and economic. Now suppose for given housing density in comparable towns, there is some density of retail stores and by the same logic, there is some density of banks or ATMs. Now if this ratio differs in comparable towns then we can see a possibility of new investment.
Every land parcel in the country is declared urban or rural in nature Sometimes this classification matches with the amenities and socioeconomic characteristics of the area and sometimes, the mismatch is glaring. It is not unusual to see decent numbers of high rises in an urban outskirt which is declared rural in revenue papers, usually for tax evasion. A simple statistic can start this debate of mismatch and raise implications for state revenues.
PBS has also geotagged the businesses related with agriculture inputs and outputs. PBS also has crop production data. Now, suppose PBS can add a layer of data comprising of soil and weather conditions in representative areas from Pakistan, then, can this information be sold to relevant stakeholders? It might have looked like a stretch of imagination but an infopreneur of Islamabad has shared this information in text messages.
He entered into JV with telcos and started selling the information to farmers. If we go back in history of Pakistan, the same function was done by Extension Service quite effectively few decades back. Can NSO team up with Telcos, met office, agriculture departments of provinces and do such ventures?
The use cases we have been discussing so far were highlighted by the UNCEBD in 2024, but previously in 2022, they only went so far as to describe Information need and use, without emphasis on what a Use Case was. The transformation in a short two years clearly shows that the UN formalized its understanding of making big data more usable for stakeholders by incorporating structure.
In 2006, Clive Humby, the Mathematician and data entrepreneur from U.K. coined the phrase Data is the new oil . Fine, let s acknowledge that Data is the new oil and as such, it should have flown across entities The flow would have moved data to information, information to decision, decision to knowledge, and knowledge to wisdom but this requires entities to be seamlessly aligned. But there are very few countries where entities are not defensive, even fewer, where entities are seamless.
If data flow is so good, if it is oil, then why is it not flowing? This is simply because we have not thought carefully about Use Cases , which I prefer to call juicy stories. Just take another example, suppose PBS can enter into data sharing arrangements with Telcos and get the mobility of cell phones from southern cities of country to northern part, say Upper Swat. Once the travellers are shown settled in Swat for few days by Telco, PBS can use this information and multiply number of days stayed with hoteling, traveling, and eating expenditure to develop estimates of tourism GDP. The data can be processed and information can be shared with provincial administrations and service providers for better decisions
Examples are endless but the questions still haunts, if all of this were so good then why ot has not started? Simple answer is that it take analyzing one s own data Thinking about possibilities and setting priorities Engage stakeholders and rope them in by showing use Cases Identifying low hanging possibilities and creating solutions thus feeling confident
Alongside, thinking about more rewarding yet difficult insights, analyzing the gaps, thinking about reducing these gaps by integrating with telco, gis, or whatever alternate data. There are tremendous possibilities but it is really upto us Would we continue burying new data in graveyards as we have been or we would start telling stories and Use Cases
