Abstract

Use of alternative data sources and modern computatinal techniques are growing in importance and interest for the compilation of official statistics all around the world. In the area of price statistics, important achivements have been made in this regard and some NSOs have been implementing and increasing the use of alternative data sources for the production of official price statistics. However, use of alternative data sources poses challenges in different aspects to NSOs ranging from availability of access to these data sources to the development of human skills and IT infrastructure necessary to cope with the demands that the adoption of such data require.

In recent years the Brazilian Institute of Geography and Statistics (IBGE) has started to conduct initiatives on the use of alternative data sources and automatic web scraping techniques to improve the compilation of the official price statistics it provides. The use of web scraping for instance has shown useful in different contexts such as reduction of manual work, help to increase the representativity of a CPI basket and possibility of methodological improvements. However, a more intensive use of such web data and incorporation of other data sources such as scanner data or administrative records with rich information for price statistics still presents many challenges.

Based on our views on these issues and the experiences that we have faced so far, we aim to discuss the main challenges encountered for the use of alternative data sources for price statistics at IBGE, some of the results and approaches adopted, limitations and potential way forward.