Using scanner data and multilateral index methods to produce CPIs: item aggregation, chain drift, seasonality and other unresolved issues
Category: International Statistical Institute
The Proposed session will be of considerable interest to many WSC attendees and especially to official statisticians from national statistical agencies across the world who are currently using or considering the use of electronic point-of-sales scanner data from retail outlets for computing their Consumer Price Index (CPIs). Price paid by consumers can now be observed at the point-of-sale across many products at high frequency, potentially allowing NSIs to, for example, to compile and publish reliable measures of monthly or even weekly measures of price change. However, there is has been no consensus on the most suitable method for calculating price indices with high frequency transactions data, such as scanner data. Scanner data have the potential for improving the timeliness of Consumer Price Index (CPI) releases and the ability to better capture changes in consumer expenditure patterns during pandemics and other crises.
Conventional fixed basket price indexes are ill-suited for this task as they are not able to capture rapid product turnover thus becoming quickly unrepresentative of consumer spending patterns, while chained indices often report excessive and unrealistic price changes, and suffer from ‘chain drift’.
Multilateral price index numbers, designed primarily for the purpose of spatial (regional as well as cross-country) price comparisons, have been advocated as a possible solution to the problem of chain drift and especially for treatment of scanner data. However, there are many multilateral index methods for NSIs to choose from and many choices to make when applying them, including sampling designs, product specifications, splicing methods to extend these indices when data from new time periods are added. Recently, Eurostat published a guide, “Multilateral Methods in the Harmonised Index of Consumer Prices” to support countries in understanding and implementing multilateral methods in the context of the HICP (Eurostat, 2020).
Given the current state of developments with respect to data availability and choice of index number methods, there is an urgent need to examine the issues involved and provide advice and recommendations to national statistical offices. The proposed session brings together experts working in this field with the aim of evaluating alternative methods to handle high-frequency price data and provide recommendations to practitioners. Outstanding questions like those illustrated above need to be resolved if national statistical agencies are to take maximum advantage of the data revolution.