MIT: The Billion Prices Project

[CrackSmokeRepublican]

The Billion Prices Project

The Billion Prices Project is an academic initiative that collects prices from hundreds of online retailers around the world on a daily basis to conduct economic research. We currently monitor daily price fluctuations of ~5 million items sold by ~300 online retailers in more than 70 countries.

This webpage showcases examples of average inflation indexes that we created to illustrate the type of statistical work that can be done with this data. Our team is currently working on developing econometric models that leverage the data to forecast future trends and conduct economic research.

http://bpp.mit.edu/daily-price-indexes/

BPP Database Key Facts
-Statistics updated every day
-5 million individual items
-70 countries
-Started in October of 2007
-Supermarkets, electronics, apparel, furniture, real estate, and more

Our indexes are designed to provide real-time information on major inflation trends, not to forecast official inflation announcements. The official CPI series are shown for reference purposes only. You can hide/show each series by clicking on their legend icon.

http://bpp.mit.edu/daily-price-indexes/

Methodology

Data collection: our data is collected every day from online retailers using a software that scans the underlying code in public webpages and stores the relevant price information in a database. The resulting dataset contains daily prices on the full population of products sold by individual retailers, allowing us to observe every single price change that takes place. Our data includes information on product descriptions, package sizes, brands, special characteristics (e.g. "organic"), and whether the item is on sale or price control. For more details on the collection methodology, see Cavallo (2010) "Scraped Data and Sticky Prices".

Daily Online Price Index Computation: The daily online index is an average of individual price changes across multiple categories and retailers. The index uses a basket of goods that changes over time as products appear and disappear from a retailer's webpage. It is updated on a daily basis and leveraged to estimate annual and monthly inflation. This index is not designed to forecast official inflation announcements, but to provide real-time information on major inflation trends.

Monthly Inflation: The monthly inflation rate is the percentage change between the average of the daily online price index of the last 30 days and the average of the previous month. For example, on the last day of September 2010, we compared the average of the daily index between September 1st and September 30th to the average of the daily index between August 1st and August 31st. On the last day of each month, the value of our monthly inflation is equivalent to the monthly statistic reported by official offices.

Annual Inflation: The annual inflation rate is the percentage change between the average of the daily online price index of the last 30 days and the average for the same period a year ago. For example, on the last day of September 2010, we compare the average of the daily index between September 1st and September 30th 2010 to the average of the daily index between September 1st and September 30th 2009. On the last day of each month, the value of our annual inflation is equivalent to the annual (year-to-year) statistic reported by official offices.