Downloads

In addition to the HTTP Archive source code being open source, all of the data gathered is also available for download.

Instructions

The downloaded files were generated by mysqldump. The mysqldump files do not contain the commands to create the MySQL database and tables. To restore these mysqldumps:

  1. Import the schema dump to create the tables.
  2. Import the desired crawl dump using this command:
    gunzip -c MYSQLDUMP_FILE.gz | mysql -u MYSQL_USERNAME -pMYSQL_PASSWORD -h MYSQL_HOSTNAME MYSQL_DB
  3. If you want to run a private instance of the source code, you need to also import the stats and crawls dumps.

Files

These files define the schema and the meta-level tables:

There's a download file for each crawl for desktop ("IE") and mobile ("iPhone"):