Directory Contents

Please consider making a donation (https://pushshift.io/donations) if you download a lot of data. This helps offset the costs of my time collecting data and providing bandwidth to make these files available to the public. Thank you!


If you have any questions about the data formats of the files or any other questions, please feel free to contact me at jason@pushshift.io


In this directory, you will notice that some months have an .xz extension as well as a .bz2 extension. The data in both are the same, but the .xz files offer a higher compression ratio. If an .xz file is available for a specific month, I would recommend downloading that instead of the .bz2 file. Eventually all files will be available in the .xz compression type.


File sizes starting with the October 2017 data dumps are approximately 10-15% larger in size. The reason for this is due to the fact that Reddit started offering a new field for comment objects called "permalink". The values for this new field are the direct link to the comment itself.


Reddit banned the subreddit /r/incels in early November of 2017. This happened as I was re-ingesting data for the month of October, 2017. Since the data was no longer available via the Reddit API, I still had the data from my real-time ingest database. In the interest of research, I included these comments in the October 2017 dump. The comments from the real-time database will have a score of "null". This only affects a subset of /r/incels comments for the months of October and November 2017.

Filename Type Size (bytes) Date Modified
RC_2017-12-01.xz RC_2017-12-01.XZ File 271,859,516 Jan 7 2018 3:39 AM
RC_2017-12-02.xz RC_2017-12-02.XZ File 222,946,936 Jan 7 2018 3:39 AM
RC_2017-12-03.xz RC_2017-12-03.XZ File 236,128,976 Jan 7 2018 3:40 AM
RC_2017-12-04.xz RC_2017-12-04.XZ File 273,477,488 Jan 7 2018 3:40 AM
RC_2017-12-05.xz RC_2017-12-05.XZ File 282,465,016 Jan 7 2018 3:40 AM
RC_2017-12-06.xz RC_2017-12-06.XZ File 270,502,092 Jan 7 2018 3:40 AM
RC_2017-12-07.xz RC_2017-12-07.XZ File 270,774,780 Jan 7 2018 3:40 AM
RC_2017-12-08.xz RC_2017-12-08.XZ File 262,079,068 Jan 7 2018 3:40 AM
RC_2017-12-09.xz RC_2017-12-09.XZ File 217,819,608 Jan 7 2018 3:40 AM
RC_2017-12-10.xz RC_2017-12-10.XZ File 228,302,948 Jan 7 2018 3:40 AM
RC_2017-12-11.xz RC_2017-12-11.XZ File 265,658,424 Jan 7 2018 3:40 AM
RC_2017-12-12.xz RC_2017-12-12.XZ File 279,367,004 Jan 7 2018 3:40 AM
RC_2017-12-13.xz RC_2017-12-13.XZ File 281,164,868 Jan 7 2018 3:40 AM
RC_2017-12-14.xz RC_2017-12-14.XZ File 273,138,964 Jan 7 2018 3:40 AM
RC_2017-12-15.xz RC_2017-12-15.XZ File 264,836,488 Jan 7 2018 3:40 AM
RC_2017-12-16.xz RC_2017-12-16.XZ File 216,910,692 Jan 7 2018 3:40 AM
RC_2017-12-17.xz RC_2017-12-17.XZ File 224,910,456 Jan 7 2018 3:41 AM
RC_2017-12-18.xz RC_2017-12-18.XZ File 264,571,212 Jan 7 2018 3:41 AM
RC_2017-12-19.xz RC_2017-12-19.XZ File 269,985,316 Jan 7 2018 3:42 AM
RC_2017-12-20.xz RC_2017-12-20.XZ File 268,337,520 Jan 7 2018 3:42 AM
RC_2017-12-21.xz RC_2017-12-21.XZ File 265,264,424 Jan 7 2018 3:42 AM
RC_2017-12-22.xz RC_2017-12-22.XZ File 242,367,004 Jan 7 2018 3:42 AM
RC_2017-12-23.xz RC_2017-12-23.XZ File 206,332,352 Jan 7 2018 3:43 AM
RC_2017-12-24.xz RC_2017-12-24.XZ File 205,147,776 Jan 7 2018 3:43 AM
RC_2017-12-25.xz RC_2017-12-25.XZ File 182,420,832 Jan 7 2018 3:43 AM
RC_2017-12-26.xz RC_2017-12-26.XZ File 209,206,796 Jan 7 2018 3:43 AM
RC_2017-12-27.xz RC_2017-12-27.XZ File 238,340,672 Jan 7 2018 3:44 AM
RC_2017-12-28.xz RC_2017-12-28.XZ File 249,948,812 Jan 7 2018 3:44 AM
RC_2017-12-29.xz RC_2017-12-29.XZ File 248,044,424 Jan 7 2018 3:45 AM
RC_2017-12-30.xz RC_2017-12-30.XZ File 227,084,428 Jan 7 2018 3:45 AM
RC_2017-12-31.xz RC_2017-12-31.XZ File 224,159,628 Jan 7 2018 3:45 AM