Directory Contents

Please consider making a donation (https://pushshift.io/donations) if you download a lot of data. This helps offset the costs of my time collecting data and providing bandwidth to make these files available to the public. Thank you!


If you have any questions about the data formats of the files or any other questions, please feel free to contact me at jason@pushshift.io


In this directory, you will notice that some months have an .xz extension as well as a .bz2 extension. The data in both are the same, but the .xz files offer a higher compression ratio. If an .xz file is available for a specific month, I would recommend downloading that instead of the .bz2 file. Eventually all files will be available in the .xz compression type.


File sizes starting with the October 2017 data dumps are approximately 10-15% larger in size. The reason for this is due to the fact that Reddit started offering a new field for comment objects called "permalink". The values for this new field are the direct link to the comment itself.


Reddit banned the subreddit /r/incels in early November of 2017. This happened as I was re-ingesting data for the month of October, 2017. Since the data was no longer available via the Reddit API, I still had the data from my real-time ingest database. In the interest of research, I included these comments in the October 2017 dump. The comments from the real-time database will have a score of "null". This only affects a subset of /r/incels comments for the months of October and November 2017.

Filename Type Size (bytes) Date Modified
RC_2018-01-01.xz LZMA2 Compressed Reddit Comments (JSON objects) 201,491,588 Feb 12 2018 8:52 PM
RC_2018-01-02.xz LZMA2 Compressed Reddit Comments (JSON objects) 257,462,668 Feb 12 2018 8:52 PM
RC_2018-01-03.xz LZMA2 Compressed Reddit Comments (JSON objects) 272,128,832 Feb 12 2018 8:52 PM
RC_2018-01-04.xz LZMA2 Compressed Reddit Comments (JSON objects) 263,591,508 Feb 12 2018 8:53 PM
RC_2018-01-05.xz LZMA2 Compressed Reddit Comments (JSON objects) 276,650,320 Feb 12 2018 8:53 PM
RC_2018-01-06.xz LZMA2 Compressed Reddit Comments (JSON objects) 231,980,432 Feb 12 2018 8:53 PM
RC_2018-01-07.xz LZMA2 Compressed Reddit Comments (JSON objects) 238,647,200 Feb 12 2018 8:53 PM
RC_2018-01-08.xz LZMA2 Compressed Reddit Comments (JSON objects) 265,732,596 Feb 12 2018 8:53 PM
RC_2018-01-09.xz LZMA2 Compressed Reddit Comments (JSON objects) 282,587,992 Feb 12 2018 8:53 PM
RC_2018-01-10.xz LZMA2 Compressed Reddit Comments (JSON objects) 284,180,712 Feb 12 2018 8:53 PM
RC_2018-01-11.xz LZMA2 Compressed Reddit Comments (JSON objects) 283,236,364 Feb 12 2018 8:54 PM
RC_2018-01-12.xz LZMA2 Compressed Reddit Comments (JSON objects) 276,506,628 Feb 12 2018 8:54 PM
RC_2018-01-13.xz LZMA2 Compressed Reddit Comments (JSON objects) 232,455,056 Feb 12 2018 8:54 PM
RC_2018-01-14.xz LZMA2 Compressed Reddit Comments (JSON objects) 241,863,632 Feb 12 2018 8:54 PM
RC_2018-01-15.xz LZMA2 Compressed Reddit Comments (JSON objects) 270,389,580 Feb 12 2018 8:54 PM
RC_2018-01-16.xz LZMA2 Compressed Reddit Comments (JSON objects) 283,671,148 Feb 12 2018 8:54 PM
RC_2018-01-17.xz LZMA2 Compressed Reddit Comments (JSON objects) 285,712,836 Feb 12 2018 8:54 PM
RC_2018-01-18.xz LZMA2 Compressed Reddit Comments (JSON objects) 283,337,320 Feb 12 2018 8:55 PM
RC_2018-01-19.xz LZMA2 Compressed Reddit Comments (JSON objects) 277,897,516 Feb 12 2018 8:55 PM
RC_2018-01-20.xz LZMA2 Compressed Reddit Comments (JSON objects) 224,279,488 Feb 12 2018 7:30 PM
RC_2018-01-21.xz LZMA2 Compressed Reddit Comments (JSON objects) 242,182,372 Feb 12 2018 7:30 PM
RC_2018-01-22.xz LZMA2 Compressed Reddit Comments (JSON objects) 286,728,736 Feb 12 2018 7:20 PM
RC_2018-01-23.xz LZMA2 Compressed Reddit Comments (JSON objects) 290,508,688 Feb 12 2018 7:21 PM
RC_2018-01-24.xz LZMA2 Compressed Reddit Comments (JSON objects) 286,541,304 Feb 12 2018 7:26 PM
RC_2018-01-25.xz LZMA2 Compressed Reddit Comments (JSON objects) 289,371,304 Feb 12 2018 7:29 PM
RC_2018-01-26.xz LZMA2 Compressed Reddit Comments (JSON objects) 277,301,600 Feb 12 2018 7:31 PM
RC_2018-01-27.xz LZMA2 Compressed Reddit Comments (JSON objects) 229,186,324 Feb 12 2018 7:35 PM
RC_2018-01-28.xz LZMA2 Compressed Reddit Comments (JSON objects) 235,834,624 Feb 12 2018 7:38 PM
RC_2018-01-29.xz LZMA2 Compressed Reddit Comments (JSON objects) 280,791,856 Feb 12 2018 7:38 PM
RC_2018-01-30.xz LZMA2 Compressed Reddit Comments (JSON objects) 291,447,164 Feb 12 2018 8:55 PM
RC_2018-01-31.xz LZMA2 Compressed Reddit Comments (JSON objects) 291,719,444 Feb 12 2018 8:55 PM
RC_2018-02-01.xz LZMA2 Compressed Reddit Comments (JSON objects) 289,104,140 Mar 10 2018 11:28 AM
RC_2018-02-02.xz LZMA2 Compressed Reddit Comments (JSON objects) 285,506,500 Mar 10 2018 11:28 AM
RC_2018-02-03.xz LZMA2 Compressed Reddit Comments (JSON objects) 233,527,844 Mar 10 2018 11:28 AM
RC_2018-02-04.xz LZMA2 Compressed Reddit Comments (JSON objects) 240,203,632 Mar 10 2018 11:28 AM
RC_2018-02-05.xz LZMA2 Compressed Reddit Comments (JSON objects) 280,892,740 Mar 10 2018 11:29 AM
RC_2018-02-06.xz LZMA2 Compressed Reddit Comments (JSON objects) 289,659,524 Mar 10 2018 11:29 AM
RC_2018-02-07.xz LZMA2 Compressed Reddit Comments (JSON objects) 297,964,020 Mar 10 2018 11:29 AM
RC_2018-02-08.xz LZMA2 Compressed Reddit Comments (JSON objects) 296,955,308 Mar 10 2018 11:29 AM
RC_2018-02-09.xz LZMA2 Compressed Reddit Comments (JSON objects) 284,746,912 Mar 10 2018 11:29 AM
RC_2018-02-10.xz LZMA2 Compressed Reddit Comments (JSON objects) 241,526,116 Mar 10 2018 11:29 AM
RC_2018-02-11.xz LZMA2 Compressed Reddit Comments (JSON objects) 242,067,128 Mar 10 2018 11:29 AM
RC_2018-02-12.xz LZMA2 Compressed Reddit Comments (JSON objects) 282,219,184 Mar 10 2018 11:29 AM
RC_2018-02-13.xz LZMA2 Compressed Reddit Comments (JSON objects) 294,497,316 Mar 10 2018 11:29 AM
RC_2018-02-14.xz LZMA2 Compressed Reddit Comments (JSON objects) 290,163,976 Mar 10 2018 11:29 AM
RC_2018-02-15.xz LZMA2 Compressed Reddit Comments (JSON objects) 287,401,276 Mar 10 2018 11:29 AM
RC_2018-02-16.xz LZMA2 Compressed Reddit Comments (JSON objects) 290,034,184 Mar 10 2018 11:29 AM
RC_2018-02-17.xz LZMA2 Compressed Reddit Comments (JSON objects) 235,801,868 Mar 10 2018 11:29 AM
RC_2018-02-18.xz LZMA2 Compressed Reddit Comments (JSON objects) 258,003,820 Mar 10 2018 11:29 AM
RC_2018-02-19.xz LZMA2 Compressed Reddit Comments (JSON objects) 275,976,808 Mar 10 2018 11:29 AM
RC_2018-02-20.xz LZMA2 Compressed Reddit Comments (JSON objects) 294,847,144 Mar 10 2018 11:29 AM
RC_2018-02-21.xz LZMA2 Compressed Reddit Comments (JSON objects) 300,724,704 Mar 10 2018 11:30 AM
RC_2018-02-22.xz LZMA2 Compressed Reddit Comments (JSON objects) 300,666,232 Mar 10 2018 11:30 AM
RC_2018-02-23.xz LZMA2 Compressed Reddit Comments (JSON objects) 294,198,912 Mar 10 2018 11:30 AM
RC_2018-02-24.xz LZMA2 Compressed Reddit Comments (JSON objects) 237,922,356 Mar 10 2018 11:30 AM
RC_2018-02-25.xz LZMA2 Compressed Reddit Comments (JSON objects) 246,607,052 Mar 10 2018 11:30 AM
RC_2018-02-26.xz LZMA2 Compressed Reddit Comments (JSON objects) 289,734,660 Mar 10 2018 11:30 AM
RC_2018-02-27.xz LZMA2 Compressed Reddit Comments (JSON objects) 298,644,600 Mar 10 2018 11:30 AM
RC_2018-02-28.xz LZMA2 Compressed Reddit Comments (JSON objects) 301,483,312 Mar 10 2018 11:30 AM
par_recovery <Directory> File <Directory> Mar 17 2018 4:06 PM
sha256sum.txt Text File 4,898 Mar 11 2018 4:22 PM