Skip to content

Latest commit

 

History

History
62 lines (33 loc) · 3.43 KB

README.md

File metadata and controls

62 lines (33 loc) · 3.43 KB
Raw

W A R N I N G : CRUCIAL FEATURE DEPRECATED. NO LONGER WORKS.

Unfortunately, Reddit has removed the ability to scan for submissions by date range. This eliminates the usefulness of this program and I'm deeply sorry to anyone who enjoyed using it for their use. Please message Reddit's technical team so we can have it back! Click here for the official announcement: https://www.reddit.com/r/changelog/comments/7tus5f/update_to_search_api/

Thanks -2hands10fingers

Reddit Image Scraper 1.0 brought to you by pyStudyGroup

Credits: /u/2hands10fingers, /u/Joe_Anonimist, /u/iakovosbelonias, tjcim

Scrapes Reddit images from any public subreddit by a user-specified date range and saves them to your computer.

Introduction

Many tools scrape Reddit but don't give you the power to grab data on your terms. This changes things.

How it Works

Use Python 3 to scrape any .jpg or .png images in the available subreddit (you can always add or remove file extensions within the code). Simply run the program in terminal using $ python3 RedditImageScraper.py under the correct directory where RedditImageScraper.py is downloaded, enter your start and ending dates, then you will select any public subreddit of your choice, and PRESTO! you will begin downloading images.

Installation

1. Download

You can download this repository directly from GitHub and place the files where you want to run the program from. Remember, where the program is located, that is where your pictures will download to. You can change file_path in config.py to direct all downloads to that directory.

2. Install Dependencies

We make sure things are easy to install and test by using the requirements.txt file. In your terminal, type $ pip install -r requirements.txt and you should have everything you need to get going.

3. Obtaining Reddit Access

Oauth2 is required. Reddit doesn't like anonymous people taking their stuff, so you will need to copy the config.py.sample to config.py then open up the config.py file to add your own username, password, client_id, and client_secret to the file. Please follow Reddit's Oauth2 simple to follow guidelines to grab the client_id and client_secret here.

Testing

RIS now comes with testing abilities and error logging. We recommend doing this within python3's virtual environement.

Setupping up the virtual testing environment:

$ python3 -m venv env

$ source env/bin/activate

$ pip install -r requirements.txt

$ pytest

Notes

Download speeds may vary depending on the length of your selected date range, the Reddit server's current performance, or your current internet speed. You will only be able to select the years 2005 (when Reddit was launched) through the current year. If you want to download all of the images for the current day, simply enter the date twice (e.g., 3-14-2017 -- 3-14-2017).

Thank you and enjoy,

2hands10fingers