Artstation web-parser that reposts images on Twitter* (not implement yet) and Tumblr
Blog link: https://art-py.tumblr.com/
Welcome! This is my first programming project ever. Meaning that this code is kinda like collection of sollutions made by others that can be found through StackOverflow. The inital idea was to use the Twitter API, but since the keys are now held by revisions I migrated to tumblr till I get my developer account approved.
The idea is to get images from Artstations with the project name, artist and link to their post. After getting the data, I use another code that makes the post to Tumblr.
It was actually a really good project to learn Python. I just new the bare minimum when I begin it and now I feel more confident with the language.
The logic behind it and a recipe to re-use the code:
1- Execute 'artstation-webcrawler.py' to scrape the website.
2- It needs a artstation link, I used the 'trending' link cause it has infinite scroll and it's always being updated.
3- Since Artstation is generated by AJAX, the artwork images is generated in JavaScript, so it needs to be read by the json.loads function.
4- It will create lists with the data gathered from the json list.
5- To be properly saved with a usable file name it will go through a little bit of Regex.
6- Then it goes to the loop that saves all the images by accessing the source link. The loop ads a number so it can be ordered by name after.
7- After the scraping, you can execute 'tumblr-API.py' to post the images.
8- You can easily find on the web how to get your API keys.
9- The bot reads all the db's .txt files that contains the necessary info to generate the post.
10- The loop that create the posts is set to ignore the .txt files.
Thanks for reading my little project. I worked on this through the course of 4 days and it was really useful to learn about lists, python libraries, variables and the biggest one: concatenation!
List of issues:
1- The bot only reads the first page. It would be nice if it was able to go through more pages automatically. ( FIXED 13/08/2018)
2- Fix the multiple database .txt by creating one huge list.
3- Make it post the images randomly.
4- Identify based only on the photo file what is the artist name and the link to the post.
5- Instead of using a sleep() to make the scraping slower so the files don't get corrupted buffer the downloaded ones somewhow.