Skip to content

debakarr/myanimelist-data-set-creator

Repository files navigation

myanimelist-data-set-creator

Collection of some simple python scripts to create https://myanimelist.net/ anime and user data set.

This is not maintained anymore. Bulk request is not encourage, hence discontinuing the Project.


Myanimelist Anime Dataset upto May 7 2018 [This may take some time to load]

For latest dataset click here [Constantly updating]


NOTE: This page contains lots of GIF. So it may take a lot of time to load. Please be patient.


How to use Anime Dataset Generator

This script can be used to download anime dataset from Myanimelist using an unofficial MyAnimeList REST API, Jikan.

Column metadata:

  • animeID: id of anime as in anime url https://myanimelist.net/anime/1
  • name: title of anime
  • premiered: premiered on. default format (season year)
  • genre: list of genre
  • type: type of anime (example TV, Movie etc)
  • episodes: number of episodes
  • studios: list of studio
  • source: source of anime (example original, manga, game etc)
  • scored: score of anime
  • scoredBy: number of member scored the anime
  • members: number of member added anime to their list

Using Heroku

  • Now you need to enable Google sheet API for your project.

NOTE: If the worker doesn't starts amnually, you can start it using the following command: heroku ps:scale worker=1

  • Final Product:


Using your own PC/Laptop

Syntax

python getAnime.py starting_index ending_index [output_file.csv]

Demo:


How to use User Daataset Generator

This script can be used to download user dataset from Myanimelist using an API, Kuristina.

Column metadata:

  • userID: MAL user ID
  • animeID: id of anime as in anime url https://myanimelist.net/anime/ID
  • score: score by the use for anime with id = animeID (if user haven't score the anime then this field is 0).

Syntax

python getUser.py UserList.txt [User.csv]

NOTE: Make sure you have a Userlist.txt file containing the name of the users. If you don't have that then use the scrapper(scrap from club or scrap from post).

How to create User List from forum post:

For this you need to get topic ID. Go to MAL -> Community -> Forums -> Select a forum

For example for the following forums links their respective ID are highlighted in bold below:

https://myanimelist.net/forum/?topicid=1699126 -> 1699126

https://myanimelist.net/forum/?topicid=1696289 -> 1696289

After getting the topic ID, you can use createUserListFromPost script.

Syntax:
python createUserListFromPost.py topicID [UserList.txt]

How to create User List from club:

For this you need to get club ID. Go to MAL -> Community -> Clubs -> Select a club

For example for the following clubs links their respective ID are highlighted in red below:

https://myanimelist.net/clubs.php?cid=72250 -> 72250

https://myanimelist.net/clubs.php?cid=32683 -> 32683

After getting the topic ID, you can use createUserListFromClub script.

Syntax:
python createUserListFromClub.py clubID [UserList.txt]

Demo:

Create User List from forum

Create User List from club

Get user dataset


TO DO LIST

  • Scrapping Locally ✔
  • Scrapping using Heroku ✔
  • Creating Heroku Deploy Button ⌛

About

Collection of some simple python scripts to create https://myanimelist.net/ anime and user data set.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages