BookCorpus dataset gone

[koga73]

The BookCorpus dataset located at http://www.cs.toronto.edu/~mbweb/ returns a 403.
Can you provide a mirror or at the very least give a few lines as an example of how the dataset needs to be formatted?

[naotokui]

this one? http://yknzhu.wixsite.com/mbweb

[MikeDrones]

What about using the google big query? Its freaking huge.

…

On Dec 6, 2016 2:51 AM, "Nao Tokui" ***@***.***> wrote: this one? http://yknzhu.wixsite.com/mbweb — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#17 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AVOF0KDf0NhIQkje0mAk63vyZoYWo2Jsks5rFTAngaJpZM4KzEZh> .

[eyaler]

gone again

[chiphuyen]

I went to the website and it said we'd have to collect it ourselves if we want the dataset. I wonder if anyone knows what happened?

[TZWwww]

I also wonder if I must collect the dataset myself. Is there any other source to get the dataset?

[ChenXi1992]

Also looking for the same dataset, anybody finds it?

[abhinavkashyap]

The sad thing is there are new papers that reference this dataset and reference the old paper for the corpus. The dataset has vanished from their website and there are no answers from any of the authors. I am searching and it is not available anywhere.

[soskek]

@TZWwww
I wrote a crawler for the original site, smashwords.com. Please use this.
https://github.com/soskek/bookcorpus

[ameet-1997]

Do you think it is possible to host the dataset for a while? I run into HTTP errors on some of the books, so this script ends up downloading only 7000/11000 books.

[soskek]

I cannot. Instead, see another option: soskek/bookcorpus#24 (comment)

[pushkarparanjpe]

Would be ideal, if it were via distributed P2P file sharing systems.