R18.com is closing down

[ghost]

This is not an issue yet, but will be as Javinizer scrapes some of it's data from there.

R18.com is going to close down in January 31st of 2023 so might be good idea to start thinking alternatives for it

[madmax2]

how about storing all the data of r18 on another server or database?

[ghost]

how about storing all the data of r18 on another server or database?

That's not really an ideal thing because someone would have to take the burden of hosting said server or database and those aren't free, and there is plenty of alternatives that do exist like Javlibrary.
The point of this issue was to just point out that there will soon be deprecated code that can be removed entirely

[ghost]

this will also break the actor image scraping, because it uses images from r18's site. All of the images link to r18 even if you don't scrape r18. Unless there's a way to set it to scrape actor images from dmm instead that I'm missing.

I'm not 100% about Javinizer having that option but config doesn't seem to have an option for it.. But if DMM does have the actor images, should be fairly simple fix.
Although this feels like an abandoned project because there has been no updates or any replies to issues from jvlame for almost a year now so might be better to start look into alternative scripts or hope someone forks this and starts to maintain their own version of it

[endofthescope]

this will also break the actor image scraping, because it uses images from r18's site. All of the images link to r18 even if you don't scrape r18. Unless there's a way to set it to scrape actor images from dmm instead that I'm missing.

I'm not 100% about Javinizer having that option but config doesn't seem to have an option for it.. But if DMM does have the actor images, should be fairly simple fix. Although this feels like an abandoned project because there has been no updates or any replies to issues from jvlame for almost a year now so might be better to start look into alternative scripts or hope someone forks this and starts to maintain their own version of it

I deleted my previous comment because I found an option for it in the GUI, "scraper.option.dmm.scrapeactress" (haven't tested but assumed this does the job). Seems we'll be fine, but anyone who has .nfo files that link to r18 URLs for actor images will have to change those manually, assuming they don't have it set to use local actor images. My plex setup is gonna be a real pain to redo in a couple months but at least it's still possible if I rescrape everything.

The fix for that should also be simple, you just need to change r18.com to dmm.co.jp in the URL, it has the same format otherwise.

edit: I tested it and with that option on, it's still linking to r18 images and downloading images from R18. So I'm lost.

[madmax2]

That's not really an ideal thing because someone would have to take the burden of hosting said server or database and those aren't free, and there is plenty of alternatives that do exist like Javlibrary.
The point of this issue was to just point out that there will soon be deprecated code that can be removed entirely

Yeah javlibrary can do but it has no screens which R18 has plenty (as well as trailers).
This gonna be very hard to scrape for screens and trailers once r18 is gone..

it is not abandon but the coder just not updating.
check the discord to see what is the status

[ghost]

That's not really an ideal thing because someone would have to take the burden of hosting said server or database and those aren't free, and there is plenty of alternatives that do exist like Javlibrary.
The point of this issue was to just point out that there will soon be deprecated code that can be removed entirely

Yeah javlibrary can do but it has no screens which R18 has plenty (as well as trailers). This gonna be very hard to scrape for screens and trailers once r18 is gone..

it is not abandon but the coder just not updating. check the discord to see what is the status

About those screenshots, those still exist on DMM so scraping them is possible. Think the setting "sort.metadata.priority.screenshoturl" should be able to prioritize DMM instead of R18 if you just change it.

[madmax2]

|About those screenshots, those still exist on DMM so scraping them is possible. Think the setting "sort.metadata.priority.screenshoturl" should be able to prioritize DMM instead of R18 if you just change it.

oic, let me test that.
what about trailers, we redirect to somewhere else like javtrailer.com

do you know how I can put the metadata of the and and actor with the local storage location instead of the url of r18 in the NFO file?

Can you still scrape plot and other metadata from r18 right now?
I am getting empty tags ( , etc) not sure it is my setup or r18 is just not working anymore.

[ghost]

|About those screenshots, those still exist on DMM so scraping them is possible. Think the setting "sort.metadata.priority.screenshoturl" should be able to prioritize DMM instead of R18 if you just change it.

oic, let me test that. what about trailers, we redirect to somewhere else like javtrailer.com

do you know how I can put the metadata of the and and actor with the local storage location instead of the url of r18 in the NFO file?

Can you still scrape plot and other metadata from r18 right now? I am getting empty tags, not sure it is my setup or r18 is just no working anymore.

I don't think there is support for javtrailer, but you can switch to DMM if you change the option for "sort.metadata.priority.trailerurl".

There is lot of options for the actress thing, even switching over to DMM with "scraper.option.dmm.scrapeactress" and if you want to locally save actress images, ""sort.download.actressimg"

I scraped yesterday and plot etc were working fine so unless R18 suddenly changed something within last 24 hours more likely reason is that R18 didn't include plot or some other metadata.. But if you don't even get things like ID and genres then there is something wrong with your setup.

Also if you really want plot, you could try switching scraping to DMM and then enabling the translation options

[madmax2]

There is lot of options for the actress thing, even switching over to DMM with "scraper.option.dmm.scrapeactress" and if you want to locally save actress images, ""sort.download.actressimg"

it is already set to TRUE and download it to the folder
what I meant was points to r18 url e.g https://pics.r18.com/mono/actjpgs/mino_suzume.jpg
I want it to point the local storage = .actors folder within the folder that contains the movie file.

will this be an issue when r18 shutdown and the url will be dead?
does that mean in a media software e.g. kodi, the actor image will be now blank
even though there is local downloaded copy?

I scraped yesterday and plot etc were working fine so unless R18 suddenly changed something within last 24 hours more likely reason is that R18 didn't include plot or some other metadata.. But if you don't even get things like ID and genres then there is something wrong with your setup.

can you test now or when you are free?
I should clarify, I rescrape a movie to test
and the original nfo had plot, originaltitle, vote tag but now it is empty.

also trailers is no longer being downloaded (whereas before it did for that same movie), not sure why this is occurring.

current setting...
"sort.download.trailervid": true,
"sort.metadata.priority.trailerurl": [
"r18",
"dmmja"
],

all the japanese text is now not added to the nfo whereas it was in the old NFO.

[ghost]

will this be an issue when r18 shutdown and the url will be dead? does that mean in a media software e.g. kodi, the actor image will be now blank even though there is local downloaded copy?

Don't know, never used Kodi. You might just need to update the NFO to point to correct location for Kodi to find it

can you test now or when you are free? I should clarify, I rescrape a movie to test and the original nfo had plot, originaltitle, vote tag but now it is empty.

I just did, and still getting the plot, originaltitle and everything. And if there is no plot in english, it pulls it in japanese for me. I have not changed any settings at all for reference. You could have typo somewhere in your config file that's breaking some of the things so try resetting it maybe? I had to do that in past because of a typo which I couldn't find

also trailers is no longer being downloaded (whereas before it did for that same movie), not sure why this is occurring.

Well, My guess would be that because you can't buy anything from R18 anymore, only download and view your already purchased ones , they probably removed access to trailers too

[madmax2]

Don't know, never used Kodi. You might just need to update the NFO to point to correct location for Kodi to find it

In the nfo actor thumb tag e.g.
<thumb>https://pics.r18.com/mono/actjpgs/yosikawa_aimi.jpg </thumb>

can we get javinizer put the local image location instead?

e.g.

<thumb>./.actors/yosikawa_aimi.jpg</thumb>

I just did, and still getting the plot, originaltitle and everything. And if there is no plot in english, it pulls it in japanese for me. I have not changed any settings at all for reference. You could have typo somewhere in your config file that's breaking some of the things so try resetting it maybe? I had to do that in past because of a typo which I couldn't find

thanks for doing the test
yeah i tested with some other files and it is scraping the nfo properly (ie no blank tags) + downloaded trailer
so I guess that the data that is on R18 can no longer be scraped and if it is not on DMM or the other sites then it results in blank tags.

Seems like R18 has the most movies + trailers metadata .. so will be great loss

does https://javtrailers.com/ need api to download with javinizer if we can modify the script to add that site?

[shrublet]

I just did, and still getting the plot, originaltitle and everything. And if there is no plot in english, it pulls it in japanese for me. I have not changed any settings at all for reference. You could have typo somewhere in your config file that's breaking some of the things so try resetting it maybe? I had to do that in past because of a typo which I couldn't find

Just came back to this project after a fresh install, and it seems like R18 data is not getting pulled correctly and will fail to get the URL (relies on searching or URL which gets redirected now). This also causes DMM scraping to fail since it relies on the Get-R18Url module. For anybody that wants a quick fix before R18 fully shuts down, you can change a couple lines there to get it to work.

Add $apiURL = "https://www.r18.com/api/v4f/contents/$($contentId)?lang=en" on this line
Change this line to $webRequest = (Invoke-WebRequest -Uri $apiURL -Method Get -Verbose:$false).Content | ConvertFrom-Json

This just uses the api instead to verify the data instead which is functional as of now.

[madmax2]

@shrublet do you have the experience in coding to change the r18 to r18.dev as a source
r18.dev is a site created by some that has fully backup r18 data.

note : if you go to R18.dev then it goes some status update page of r18.dev

So you need to go some url like this first
https://r18.dev/videos/vod/movies/detail/-/id=ure00013/

[shrublet]

@shrublet do you have the experience in coding to change the r18 to r18.dev as a source r18.dev is a site created by some that has fully backup r18 data.

note : if you go to R18.dev then it goes some status update page of r18.dev

So you need to go some url like this first https://r18.dev/videos/vod/movies/detail/-/id=ure00013/

Didn't know about this, but very relieving to hear somebody has been backing up data from R18. I think changing to R18.dev would mostly just be syntax changes, but I haven't totally looked at all the files. I'm more of a hobbyist coder, but if nobody else wants to, I'll look into creating a PR probably at a later date once R18 is down in earnest.

[ghost]

@shrublet do you have the experience in coding to change the r18 to r18.dev as a source r18.dev is a site created by some that has fully backup r18 data.
note : if you go to R18.dev then it goes some status update page of r18.dev
So you need to go some url like this first https://r18.dev/videos/vod/movies/detail/-/id=ure00013/

Didn't know about this, but very relieving to hear somebody has been backing up data from R18. I think changing to R18.dev would mostly just be syntax changes, but I haven't totally looked at all the files. I'm more of a hobbyist coder, but if nobody else wants to, I'll look into creating a PR probably at a later date once R18 is down in earnest.

It will probably require bit more code changes than just changing the urls, not sure how the scraping is done in Javinizer.. But that R18.dev does seem to have JSON ( https://r18.dev/videos/vod/movies/detail/-/combined=ure00013/json ) which should make it pretty easy and quick to change the code to work with it.
Although it is relevantly new service, I do hope it sticks around

[madmax2]

@shrublet
I modify the files you said, and it is able to download the data + extrafanart
but I encounter a bug whenever it tries to download any trailers

Do have solution for the error?

BUG

Get-RunspaceData: H:\Cloud Storage\OneDrive\Documents\PowerShell\Modules\Javinizer\2.5.10\Public\Invoke-JVParallel.ps1:561
Line |
561 | Get-RunspaceData
| ~~~~~~~~~~~~~~~~
| Exception calling "EndInvoke" with "1" argument(s): "The running command stopped because the preference variable "ErrorActionPreference" or common parameter is set to Stop: [MIDE-301] [Set-JVMovie] Error occurred when
| creating trailer video file [https://awscc3001.r18.com/litevideo/freepv/m/mid/mide00301/mide00301_dmb_w.mp4] to [MIDE-301-trailer]: Exception calling "DownloadFile" with "2" argument(s): "No such host is known.
| (awscc3001.r18.com:443)""

[shrublet]

@shrublet I modify the files you said, and it is able to download the data + extrafanart but I encounter a bug whenever it tries to download any trailers

Do have solution for the error?

BUG

Get-RunspaceData: H:\Cloud Storage\OneDrive\Documents\PowerShell\Modules\Javinizer\2.5.10\Public\Invoke-JVParallel.ps1:561 Line | 561 | Get-RunspaceData | ~~~~~~~~~~~~~~~~ | Exception calling "EndInvoke" with "1" argument(s): "The running command stopped because the preference variable "ErrorActionPreference" or common parameter is set to Stop: [MIDE-301] [Set-JVMovie] Error occurred when | creating trailer video file [https://awscc3001.r18.com/litevideo/freepv/m/mid/mide00301/mide00301_dmb_w.mp4] to [MIDE-301-trailer]: Exception calling "DownloadFile" with "2" argument(s): "No such host is known. | (awscc3001.r18.com:443)""

They're no longer available as they stopped hosting them already. Just use DMM as your trailer source since they're the same (if you change the trailer url to cc3001.dmm.co.jp instead you'll see it works). You should be able to download them without having to change your location (scraping other info from DMM i found only worked while connected to VPN).

[madmax2]

@shrublet how do you change the trailer url to cc3001.dmm.co.jp

it would be good if there was a solution to add https://javtrailers.com/ as a source
since they probably have all/most of the trailers.

DMM I find does not have as many titles and therefore less trailers when compared to R18.

[shrublet]

@shrublet how do you change the trailer url to cc3001.dmm.co.jp

If you change the base url of an r18 trailer (in your case, https://awscc3001.r18.com/litevideo/freepv/m/mid/mide00301/mide00301_dmb_w.mp4 -> https://cc3001.dmm.co.jp/litevideo/freepv/m/mid/mide00301/mide00301_dmb_w.mp4), you'll see they use the same naming scheme.

DMM I find does not have as many titles and therefore less trailers when compared to R18.

DMM should have everything R18 had (and more) as I'm pretty sure DMM owned R18 and everything on R18 should have a DMM equivalent.

it would be good if there was a solution to add https://javtrailers.com/ as a source
since they probably have all/most of the trailers.

Somebody might be interested, but I think DMM suffices as a stand-in for anything from R18/DMM.

[ldjb]

Hi everyone, I'm the developer of R18.dev. It's nice to see some interest here.
R18.dev is intended to be not just a website, but also a data source with public APIs that are open for anyone to use. Javinizer is therefore more than welcome to make use of the data.

However, the service is currently still undergoing development and is only in an alpha stage. As such, there are no fully stable APIs. URLs and data formats will change. I hope to be able to provide a more stable API in the coming months. This will be announced on the project Mastodon profile as well as on the site itself. However, until then, I would advise against integrating with R18.dev because it is highly likely to break.

[gummykage]

Hi everyone, I'm the developer of R18.dev. It's nice to see some interest here. R18.dev is intended to be not just a website, but also a data source with public APIs that are open for anyone to use. Javinizer is therefore more than welcome to make use of the data.

However, the service is currently still undergoing development and is only in an alpha stage. As such, there are no fully stable APIs. URLs and data formats will change. I hope to be able to provide a more stable API in the coming months. This will be announced on the project Mastodon profile as well as on the site itself. However, until then, I would advise against integrating with R18.dev because it is highly likely to break.

Thank you so much for developing and maintaining this... This is going to matter so much over time.

[seeyabye]

Thanks @ldjb for keeping R18 alive!

We have move forward with merging R18.dev and supporting it. If there are any changes in the future, we will make changes accordingly.

Closing this for now.

[madmax2]

@seeyabye so is the javinizer updated to work with r18.dev now?

do i use the update command to get the latest version with r18.dev support?

Javinizer -UpdateModule

[seeyabye]

You can install a fresh copy of Javinizer with Install-Module Javinizer or use the -UpdateModule as you have stated. It should work starting from version 2.5.12. You can confirm that after with Javinizer -v

[madmax2]

@seeyabye
thanks.. updated to latest version
Version : 2.5.14

OK, will test it.

Anyone known issues or scraping all the data works like before?