Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aryion/eka broken #2900

Closed
left1000 opened this issue Sep 7, 2022 · 10 comments
Closed

aryion/eka broken #2900

left1000 opened this issue Sep 7, 2022 · 10 comments

Comments

@left1000
Copy link

left1000 commented Sep 7, 2022

HttpError: '503 Service Temporarily Unavailable' for 'https://aryion.com/forum/ucp.php?mode=login'

upon following that url the url works fine and the service is totally available.

-vv

[gallery-dl][debug] Starting DownloadJob for 'https://aryion.com/g4/gallery/USERNAMEOFARTIST'
[aryion][debug] Using AryionGalleryExtractor for 'https://aryion.com/g4/gallery/USERNAMEOFARTIST'
[aryion][info] Logging in as namad
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): aryion.com:443
[urllib3.connectionpool][debug] https://aryion.com:443 "POST /forum/ucp.php?mode=login HTTP/1.1" 503 None
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/forum/ucp.php?mode=login' (1/5)
[urllib3.connectionpool][debug] Resetting dropped connection: aryion.com
[urllib3.connectionpool][debug] https://aryion.com:443 "POST /forum/ucp.php?mode=login HTTP/1.1" 503 None
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/forum/ucp.php?mode=login' (2/5)
[urllib3.connectionpool][debug] Resetting dropped connection: aryion.com
[urllib3.connectionpool][debug] https://aryion.com:443 "POST /forum/ucp.php?mode=login HTTP/1.1" 503 None
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/forum/ucp.php?mode=login' (3/5)
[urllib3.connectionpool][debug] Resetting dropped connection: aryion.com
[urllib3.connectionpool][debug] https://aryion.com:443 "POST /forum/ucp.php?mode=login HTTP/1.1" 503 None
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/forum/ucp.php?mode=login' (4/5)
[urllib3.connectionpool][debug] Resetting dropped connection: aryion.com
[urllib3.connectionpool][debug] https://aryion.com:443 "POST /forum/ucp.php?mode=login HTTP/1.1" 503 None
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/forum/ucp.php?mode=login' (5/5)
[aryion][error] HttpError: '503 Service Temporarily Unavailable' for 'https://aryion.com/forum/ucp.php?mode=login'

@mikf
Copy link
Owner

mikf commented Sep 8, 2022

Still working on my end. Maybe it really was a temporary server error?
If this keeps happening for you, post the output of --write-pages and/or try using your browser cookies.

$ gallery-dl -v https://aryion.com/g4/view/510079
[gallery-dl][debug] Version 1.23.1-dev - Git HEAD: 2ed58029
[gallery-dl][debug] Python 3.10.5 - Linux-5.18.16-arch1-1-x86_64-with-glibc2.36
[gallery-dl][debug] requests 2.28.1 - urllib3 1.26.12
[gallery-dl][debug] Starting DownloadJob for 'https://aryion.com/g4/view/510079'
[aryion][debug] Using AryionPostExtractor for 'https://aryion.com/g4/view/510079'
[aryion][info] Logging in as ***
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): aryion.com:443
[urllib3.connectionpool][debug] https://aryion.com:443 "POST /forum/ucp.php?mode=login HTTP/1.1" 200 None
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=510079 HTTP/1.1" 200 0
[urllib3.connectionpool][debug] https://aryion.com:443 "GET /g4/view/510079 HTTP/1.1" 200 None
/tmp/aryion/jameshoward/510079 I'm on subscribestar now too!.jpg

@left1000
Copy link
Author

left1000 commented Sep 8, 2022

I tried using cookies. I'm not sure if my syntax is correct for --write-pages though.

.\gallery-dl -vv --write-pages https://aryion.com/g4/gallery/USERNAME
(with cookies enabled in conf file)

gives all the same errors.

I used your exact command (with cookies enabled from conf file):
.\gallery-dl -v --write-pages https://aryion.com/g4/view/510079

got output

D:\gallery-dl> .\gallery-dl -v --write-pages https://aryion.com/g4/view/510079
[gallery-dl][debug] Version 1.23.0 - Executable
[gallery-dl][debug] Python 3.7.9 - Windows-10-10.0.19041
[gallery-dl][debug] requests 2.28.1 - urllib3 1.26.12
[gallery-dl][debug] Starting DownloadJob for 'https://aryion.com/g4/view/510079'
[aryion][debug] Using AryionPostExtractor for 'https://aryion.com/g4/view/510079'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): aryion.com:443
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=510079 HTTP/1.1" 503 0
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/g4/data.php?id=510079' (1/5)
[urllib3.connectionpool][debug] Resetting dropped connection: aryion.com
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=510079 HTTP/1.1" 503 0
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/g4/data.php?id=510079' (2/5)
[urllib3.connectionpool][debug] Resetting dropped connection: aryion.com
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=510079 HTTP/1.1" 503 0
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/g4/data.php?id=510079' (3/5)
[urllib3.connectionpool][debug] Resetting dropped connection: aryion.com
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=510079 HTTP/1.1" 503 0
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/g4/data.php?id=510079' (4/5)
[urllib3.connectionpool][debug] Resetting dropped connection: aryion.com
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=510079 HTTP/1.1" 503 0
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/g4/data.php?id=510079' (5/5)
[aryion][error] HttpError: '503 Service Temporarily Unavailable' for 'https://aryion.com/g4/data.php?id=510079'

then same command as above with cookies not enabled in conf file:

PS D:\gallery-dl> .\gallery-dl -v --write-pages https://aryion.com/g4/view/510079
[gallery-dl][debug] Version 1.23.0 - Executable
[gallery-dl][debug] Python 3.7.9 - Windows-10-10.0.19041
[gallery-dl][debug] requests 2.28.1 - urllib3 1.26.12
[gallery-dl][debug] Starting DownloadJob for 'https://aryion.com/g4/view/510079'
[aryion][debug] Using AryionPostExtractor for 'https://aryion.com/g4/view/510079'
[aryion][info] Logging in as namad
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): aryion.com:443
[urllib3.connectionpool][debug] https://aryion.com:443 "POST /forum/ucp.php?mode=login HTTP/1.1" 503 None
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/forum/ucp.php?mode=login' (1/5)
[urllib3.connectionpool][debug] Resetting dropped connection: aryion.com
[urllib3.connectionpool][debug] https://aryion.com:443 "POST /forum/ucp.php?mode=login HTTP/1.1" 503 None
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/forum/ucp.php?mode=login' (2/5)
[urllib3.connectionpool][debug] Resetting dropped connection: aryion.com
[urllib3.connectionpool][debug] https://aryion.com:443 "POST /forum/ucp.php?mode=login HTTP/1.1" 503 None
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/forum/ucp.php?mode=login' (3/5)
[urllib3.connectionpool][debug] Resetting dropped connection: aryion.com
[urllib3.connectionpool][debug] https://aryion.com:443 "POST /forum/ucp.php?mode=login HTTP/1.1" 503 None
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/forum/ucp.php?mode=login' (4/5)
[urllib3.connectionpool][debug] Resetting dropped connection: aryion.com
[urllib3.connectionpool][debug] https://aryion.com:443 "POST /forum/ucp.php?mode=login HTTP/1.1" 503 None
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/forum/ucp.php?mode=login' (5/5)
[aryion][error] HttpError: '503 Service Temporarily Unavailable' for 'https://aryion.com/forum/ucp.php?mode=login'

also when opening these pages in a webbrowser they work perfectly, so the site isn't down.

@mikf
Copy link
Owner

mikf commented Sep 9, 2022

I'm not sure if my syntax is correct for --write-pages though.

The syntax is correct, but I should have explained what --write-pages actually does and I meant by "output". Sorry about that.

--write-pages makes gallery-dl write the results of HTTP requests into separate files in your current working directory.

There should now be several files called 01_https_aryion.com_forum_….txt etc in the same directory as your gallery-dl.exe, which should contain a more complete error message than "503 Service Temporarily Unavailable". Make sure they don't contain anything you don't want public and post their content here.

@left1000
Copy link
Author

left1000 commented Sep 10, 2022

04_https_aryion.com_forum_ucp.php_mode_login.txt
03_https_aryion.com_forum_ucp.php_mode_login.txt
02_https_aryion.com_forum_ucp.php_mode_login.txt
01_https_aryion.com_forum_ucp.php_mode_login.txt
05_https_aryion.com_forum_ucp.php_mode_login.txt

Enable JavaScript and cookies to continue

looks like some sort of captcha challenge?

also these text links don't give away my cookies do they?

@mikf
Copy link
Owner

mikf commented Sep 14, 2022

It's Cloudflare's IUAM challenge, it just didn't get recognized as such. (73a52a9)

To get past this, you need to get the cf_clearance cookie value for aryion.com from your browser and use it and your browser's user agent with gallery-dl.

For details, take a look at the recent nhentai issues where the same problem occurs: https://github.com/mikf/gallery-dl/issues?q=label%3Acloudflare+nhentai

also these text links don't give away my cookies do they?

they don't

@left1000
Copy link
Author

left1000 commented Sep 14, 2022

Did I do the instructions correctly? because it still doesn't work.

.\gallery-dl -vv --write-pages --cookies "D:\gallery-dl\aryion.com_cookies.txt" -o user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36" https://aryion.com/g4/view/510079

[gallery-dl][debug] Version 1.23.0 - Executable
[gallery-dl][debug] Python 3.7.9 - Windows-10-10.0.19041
[gallery-dl][debug] requests 2.28.1 - urllib3 1.26.12
[gallery-dl][debug] Starting DownloadJob for 'https://aryion.com/g4/view/510079'
[aryion][debug] Using AryionPostExtractor for 'https://aryion.com/g4/view/510079'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): aryion.com:443
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=510079 HTTP/1.1" 503 0
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/g4/data.php?id=510079' (1/5)
[urllib3.connectionpool][debug] Resetting dropped connection: aryion.com
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=510079 HTTP/1.1" 503 0
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/g4/data.php?id=510079' (2/5)
[urllib3.connectionpool][debug] Resetting dropped connection: aryion.com
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=510079 HTTP/1.1" 503 0
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/g4/data.php?id=510079' (3/5)
[urllib3.connectionpool][debug] Resetting dropped connection: aryion.com
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=510079 HTTP/1.1" 503 0
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/g4/data.php?id=510079' (4/5)
[urllib3.connectionpool][debug] Resetting dropped connection: aryion.com
[urllib3.connectionpool][debug] https://aryion.com:443 "HEAD /g4/data.php?id=510079 HTTP/1.1" 503 0
[aryion][debug] '503 Service Temporarily Unavailable' for 'https://aryion.com/g4/data.php?id=510079' (5/5)
[aryion][error] HttpError: '503 Service Temporarily Unavailable' for 'https://aryion.com/g4/data.php?id=510079'

I must have done something wrong because this time all the files from --write-pages are left blank

03_https_aryion.com_g4_data.php_id_510079.txt
02_https_aryion.com_g4_data.php_id_510079.txt
01_https_aryion.com_g4_data.php_id_510079.txt
05_https_aryion.com_g4_data.php_id_510079.txt
04_https_aryion.com_g4_data.php_id_510079.txt

@mikf
Copy link
Owner

mikf commented Sep 14, 2022

You did do it correctly from what I can tell, but the cf_clearance cookie and others are most likely outdated or for some reason don't match the user agent or something else. The whole process is rather fiddly.

I usually use a private browser window for this, where it has to do a new browser check every time and it provides "fresh" cookies that definitely work, at least for me.

I must have done something wrong because this time all the files from --write-pages are left blank

No, you haven't. An empty response is expected for HEAD requests. You only get an actual response for GET/POST/etc.

@left1000
Copy link
Author

Actually I have another problem.... aryion.com's cookie doesn't contain a cf_clearance value. Which leaves me a little puzzled.

@mikf
Copy link
Owner

mikf commented Sep 16, 2022

Mmh, it seems that only requests from gallery-dl trigger the Cloudflare check, so doing the usual cf_clearance thing is off the table.
Try -o browser=firefox. Maybe using HTTP requests that look more like they are coming from a real browser helps here and bypasses the Cloudflare block.

@left1000
Copy link
Author

It works

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants