-
Notifications
You must be signed in to change notification settings - Fork 10.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Crunchyroll] ERROR: Unable to download webpage: HTTP Error 403: Forbidden #28398
Comments
The Crunchyroll extractor cannot download entire seasons. Only separate episodes. |
I have used this command line (youtube-dl -f best[format_id*=enUS] --playlist-start [number] https://www.crunchyroll.com/[show]) to download a lot of series from Crunchyroll. My issue is that I can't seem to get anything to download from Crunchyroll anymore whether it's a playlist or an individual episode. It's not even returning a format list these days, only the 403 error. This wasn't a problem until about a few weeks ago. After repeated attempts a couple days ago, I did get one episode to download. Of course that's kind of a pain to do every time I'd like to download something. |
@tasdanduil I guess they've made some changes to their API. |
Alright. Thanks for the recommendation. I'll probably be using that from now on. |
@tasdanduil No problem. I believe all the same shows are available there too, because it is essentially the same Crunchyroll but with Hidive and a few other anime and cartoon streaming services bundled together. |
Do you know if this also applies to |
@Columbo199X I have no idea. |
Hello I am getting this error too but it's just for one file not a playlist |
im getting the same when trying to download trying to use it on Debian Sid fully updated as of today
|
The problem seems to be that, when ytdl tries to After a lot of trial and error, I've found a workaround though: It's fairly fragile (one slight difference in the UA string makes it 403 again and also invalidates the cookie file?) but looks to be working well enough. Seems like CR is getting more sophisticated with blocking automated tools and they've had a DRM rollout looming for a while too... |
It seems to be the Seems this is a cloudflare cookie: https://support.cloudflare.com/hc/en-us/articles/200170156-Understanding-the-Cloudflare-Cookies#12345681 |
@6cUbi57z I havent messed with using cookies and youtube-dl how would I get the specific cookie needed to resolve the issue? |
@omegahack0 the info about the cookie was more for someone trying to fix the bug. For the workaround, you will need to go to crunchyroll in your browser and then use something like https://addons.mozilla.org/en-US/firefox/addon/cookies-txt/ to export the cookies for the site. Then use the cookie file paramter to point to the file and probably the user agent parameter to match your browser as closely as possible. The workaround doesn't work in all cases though as the purpose of the cookie is to prevent automated programs like this from accessing the site. |
If it breaks, you usually only need to re-export the cookies file from your browser. |
According to Cloudflare, this is the fingerprinting technique used to compute the cookie and/or deny access. |
- Add workaround for ytdl-org/youtube-dl#28398 - Refactor some code - Use PrettyTable on anime selection
- Add workaround for ytdl-org/youtube-dl#28398 - Refactor some code - Use PrettyTable on anime selection
The ticket looks to have been rasied using the broken site template which doesn't specifically prompt for urls. The poster has included a log complete with command line arguments which includes the url and start index of the playlist among plenty of other information. If they've done something wrong I don't know what it is and would probably make the same mistake. |
same problem:
|
Although this may be relevant, yt-dlp's extractor for CR currently has significant changes relative to yt-dl's (login and beta site support, in general). If you'd like your issue to be progressed you should raise it on the yt-dlp tracker. |
As with some other CF-blocked sites, using $ python -m youtube_dl --ignore-config -f worst -v --test 'https://www.crunchyroll.com/sword-art-online/episode-1-the-world-of-swords-606739' --user-agent 'Mozilla/5.0'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--ignore-config', u'-f', u'worst', u'-v', u'--test', u'https://www.crunchyroll.com/sword-art-online/episode-1-the-world-of-swords-606739', u'--user-agent', u'Mozilla/5.0']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: cc179df34
[debug] Python version 2.7.17 (CPython) - Linux-4.4.0-210-generic-i686-with-Ubuntu-16.04-xenial
[debug] exe versions: avconv 4.3, avprobe 4.3, ffmpeg 4.3, ffprobe 4.3
[debug] Proxy map: {}
[crunchyroll] 606739: Downloading webpage
[crunchyroll] 606739: Downloading adaptive_hls-audio-jaJP information
[crunchyroll] 606739: Downloading adaptive_hls-audio-jaJP-hardsub-ptBR information
[crunchyroll] 606739: Downloading adaptive_hls-audio-jaJP-hardsub-esLA information
[crunchyroll] 606739: Downloading adaptive_hls-audio-jaJP-hardsub-enUS information
[crunchyroll] 606739: Downloading adaptive_hls-audio-jaJP-hardsub-arME information
[crunchyroll] 606739: Downloading media info
WARNING: Unable to download XML: HTTP Error 404: Not Found
[debug] Invoking downloader on u'https://pl.crunchyroll.com/evs3/d4e9300717fae6c66a7e80909e734c5a/assets/uhcpzj0r97j1i92_1672805.mp4/index-v1-a1.m3u8?Expires=1655248090&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9wbC5jcnVuY2h5cm9sbC5jb20vZXZzMy9kNGU5MzAwNzE3ZmFlNmM2NmE3ZTgwOTA5ZTczNGM1YS9hc3NldHMvdWhjcHpqMHI5N2oxaTkyXyoubTN1OD8qIiwiQ29uZGl0aW9uIjp7IkRhdGVMZXNzVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNjU1MjQ4MDkwfX19XX0_&Signature=sOq2~nYrvIXkzpzP2tfjwCPDlX5pUqInkuGjR0PdYz~xjuDZaDffgH5dcq3o4jUdj7pxpf3160ShmkZuBnP-B66pxx1hZpnWLXKUYFteSc-c-fdgyMEL6mVGtCSwWI0R~42D-wiHKgjzi1mDezmfDRXFujn44aOrJsVyY4AfZIoksL6DYY5xKzGCo~~ytYgL5k02zito-BF7nSIx7WUGnfM0LgaxF4VbGEyFlW1k9~T-FGI~dwGBOAlynZQbykOWUUgPxK6qo-oZLWdp72HpNhA5FO0mZa3UzFhdyCB3qmxEvoCGr7xk1rgVS6MKLg9BpjXjt5jWvbV~JvPbbYmNzw__&Key-Pair-Id=APKAJMWSQ5S7ZB3MF5VA&cdn=ll-prod'
[download] Destination: Sword Art Online Episode 1 – The World of Swords-606739.mp4
[debug] ffmpeg command line: ffmpeg -y -loglevel verbose -headers 'Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Cookie: __cf_bm=84USPPLOxkzBIJ2EcxeOaIArQjedWU4Oumv78DopESw-1655075273-0-ATGxWTE/gbLaCZfox7ck/ByLG9fdnzj4PQngxILeB05d8OpH2f4L88THznuXQ15z6euQnVKvVtPIowOL8DN1BR1hUZoohCRF6L5Zhu3uCYKJ; c_visitor=555b042c-5425-4fde-b889-9f6b4ddf51d8; session_id=3e07a094fa62a34c580c670afc258ab9
' -i 'https://pl.crunchyroll.com/evs3/d4e9300717fae6c66a7e80909e734c5a/assets/uhcpzj0r97j1i92_1672805.mp4/index-v1-a1.m3u8?Expires=1655248090&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9wbC5jcnVuY2h5cm9sbC5jb20vZXZzMy9kNGU5MzAwNzE3ZmFlNmM2NmE3ZTgwOTA5ZTczNGM1YS9hc3NldHMvdWhjcHpqMHI5N2oxaTkyXyoubTN1OD8qIiwiQ29uZGl0aW9uIjp7IkRhdGVMZXNzVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNjU1MjQ4MDkwfX19XX0_&Signature=sOq2~nYrvIXkzpzP2tfjwCPDlX5pUqInkuGjR0PdYz~xjuDZaDffgH5dcq3o4jUdj7pxpf3160ShmkZuBnP-B66pxx1hZpnWLXKUYFteSc-c-fdgyMEL6mVGtCSwWI0R~42D-wiHKgjzi1mDezmfDRXFujn44aOrJsVyY4AfZIoksL6DYY5xKzGCo~~ytYgL5k02zito-BF7nSIx7WUGnfM0LgaxF4VbGEyFlW1k9~T-FGI~dwGBOAlynZQbykOWUUgPxK6qo-oZLWdp72HpNhA5FO0mZa3UzFhdyCB3qmxEvoCGr7xk1rgVS6MKLg9BpjXjt5jWvbV~JvPbbYmNzw__&Key-Pair-Id=APKAJMWSQ5S7ZB3MF5VA&cdn=ll-prod' -c copy -fs 10241 -f mp4 'file:Sword Art Online Episode 1 – The World of Swords-606739.mp4.part'
ffmpeg version 4.3-2ubuntu0~ppa16.04+8 Copyright (c) 2000-2020 the FFmpeg developers
...
libavutil 56. 51.100 / 56. 51.100
libavcodec 58. 91.100 / 58. 91.100
libavformat 58. 45.100 / 58. 45.100
libavdevice 58. 10.100 / 58. 10.100
libavfilter 7. 85.100 / 7. 85.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 7.100 / 5. 7.100
libswresample 3. 7.100 / 3. 7.100
libpostproc 55. 7.100 / 55. 7.100
[tcp @ 0x1605000] Starting connection attempt to 108.156.28.89 port 443
[tcp @ 0x1605000] Successfully connected to 108.156.28.89 port 443
[hls @ 0x1601040] Skip ('#EXT-X-ALLOW-CACHE:YES')
[hls @ 0x1601040] Skip ('#EXT-X-VERSION:5')
[hls @ 0x1601040] HLS request for url 'https://ll.v.vrv.co/evs3/d4e9300717fae6c66a7e80909e734c5a/assets/uhcpzj0r97j1i92_1672805.mp4/seg-1-v1-a1.ts?t=exp=1655248094~acl=/evs3/d4e9300717fae6c66a7e80909e734c5a/assets/uhcpzj0r97j1i92_1672805.mp4/*~hmac=6b50d6617f56fe5978aee26483b5e58d15cc5e00ae7fb22f44077c8b0ac3293b', offset 0, playlist 0
[hls @ 0x1601040] Opening 'https://ll.v.vrv.co/evs3/d4e9300717fae6c66a7e80909e734c5a/assets/uhcpzj0r97j1i92_1672805.mp4/encryption.key?t=exp=1655248094~acl=/evs3/d4e9300717fae6c66a7e80909e734c5a/assets/uhcpzj0r97j1i92_1672805.mp4/*~hmac=6b50d6617f56fe5978aee26483b5e58d15cc5e00ae7fb22f44077c8b0ac3293b' for reading
[tcp @ 0x18ae700] Starting connection attempt to 87.248.214.8 port 443
[tcp @ 0x18ae700] Successfully connected to 87.248.214.8 port 443
[AVIOContext @ 0x1b1ed00] Statistics: 16 bytes read, 0 seeks
[hls @ 0x1601040] Opening 'crypto+https://ll.v.vrv.co/evs3/d4e9300717fae6c66a7e80909e734c5a/assets/uhcpzj0r97j1i92_1672805.mp4/seg-1-v1-a1.ts?t=exp=1655248094~acl=/evs3/d4e9300717fae6c66a7e80909e734c5a/assets/uhcpzj0r97j1i92_1672805.mp4/*~hmac=6b50d6617f56fe5978aee26483b5e58d15cc5e00ae7fb22f44077c8b0ac3293b' for reading
[tcp @ 0x1b1f980] Starting connection attempt to 87.248.214.8 port 443
[tcp @ 0x1b1f980] Successfully connected to 87.248.214.8 port 443
[h264 @ 0x18c9c00] Reinit context to 432x240, pix_fmt: yuv420p
Input #0, hls, from ...
Duration: 00:23:40.00, start: 0.226000, bitrate: 0 kb/s
Program 0
Metadata:
variant_bitrate : 0
Stream #0:0: Video: h264 (Constrained Baseline), 1 reference frame ([27][0][0][0] / 0x001B), yuv420p(left), 428x240 (432x240) [SAR 320:321 DAR 16:9], 23.98 fps, 23.98 tbr, 90k tbn, 47.95 tbc
Metadata:
variant_bitrate : 0
Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 22050 Hz, stereo, fltp
Metadata:
variant_bitrate : 0
Stream #0:2: Data: timed_id3 (ID3 / 0x20334449)
Metadata:
variant_bitrate : 0
Output #0, mp4, to 'file:Sword Art Online Episode 1 – The World of Swords-606739.mp4.part':
Metadata:
encoder : Lavf58.45.100
Stream #0:0: Video: h264 (Constrained Baseline), 1 reference frame (avc1 / 0x31637661), yuv420p(left), 428x240 (0x0) [SAR 320:321 DAR 16:9], q=2-31, 23.98 fps, 23.98 tbr, 90k tbn, 90k tbc
Metadata:
variant_bitrate : 0
Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 22050 Hz, stereo, fltp
Metadata:
variant_bitrate : 0
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Automatically inserted bitstream filter 'aac_adtstoasc'; args=''
No more output streams to write to, finishing.
Not writing 'clli' atom. No content light level info.
Not writing 'mdcv' atom. Missing mastering metadata.
frame= 22 fps=0.0 q=-1.0 Lsize= 13kB time=00:00:00.87 bitrate= 125.4kbits/s speed=1.87e+03x
video:5kB audio:6kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 14.250042%
Input file #0 (...):
Input stream #0:0 (video): 22 packets read (5520 bytes);
Input stream #0:1 (audio): 16 packets read (6494 bytes);
Input stream #0:2 (data): 0 packets read (0 bytes);
Total: 38 packets (12014 bytes) demuxed
Output file #0 (file:Sword Art Online Episode 1 – The World of Swords-606739.mp4.part):
Output stream #0:0 (video): 22 packets muxed (5520 bytes);
Output stream #0:1 (audio): 16 packets muxed (6494 bytes);
Total: 38 packets (12014 bytes) muxed
[AVIOContext @ 0x1c4b900] Statistics: 2 seeks, 4 writeouts
[AVIOContext @ 0x1b25980] Statistics: 35392 bytes read, 0 seeks
[AVIOContext @ 0x184a700] Statistics: 103994 bytes read, 0 seeks
[ffmpeg] Downloaded 13726 bytes
[download] 100% of 13.40KiB in 00:03
$ Elsewhere it has been shown that sending the "correct" For pages that need a login, it may, as above, be necessary to use the UA sent by the browser that gathered the login cookies instead. |
Checklist
Verbose log
Description
I came across this issue not too long ago. Haven't been able to fix it on my own, so here I am. I've tried adding cookies.txt to the command line as well as the user agent, neither solved the issue.
The text was updated successfully, but these errors were encountered: