[Udemy] ERROR: Unable to download webpage: HTTP Error 403: Forbidden #29915

fhcaglayan · 2021-09-06T14:00:39Z

Checklist

I'm reporting a broken site support
I've verified that I'm running youtube-dl version 2021.06.06
I've checked that all provided URLs are alive and playable in a browser
I've checked that all URLs and arguments with special characters are properly quoted or escaped
I've searched the bugtracker for similar issues including closed ones

Verbose log


C:\>youtube-dl.exe -u **** -p **** https://www.udemy.com/course/learn-advanced-java/ --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-u', 'PRIVATE', '-p', 'PRIVATE', 'https://www.udemy.com/course/learn-advanced-java/', '--verbose']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2021.06.06
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: none
[debug] Proxy map: {}
[udemy:course] Downloading login popup
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpkqxnwl31\build\youtube_dl\extractor\common.py", line 634, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpkqxnwl31\build\youtube_dl\YoutubeDL.py", line 2288, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default```

Description

I was trying to download the learn advanced Java course which I have bought from Udemy for offline watching purposes. However, youtube-dl gives me an error when trying to do so.

The text was updated successfully, but these errors were encountered:

pbpalomino · 2021-09-08T18:07:28Z

Your URL is not right.

try

youtube-dl.exe -u **** -p **** https://www.udemy.com/learn-advanced-java/ --verbose

fhcaglayan · 2021-09-08T18:19:53Z

Same problem unfortunately:

C:\>youtube-dl.exe -u **** -p **** https://www.udemy.com/learn-advanced-java/ --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-u', 'PRIVATE', '-p', 'PRIVATE', 'https://www.udemy.com/learn-advanced-java/', '--verbose']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2021.06.06
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: none
[debug] Proxy map: {}
[udemy:course] Downloading login popup
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpkqxnwl31\build\youtube_dl\extractor\common.py", line 634, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpkqxnwl31\build\youtube_dl\YoutubeDL.py", line 2288, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

fhcaglayan · 2021-09-08T18:20:21Z

If you want in private I can share my account details with a temporary password so you can try for yourself as well.

pbpalomino · 2021-09-08T18:56:54Z

@fhcaglayan may be an unescaped string from you password.

You are not able to login.

Try using cookies.

this works on my machine

youtube-dl ^
--cookies udemy_cookies.txt ^
-o '%(playlist)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s. %(title)s.%(ext)s' ^
'YOURURL' ^
--verbose

^ splits large lines on windows cmd.

fhcaglayan · 2021-09-08T19:26:39Z

I have also tried with password temp1234... Might that also be unescaped?

…

On Wed, 8 Sep 2021, 20:57 ppk ***@***.***> wrote: @fhcaglayan <https://github.com/fhcaglayan> may be an unescaped string from you password Try using cookies. this works on my machine youtube-dl ^ --cookies udemy_cookies.txt ^ -o '%(playlist)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s. %(title)s.%(ext)s' ^ 'YOURURL' ^ --verbose ^ splits large lines. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#29915 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AVP3EC5EVENRFKDJ7DBYZV3UA6WYHANCNFSM5DQQTDZA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

lorenzlars · 2021-10-17T09:56:36Z

I also have the same issue. I tried it with escaped username and password and also with cookie. I also tried the url with /courses in it and without it. And I tried ending the url with / and without. I think something has changed on Udemy side which causes the issue.

dirkf · 2021-11-17T15:33:27Z

Duplicate of #29410.

brskl88 · 2021-11-28T10:10:00Z

Hey everybody.I have almost the same problem.My problem is when i start to download, it downloads first video without any problems.It is on my hard disk.But when it tries to download second one it gives the error.I use cookie method and i export the cookie file by "Export Cookies" extension on Firefox.But on my case it gives another error when it tries to download second video.It says

[udemy] 15779426: Downloading lecture JSON
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug

So i add my verbose output here.

Verbose Output

youtube-dl --cookies cookie.txt --verbose https://www.udemy.com/the-advanced-cryptocurrency-trading-course-with-strategies/
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--cookies', 'cookie.txt', '--verbose', 'https://www.udemy.com/the-advanced-cryptocurrency-trading-course-with-strategies/']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.06.06
[debug] Python version 3.8.10 (CPython) - Linux-5.11.0-27-generic-x86_64-with-glibc2.29
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4, rtmpdump 2.4
[debug] Proxy map: {}
[udemy:course] the-advanced-cryptocurrency-trading-course-with-strategies: Downloading webpage
[udemy:course] 2500544: Downloading course curriculum
[download] Downloading playlist: 2500544
[udemy:course] playlist 2500544: Collected 39 video ids (downloading 39 of them)
[download] Downloading video 1 of 39
[udemy] 15929840: Downloading webpage
[udemy] 15929840: Downloading lecture JSON
[udemy] 19874331: Downloading m3u8 information
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on 'https://www.udemy.com/assets/19874331/files/2019-09-10_20-55-46-ceaae9486ee7f497888103ddf974b304/2/hls/AVC_1920x1080_800k_AAC-HE_64k/aa0025a2af4592ba37f10526c90a4b4ff6c6.m3u8?provider=cdn77&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJwYXRoIjoiMjAxOS0wOS0xMF8yMC01NS00Ni1jZWFhZTk0ODZlZTdmNDk3ODg4MTAzZGRmOTc0YjMwNC8yLyIsImV4cCI6MTYzODEwOTM4MH0.7JGx6OM1vh3_hOMK2Pt3W18fYzckuL3FdWkljG4LK7A&v=1'
[download] What You Will Learn in This Course-19874331.mp4 has already been downloaded
[download] 100% of 19.40MiB
[debug] ffmpeg command line: ffprobe -show_streams 'file:What You Will Learn in This Course-19874331.mp4'
[ffmpeg] Fixing malformed AAC bitstream in "What You Will Learn in This Course-19874331.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:What You Will Learn in This Course-19874331.mp4' -c copy -f mp4 -bsf:a aac_adtstoasc 'file:What You Will Learn in This Course-19874331.temp.mp4'
[download] Downloading video 2 of 39
[udemy] 15779426: Downloading webpage
[udemy] 15779426: Downloading lecture JSON
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/youtube_dl/YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/udemy.py", line 408, in _real_extract
    self._sort_formats(formats, field_preference=('height', 'width', 'tbr', 'format_id'))
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/common.py", line 1374, in _sort_formats
    raise ExtractorError('No video formats found')
youtube_dl.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

So is there any solution to fix it?Or do i make any mistakes when writing my command?Thank you.

dirkf · 2021-11-28T11:47:22Z

That looks like #25656, not that it helps.

brskl88 · 2021-11-28T13:40:06Z

That looks like #25656, not that it helps.

Thank you for the response.I already removed that part of the link as it says to remove "/course/".but it didn't fix the problem.is there another solution to fix this?the interesting thing is everything works for first video and it downloads it.but for second one it gives the error.

dirkf · 2021-11-28T22:39:57Z

Try loading the failing video in your browser. That may reveal what the problem is.

zdegeorge · 2021-12-23T14:56:06Z

I was having the 403 issue, and then I was having an extract login-form form issue, but I was able to resolve both of them and get youtube-dl working with Udemy.

First I was able to resolve the 403 by logging into the website in chrome and then downloading the cookie file and including it with the '--cookie /path/to/cookie.txt' option. (See How do I pass cookies to youtube-dl). But then I was still getting the error:

ERROR: Unable to extract login-form form; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

It turns out this was caused because in my youtube-dl request I was passing -u username -p password options to the request which was causing the login popup preventing the download.

Also to note, in the URL for a course in Udemy, i.e. https://www.udemy.com/course/understanding-typescript/, there is an extra /course/ that needs to be removed for the request. So the URL should instead look like https://www.udemy.com/understanding-typescript/.

Put all this together and a valid request should look like:

youtube-dl --cookie /path/to/cookies.txt -o '~/your/path/%(playlist)s/%(chapter_number)s - %(chapter)s/%(title)s.%(ext)s' https://www.udemy.com/COURSE-NAME/

Where you replace the path to the cookie file, the path to the output directory, and the course name in the Udemy URL.

Hope this helps :)

dirkf · 2021-12-23T15:17:44Z

... in the URL for a course in Udemy, i.e. https://www.udemy.com/course/understanding-typescript/, there is an extra /course/ that needs to be removed for the request ...

This should be handled automatically. Could you post some -v -F logs showing behaviour with/without course/? Or if this patch makes it work with the raw URL as quoted above, let us know:

--- old/youtube-dl/youtube_dl/extractor/udemy.py
+++ new/youtube-dl/youtube_dl/extractor/udemy.py
@@ -17,6 +17,7 @@
     float_or_none,
     int_or_none,
     js_to_json,
+    remove_start,
     sanitized_Request,
     try_get,
     unescapeHTML,
@@ -437,6 +438,7 @@
     def _real_extract(self, url):
         course_path = self._match_id(url)
 
+        course_path = remove_start(course_path, 'course/')
         webpage = self._download_webpage(url, course_path)
 
         course_id, title = self._extract_course_info(webpage, course_path)

dirkf · 2021-12-29T14:54:40Z

On reflection I can't see how stripping course/ could affect the extraction unless through redirection back to the unstripped URL.

There seem to be two major problems here:

the extractor hasn't been properly updated for more than a year, judging by old failed PRs that appear to target the current site structure;
yt-dl web requests are being subjected to some sort of protocol violation presumably imposed through Cloudflare, so that any request may randomly be rejected with 403 (this is how redirection might have an effect).

This happens despite trying Python versions from 2.7 to 3.9, removing the Accept: header, varying the UA, ...

From the metadata, the site may possibly also be serving videos with DRM.

It has an optional Download button that the Instructor can enable. It may be easier to ask for that.

thelonghop · 2021-12-31T16:03:00Z

I'm getting this error as well. Have tried with -u/-p and with cookies, with and without /courses/

$ youtube-dl --verbose --cookies ./network008 -o '%(playlist)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s. %(title)s.%(ext)s' https://www.udemy.com/comptia-network-n10-008/
[debug] System config: [u'-o', u'/home/user1/Videos/ytdl/%(title)s']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'--cookies', u'./network008', u'-o', u'%(playlist)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s. %(title)s.%(ext)s', u'https://www.udemy.com/comptia-network-n10-008/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 2.7.18 (CPython) - Linux-5.13.0-22-generic-x86_64-with-LinuxMint-20.2-uma
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4, rtmpdump 2.4
[debug] Proxy map: {}
[udemy:course] comptia-network-n10-008: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 634, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2288, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

dirkf · 2021-12-31T16:18:20Z

Looks like the same problem. If someone can work out how to fix Cloudflare's protocol violation (2nd bullet above) it could be possible to make further progress.

agails · 2022-02-09T13:17:22Z

I was having the 403 issue, and then I was having an extract login-form form issue, but I was able to resolve both of them and get youtube-dl working with Udemy.

First I was able to resolve the 403 by logging into the website in chrome and then downloading the cookie file and including it with the '--cookie /path/to/cookie.txt' option. (See How do I pass cookies to youtube-dl). But then I was still getting the error:
ERROR: Unable to extract login-form form; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
It turns out this was caused because in my youtube-dl request I was passing -u username -p password options to the request which was causing the login popup preventing the download.

Also to note, in the URL for a course in Udemy, i.e. https://www.udemy.com/course/understanding-typescript/, there is an extra /course/ that needs to be removed for the request. So the URL should instead look like https://www.udemy.com/understanding-typescript/.

Put all this together and a valid request should look like:
youtube-dl --cookie /path/to/cookies.txt -o '~/your/path/%(playlist)s/%(chapter_number)s - %(chapter)s/%(title)s.%(ext)s' https://www.udemy.com/COURSE-NAME/
Where you replace the path to the cookie file, the path to the output directory, and the course name in the Udemy URL.

Hope this helps :)

Thank you so much, your instructions worked like a charm! This should be highlighted on the main page!

nicheath · 2022-03-05T19:56:48Z

youtube-dl 2021.12.17 still has a problem using -u and -p however using --cookies is a workaround.

dirkf · 2022-03-08T23:59:19Z

What's wrong?

...
ERROR: Unable to extract course id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
...

See #30719.

GwamakaCharles · 2022-05-16T08:55:23Z

TESTED AND WORKS ON A MAC OS

Install youtube-dl through brew install youtube-dl. If you used curl, you can remove it through rm -rf /usr/local/bin/youtube-dl.
If you face symlink permissions ERRORS, run sudo chown -R $(whoami) $(brew --prefix)/* then brew doctor and fix each error according to the instructions.
brew cleanup
brew link youtube-dl
cd . (navigate to your favorable location that you wish to put your videos folder)
Login to udemy and get site cookies text file. I used Get Cookies.txt.
mkdir MyVideos. (name it as you wish)
youtube-dl --cookies udemy.com_cookies.txt -o '~/MyVideos/%(playlist)s/%(chapter_number)s - %(chapter)s/%(title)s.%(ext)s' https://www.udemy.com/<course-name>
Done! 🎮

dirkf · 2023-04-13T17:45:45Z

Still duplicate of #29410.

This comment was marked as duplicate.

Sign in to view

dirkf mentioned this issue Oct 5, 2021

[Udemy] ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()) #30053

Closed

6 tasks

This comment was marked as off-topic.

Sign in to view

HyeongminKim mentioned this issue Oct 19, 2021

List of duplicate documents #30130

Closed

dirkf mentioned this issue Nov 17, 2021

impossible to download Udemy course videos #30246

Closed

6 tasks

This comment was marked as duplicate.

Sign in to view

dirkf mentioned this issue Dec 28, 2021

[Udemy] No video formats found #30382

Closed

This comment has been minimized.

Sign in to view

This comment was marked as off-topic.

Sign in to view

dmiraj mentioned this issue Jan 31, 2022

Fix forbidden403 #30573

Open

11 tasks

dirkf changed the title ~~ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug~~ [Udemy] ERROR: Unable to download webpage: HTTP Error 403: Forbidden Feb 26, 2022

dirkf mentioned this issue Mar 4, 2022

udemy videos cannot download (login page) #30714

Closed

This comment was marked as off-topic.

Sign in to view

This comment was marked as resolved.

Sign in to view

dirkf added broken-IE problem with existing site extraction account-needed duplicate labels Apr 27, 2022

dirkf mentioned this issue Jan 21, 2023

Unable to download Udemy paid lectures offline #31482

Closed

This comment was marked as duplicate.

Sign in to view

dirkf closed this as not planned Won't fix, can't repro, duplicate, stale Apr 13, 2023

dirkf mentioned this issue Apr 13, 2023

Unable to download video from Udemy 'HTTP Error 403: Forbidden' (login page) #29410

Open

5 tasks

[Udemy] ERROR: Unable to download webpage: HTTP Error 403: Forbidden #29915

[Udemy] ERROR: Unable to download webpage: HTTP Error 403: Forbidden #29915

Comments

fhcaglayan commented Sep 6, 2021

Checklist

Verbose log

Description

This comment was marked as duplicate.

pbpalomino commented Sep 8, 2021

fhcaglayan commented Sep 8, 2021 • edited by dirkf Loading

fhcaglayan commented Sep 8, 2021

pbpalomino commented Sep 8, 2021 • edited Loading

fhcaglayan commented Sep 8, 2021 via email

This comment was marked as duplicate.

This comment was marked as off-topic.

lorenzlars commented Oct 17, 2021

dirkf commented Nov 17, 2021 • edited Loading

brskl88 commented Nov 28, 2021 • edited Loading

dirkf commented Nov 28, 2021

brskl88 commented Nov 28, 2021

dirkf commented Nov 28, 2021

This comment was marked as duplicate.

zdegeorge commented Dec 23, 2021

dirkf commented Dec 23, 2021 • edited Loading

dirkf commented Dec 29, 2021

thelonghop commented Dec 31, 2021 • edited by dirkf Loading

dirkf commented Dec 31, 2021

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment was marked as off-topic.

This comment was marked as off-topic.

agails commented Feb 9, 2022

nicheath commented Mar 5, 2022 • edited Loading

This comment was marked as off-topic.

This comment was marked as resolved.

dirkf commented Mar 8, 2022

GwamakaCharles commented May 16, 2022

TESTED AND WORKS ON A MAC OS

This comment was marked as duplicate.

dirkf commented Apr 13, 2023

fhcaglayan commented Sep 8, 2021 •

edited by dirkf

Loading

pbpalomino commented Sep 8, 2021 •

edited

Loading

dirkf commented Nov 17, 2021 •

edited

Loading

brskl88 commented Nov 28, 2021 •

edited

Loading

dirkf commented Dec 23, 2021 •

edited

Loading

thelonghop commented Dec 31, 2021 •

edited by dirkf

Loading

nicheath commented Mar 5, 2022 •

edited

Loading