[nebula] Add new extractor (Nebula / watchnebula.com) #27859
Conversation
When the 'sessionid' cookie is submitted to the `/auth/login/` endpoint, the response is always a 403. This typically happens when youtube_dl is run with both `--netrc` and `--cookies` as your default configuration. In that situation, the first authentication succeeds and stores the `sessionid` cookie in the cookie jar. During subsequent authentication attempts, the cookie is sent alongside and causes the authentication to fail. This is very unexpected and we therefore specifically handle this case.
When Nebula isn't accessed for a while, the Zype access token stored on the Nebula backend expires. It is then no longer returned by the user endpoint. The Nebula frontend has the same issue and keeps polling for the Zype token in this case. This isn't implemented in this extractor yet, but at least a specific error message now prints some helpful advice.
|
This is awesome! Thank you so much. |
|
I should mention that I've simultaneously developed an extractor for the yt-dlp fork here: yt-dlp/yt-dlp#122 It has seen a couple of internal improvements. I could backport them to ytdl-org if you let me know there's some renewed interest. I just haven't bothered so far, because I assumed this PR to be dead in the water after almost a year of no activity. |
|
As of 2021-04-11, an improved fork of this PR has been released with yt-dlp. |
And clean up sorting
* somewhat like JSON.stringify()
* replaces json.dumps(..., separators=(',',':')).encode('utf-8')
* more kwarg options available
|
This should now be as up-to-date with yt-dlp as possible. The account-needed functions need to be tested. Anyone with a Nebula account is invited to test the PR, disabling the |
|
Disclaimer: I'm not currently an active subscriber of Nebula and also haven't developed the latest version of the yt-dlp integration. (I did kickstart it and maintained it for a good while, though.) One bit you might want to change is the |
|
Thanks for chiming in and for letting me [ab]use your PR branch!
I would, but then people couldn't so easily switch between this extractor and the yt-dlp one. It's a half-empty/half-full situation. |
You're very welcome! I love that someone is still using the remnants of my dirty little hack. 😉
True. As usual, there's no right answer when it comes to compatibility. |
Before submitting a pull request make sure you have:
In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:
What is the purpose of your pull request?
Description of your pull request and other information
(This is a post-DMCA repost of #24805 on a new fork. Read below for details.)
This is a new extractor for the video platform Nebula (https://watchnebula.com/), created by the streamer community Standard. Nebula uses the Zype video infrastructure and this extractor is using the url_transparent mode to hand off video extraction to the Zype extractor (which has shown to be super reliable here).
All videos on Nebula require a subscription to access. Multiple authentication methods are supported, with the most convenient probably being
.netrc, but a few alternatives (cookie jar, environment variable, token in--videopassword) have been implemented as well.Initial discussion has occured on issue #21258. I didn't open the issue, but hijacked it for my implementation.
Afterwards, development discussion has occured over multiple months on PR #24805. The fork that PR was based on has been destroyed by the DMCA takedown. According to GitHub Support, a transfer of that PR to a different fork is not possible, which is why I'm now opening this new PR for the new fork (original commits are the same).
As there are no known freely available videos, the three test cases only pass when an auth token to a user account with a (paid) subscription is present. This can be specified using the
NEBULA_TOKENenvironment variable (as I couldn't figure out a way to use--netrcin unit tests).I have aimed to provide comprehensive documentation of approach and behavior in class and method docstrings.