Add new exceptions to be able to display different error messages in apps #509

AudricV · 2021-01-10T18:26:20Z

This PR add six new exceptions to be able to display different error messages in NewPipe when user tries to play an unplayable content. All of these exceptions extend the ContentNotAvailableException.

AgeRestrictedContentException, for age-restricted content which has no video or audio streams.
GeographicRestrictionException, for georestricted content (used for SoundCloud and YouTube georestricted videos, not working on YouTube Music georestricted tracks because YouTube shows "This video is not available")
PaidContentException, for paid content like YouTube movies or YouTube videos that require a channel subscription
PrivateContentException, for private YouTube videos (can be used for other services)
SoundCloudGoPlusContentException, for SoundCloud Go+ tracks
YoutubeMusicPremiumContentException, for YouTube videos restricted to YouTube Music Premium members
I carefully read the contribution guidelines and agree to them.
I have tested the API against NewPipe.
I agree to create a pull request for NewPipe as soon as possible to make it compatible with the changed API. Better error messages for SoundCloud and YouTube unavailable contents NewPipe#5385

Stypox · 2021-01-17T19:31:10Z

That's normal for soundcloud. It happens since there there are no valid audio formats to play

Stypox · 2021-01-17T20:38:55Z

You could do that in another PR and not touch anything about streams here

AudricV · 2021-01-18T17:51:14Z

@Stypox Done. Code is ready for review.

Stypox

Except from that one thing it looks good ;-)

...or/src/main/java/org/schabi/newpipe/extractor/exceptions/GeographicRestrictionException.java

extractor/src/main/java/org/schabi/newpipe/extractor/exceptions/SoundCloudGoPlusException.java

AudricV · 2021-01-20T18:38:58Z

@Stypox Changes requested applied.

Stypox

Looks good, thank you! This will be merged after 0.20.9 is released

TobiGr

Looks good.
Can be merge once the conflict is solved.

AudricV · 2021-02-15T22:16:04Z

@TobiGr Unfortunately, it doesn't work for some videos from type of https://www.youtube.com/watch?v=ayI2iBwGdxw (members-only content), so it shouldn't be merged until this is fixed.

opusforlife2 · 2021-02-19T17:47:16Z

@TiA4f8R You could always do that in a later PR and let this one be merged. 4 out of 5 or 80% is pretty good, no?

AudricV · 2021-02-19T18:19:15Z

@opusforlife2 You're right. Also, when I updated my PR, I added a check and if the reason message is null, NewPipe shows now the default Content Unavailable message. With this check, no Could not get any stream exception is thrown so this bug isn't a blocker and the PR can be merged without something strange.

TobiGr · 2021-02-19T21:03:30Z

Some tests fail, because the exceptions changed. Please fix them

AudricV · 2021-02-19T21:45:16Z

@TobiGr Done. Thanks to letting me know this because it would have broken the extraction of unexisiting YouTube videos. So it was not tests but my code.

rocketinventor · 2021-02-23T15:11:00Z

Hi, I was testing this out with the apk from Pull Request #5385 by @TiA4f8R (which implements these error messages) and I found another type of error that is not on this list - Restricted Mode errors...

If YouTube's restricted mode is enabled in NewPipe and a "restricted" video is opened in the player then a generic "Content unavailable" message will be displayed. Someone who is unaware of this will think that there is an issue with the video extractor.

There should be a way to detect this... Does anybody know how to do this?

TobiGr · 2021-02-23T16:16:16Z

@rocketinventor That might need an exception. However, we detected that before, but after some refactoring, the detection does not work any more. Good catch!
That should be easy to fix in this PR.

I suggest to do the following (@XiangRongLin, @B0pol, @Stypox any other ideas how to solve this?):

in https://github.com/TiA4f8R/NewPipeExtractor/blob/f3c311e5ffb12c0be155a45852e50a6d146d8cf6/extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/extractors/YoutubeStreamExtractor.java#L720-L723, check if we already have a status and save it before fetching the new video info.We should do this, because info from embedded videos is not that detailed and some videos (especially age-restricted ones) cannot be played via the embedded player.
if we did not get a good result with the new info, handle the previously stored message, e.g. in this case set ageLimit to 18. We get almost every info, but streamingData. So we cannot extract the duration, subtitles as well as audio and video streams. This will later throw an exception "Could not get any stream", but at least the info about the stream is displayed.
The front-ends know how to handle this.

For later (maybe not in this PR): we should a add a AgeRestrictedContentException. We will need this in the future once YouTube disabled age restricted videos completely for users who are not logged in. As far as I can see, it is not possible to obtain the streamingData from the given JSON now. I don't know how much info is going to be available after YouTube applied some more changes to age restricted content.

And why the hell are the tests failing again -_-

AudricV · 2021-02-23T16:26:43Z

And why the hell are the tests failing again -_-

It's a SoundCloud intermittent test failure (the search test), so don't worry ;)

rocketinventor · 2021-02-24T09:43:59Z

@TiA4f8R would your method work for opening a video through a direct link, or only from within the app?

Also, just to be clear, there are two levels of age restricted content (in YouTube) - each corresponding with thier own setting in NewPipe:

Potentially Mature Content - Videos
blocked by "Restricted Mode":

Turn on YouTube's "Restricted Mode"
YouTube provides a "Restricted Mode" which hides potentially mature content

Restricted (Mature) Content - Videos unblocked through the following opt-in:

Show age restricted content
Show content possibly unsuitable for children because it has an age limit (like 18+)

If there is a way to detect one of these vs the other, then they should probably each have there own exceptions. If not, the error messages, at least, should be different (depending on which content restriction settings are in place from within NewPipe).

I.e. If a video is blocked while "Restricted Mode" is on then the message maybe should suggest to disable "Restricted Mode" to see the video.

rocketinventor · 2021-02-24T11:15:58Z

I just tested this with the video URL for a "restricted" video from #4869 (same apk as before). The "This video is age restricted." error message definitely does work when "restricted mode" is off.

…ewPipe Extractor to be able to display different error messages This commit adds two new exceptions in NewPipe Extractor: GeographicRestrictionException and SoundCloudGoPlusException (which extend to ContentNotAvailableException). These exceptions allow showing different error messages to user when a content isn't available in his/her/its country (only used for now by SoundCloudStreamExtractor) or when the content is a SoundCloud Go+ track.

These exceptions are thrown on a test with the error messages text, because YouTube returns only "UNPLAYABLE" status in most error cases. Tests are based with English strings, so changing the lang used by extractor will throw the generic exception (ContentNotAvailableException).

…tentException

Also check if related streams are empty if they are expected to be empty.

The ContentNotSupportedException is thrown because no supported audio streams where extracted. However, SoundCLoud does not check, whether there are any streams available. This commit should be reverted in TeamNewPipe#526

AgeRestrictedException will be thrown only if the reason message equals to "Sign in to confirm your age" and if the age limit is 18.

Use final where possible in YoutubeStreamExtractor and do some other code style improvements

AudricV mentioned this pull request Jan 10, 2021

Better error messages for SoundCloud and YouTube unavailable contents TeamNewPipe/NewPipe#5385

Merged

2 tasks

AudricV force-pushed the soundcloud-improvements branch 3 times, most recently from ede34f5 to ea780f2 Compare January 17, 2021 17:49

AudricV force-pushed the soundcloud-improvements branch 3 times, most recently from 639ed9a to fc2734f Compare January 18, 2021 17:49

AudricV changed the title ~~Add two new exceptions to extractor + try to support SoundCloud HLS playback (with a workaround)~~ Add two new exceptions to extractor Jan 18, 2021

AudricV marked this pull request as ready for review January 18, 2021 17:50

AudricV force-pushed the soundcloud-improvements branch from fc2734f to c20d35d Compare January 18, 2021 18:15

Stypox requested changes Jan 18, 2021

View reviewed changes

...or/src/main/java/org/schabi/newpipe/extractor/exceptions/GeographicRestrictionException.java Outdated Show resolved Hide resolved

extractor/src/main/java/org/schabi/newpipe/extractor/exceptions/SoundCloudGoPlusException.java Outdated Show resolved Hide resolved

Stypox approved these changes Jan 21, 2021

View reviewed changes

AudricV force-pushed the soundcloud-improvements branch 2 times, most recently from 6742295 to 2956d8e Compare January 23, 2021 17:02

AudricV force-pushed the soundcloud-improvements branch 2 times, most recently from 35e497c to d27b627 Compare February 5, 2021 18:13

AudricV force-pushed the soundcloud-improvements branch from d27b627 to 716edef Compare February 14, 2021 14:25

AudricV changed the title ~~Add two new exceptions to extractor~~ Add five new exceptions to be able to display different error messages in apps Feb 14, 2021

AudricV force-pushed the soundcloud-improvements branch from 716edef to 646649a Compare February 15, 2021 16:53

TobiGr approved these changes Feb 15, 2021

View reviewed changes

AudricV force-pushed the soundcloud-improvements branch from 646649a to 38a2927 Compare February 19, 2021 17:53

TobiGr added enhancement New feature or request soundcloud service, https://soundcloud.com/ youtube service, https://www.youtube.com/ labels Feb 22, 2021

TobiGr changed the title ~~Add five new exceptions to be able to display different error messages in apps~~ Add new exceptions to be able to display different error messages in apps Feb 24, 2021

TobiGr requested a review from Stypox February 24, 2021 13:28

TobiGr approved these changes Feb 24, 2021

View reviewed changes

AudricV force-pushed the soundcloud-improvements branch from 5499df7 to f62404d Compare February 26, 2021 14:53

AudricV added 6 commits March 5, 2021 16:38

Add tests and rename SoundcloudGoPlusException to SoundcloudGoPlusCon…

3925204

…tentException

Fix YouTube tests

59d6d3f

Fix SoundCloud CreativeCommonsPlaysWellWithOthers test

4e45aef

Fix SoundCloud tests

f15d783

AudricV force-pushed the soundcloud-improvements branch from f62404d to e55284b Compare March 5, 2021 15:38

TobiGr and others added 7 commits March 5, 2021 16:38

Add expectedDescriptionIsEmpty() to DefaultStreamExtractorTest

2115874

Also check if related streams are empty if they are expected to be empty.

Use final in exceptions and add AgeRestrictedException

448b687

AgeRestrictedException will be thrown only if the reason message equals to "Sign in to confirm your age" and if the age limit is 18.

Handle age-restricted videos

da3cfa9

Update PaidContent exception

890cbba

Prevent NullPointerException and use equalsIgnorecase

d0a1041

Code style improvements

e55284b

Use final where possible in YoutubeStreamExtractor and do some other code style improvements

TobiGr merged commit 7e6f464 into TeamNewPipe:dev Mar 7, 2021

AudricV deleted the soundcloud-improvements branch March 7, 2021 15:41

This was referenced Mar 26, 2021

Release 0.21.0 TeamNewPipe/NewPipe#5872

Closed

Release 0.21.0 TeamNewPipe/NewPipe#5871

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add new exceptions to be able to display different error messages in apps #509

Add new exceptions to be able to display different error messages in apps #509

AudricV commented Jan 10, 2021 •

edited by TobiGr

Loading

Stypox commented Jan 17, 2021

Stypox commented Jan 17, 2021

AudricV commented Jan 18, 2021

Stypox left a comment

AudricV commented Jan 20, 2021

Stypox left a comment

TobiGr left a comment

AudricV commented Feb 15, 2021

opusforlife2 commented Feb 19, 2021

AudricV commented Feb 19, 2021

TobiGr commented Feb 19, 2021

AudricV commented Feb 19, 2021 •

edited

Loading

rocketinventor commented Feb 23, 2021

TobiGr commented Feb 23, 2021

AudricV commented Feb 23, 2021

rocketinventor commented Feb 24, 2021

rocketinventor commented Feb 24, 2021

Add new exceptions to be able to display different error messages in apps #509

Add new exceptions to be able to display different error messages in apps #509

Conversation

AudricV commented Jan 10, 2021 • edited by TobiGr Loading

Stypox commented Jan 17, 2021

Stypox commented Jan 17, 2021

AudricV commented Jan 18, 2021

Stypox left a comment

Choose a reason for hiding this comment

AudricV commented Jan 20, 2021

Stypox left a comment

Choose a reason for hiding this comment

TobiGr left a comment

Choose a reason for hiding this comment

AudricV commented Feb 15, 2021

opusforlife2 commented Feb 19, 2021

AudricV commented Feb 19, 2021

TobiGr commented Feb 19, 2021

AudricV commented Feb 19, 2021 • edited Loading

rocketinventor commented Feb 23, 2021

TobiGr commented Feb 23, 2021

AudricV commented Feb 23, 2021

rocketinventor commented Feb 24, 2021

rocketinventor commented Feb 24, 2021

AudricV commented Jan 10, 2021 •

edited by TobiGr

Loading

AudricV commented Feb 19, 2021 •

edited

Loading