Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Twitter problem #99

Closed
wankio opened this issue Aug 28, 2018 · 10 comments
Closed

Twitter problem #99

wankio opened this issue Aug 28, 2018 · 10 comments
Labels
feature-request

Comments

@wankio
Copy link
Contributor

wankio commented Aug 28, 2018 •
edited
Loading

Today when i try to rip some user, I realize it rip all images but not video

https://twitter.com/FappQueens/media

edit : ge/exhentai can't login, it required forum login first
@wankio
Copy link
Contributor Author

wankio commented Sep 17, 2018 •
edited
Loading

when i test, some twitter have 2-6k media files but it only ripped 1000-1900files / 2-6000 total pics....that's because we got limited access in old post right ?....

with 1000 result or less, it rip pretty good and really fast too

@wankio wankio mentioned this issue Sep 26, 2018
Closed
@caoli5288
Copy link

Yes. cannot download video.
Also request this feature.

mikf added a commit that referenced this issue Sep 30, 2018
@mikf
[twitter] add experimental 'videos' option (#99)
f8b3b00 
Enabling this option will detect videos in tweets and output them as
"unsupported" URLs, so that these can then be downloaded with youtube-dl

There are a lot of improvements to be made to the current
implementation, but it works and does what it is supposed to, even if
inefficient as can be ...
@mikf
Copy link
Owner

mikf commented Sep 30, 2018

As a makeshift solution, you can now enable the twitter.videos option to tell the extractor to detect video tweets and output their URLs as unsupported URL and later download them with youtube-dl:

$ gallery-dl --write-unsupported videotweets -o videos=true https://twitter.com/foobar/media
$ youtube-dl --batch-file videotweets

I'll probably add some sort of youtube-dl integration directly into gallery-dl (ref. to #39, #72), but for the time being, that should work as well.

@wankio
Copy link
Contributor Author

wankio commented Oct 3, 2018

nice ty

@mikf
Copy link
Owner

mikf commented Oct 9, 2018

I'm a bit late to mention it, but I implemented a native youtube-dl downloader module: 188876d
If you have it installed, you now only need to do

gallery-dl -o videos=true https://twitter.com/foobar/media

and it will automatically use youtube-dl to download videos.

@caoli5288
Copy link

can this option persist in gallery-dl.conf?

@mikf
Copy link
Owner

mikf commented Oct 10, 2018

Of course. -o is just a shortcut to set/overwrite config file options from the command-line.

@wankio
Copy link
Contributor Author

wankio commented Oct 11, 2018 •
edited
Loading

oh nice :) tested and worked...if youtube-dl can built-in so i think with instagram, you can use this https://github.com/rarcega/instagram-scraper/

@caoli5288
Copy link

caoli5288 commented Oct 11, 2018 •
edited
Loading

@mikf Can shown progress while downloading videos? Long silence is disturbing. :)

@mikf
Copy link
Owner

mikf commented Nov 17, 2018

You can show youtube-dl's output and progress meter by setting downloader.ytdl.logging to false.
There are now also some more options that might be of interest:

youtube-dl integration and video downloads, at least for Twitter and a few more sites, is done, so I'm closing this.

@mikf mikf closed this as completed Nov 17, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature-request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@caoli5288 @mikf @wankio