SOCKS proxy support

[FiloSottile]

The necessity arose in #171 and even I needed it to debug a geo-blocked video.
There was an attempt on #305 that faced license issues.

I implemented it some time ago with SocksiPy-branch but we don't like external dependencies, do we?

[FiloSottile]

The quick and dirty patch is to install http://code.google.com/p/socksipy-branch/ and put the following after the import statements in utils.py:

import socks
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, HOSTNAME, PORT)
socks.wrapmodule(urllib2)

I can't guarantee that this does not have leaks (actually, I am pretty sure that this might suffer from DNS leaks)

cc: @lenik

[FiloSottile]

For a long-term solution we need either to:

• decide to depend on an external package, upon checking for it etc.
• convince the author of the Initial implementation of SOCKS4/SOCKS5 support #305 thing to release under the Public Domain
• write our implementation (no, actually, maybe no, eh? ;) )

Some reference: http://stackoverflow.com/questions/2317849/how-can-i-use-a-socks-4-5-proxy-with-urllib2

[andreasnuesslein]

I tried @FiloSottile 's #402 (comment) solution. Two pointers:

• You need at least SocksiPy-branch-1.02. 1.01 which is available in pipy doesn't know wrapmodule
• In the current youtube-dl, urllib2 is named compat_urllib_request.. so I included the relevant code around line 170:

import socks
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, 'localhost', 9999)
socks.wrapmodule(compat_urllib_request)

[abourget]

From this page http://www.linuxjournal.com/content/use-ssh-create-http-proxy comments:

Submitted by Kvetch on Wed, 04/22/2009 - 22:46.

If you find a protocol/port blocked by a corp. proxy or just needing to tunnel more than just     simple http/https thru your SOCKS proxy but your app doesn't support SOCKS check out     proxychains. http://sourceforge.net/projects/proxychains/
Install proxychains
Create your ssh SOCKS proxy
# ssh -fqND 5555 thedoc@tardis
Edit the /etc/proxychains.conf
uncomment the "dynamic_chain", comment out the line "strict_chain" and add a line containing the ssh socks5 port "socks5 127.0.0.1 5555"
Then just run your app with the normal cli command but with proxychains prepending the line.
For example to run the command "emerge --sync" (part of everyone's favorite package     management tool, portage) just do this
# proxychains emerge --sync

So this would be proxychains youtube-dl http://your.video.example.com/path/to/vid

[Nowaker]

+1 for this.

[ghost]

I looked for SOCKS support when trying to get round a geo-restricted source. Surprised it isn't already implemented in master to be honest.

[phihag]

@Orbixx If you can point us to a pull request of yours which implements SOCKS, we're happy to merge that into master.

[Don42]

@phihag Does the pull request have to be free of new dependencies or is it ok to depend on socksipy?

[hadifarnoud]

+1 for this one. any updates?

[arioch]

+1

[mateon1]

+1

[agate]

+1

[wenerme]

+1

[wuhaa11]

+1

[NoAnyLove]

+1

[Don42]

Feature is implemented in PR #6353
Please let me know if it works for you. It allowed me to view music videos on youtube from Germany.
Without the socks proxy they were blocked.

[Don42]

I could was not able to build the windows binary with py2exe, so I could not test that.

The feature will not work under windows or anywhere else without the PySocks package installed.
Normal operation of youtube-dl is not hindered, but it will print an error_message when the user tries to use a SOCKS proxy without the package.

I first tried to use the "official" socksipy package, but that is no longer maintained and I needed a newer feature that is only in PySocks to implement a feature @yan12125 wanted.

[yan12125]

@Don42 I guess technically it's possible to bundle PySocks with youtube-dl via py2exe, while there are legal concerns. youtube-dl is released under UNLICENSE, while PySocks is under BSD like license. I'm not sure whether it's OK to bundle them together.

[Don42]

@yan12125 The License of PySocks states that Redistribution is allowed when the "Copyright Notice, list of conditions and the disclaimer is reproduced in the documentation and/or other materials". I don't know what kind of documentation is available for the exe, but I should be possible to add it somewhere in there. Or maybe add a "--license" command and print it when on windows.
You could always ask the developer if that would be enough.

In any case, the way I implemented the feature would allow it to be a Linux/OSX only feature where the user would have to install PySocks if and when they want to use a SOCKS proxy.

[tulsidas]

+1

[limonte]

The same issue here, can't download videos using server in Germany.

[OliverUv]

+1

Using proxychains doesn't work with youtube-dl for me, though it works for almost everything else. Doesn't even work when I do proxychains bash, then youtube-dl inside of that, which is a workaround that works for npm.

Here's the output I get after waiting a few minutes for a timeout: https://gist.github.com/7266d1ee29800f458772

[EvanCarroll]

Love to see this 👍

[Ianvdl]

+1

Has anyone discovered a workaround for this yet?

[Don42]

@Ianvdl Workaround for what exactly? I have a branch that works with SOCKS proxies. It might be a bit old, but it still works. As it probably won't be merged I am not updating it regularly.

[Ianvdl]

@Don42 I'll take a look at your fork sometime to see if I can merge it with the latest version. I find myself having to update often due to youtube's constant changes, so I expect that your version (given that it is outdated) might not be sufficient.

[Don42]

@Ianvdl That is possible. I tested it a few weeks back and it still worked. The changes are quite localized, so it should be easily merged. The only problem was, that it has a dependency and I couldn't add that to the windows exe builder. But it is a soft dependency, when it's not available everything except the SOCKS proxy will work as expected.

[wenerme]

Suggest use cow to convert you socks to http proxy. then

youtube-dl --proxy http://127.0.0.1:7777 -i 'https://www.youtube.com/playlist?list=PLe5-IOvkHavsbjTT1mjsV3ai1I6ZXwotA'

[EvanCarroll]

@zqwroiez that is so fucking cool. You++.

[jab]

Looks like @zqwroiez's comment and account have both been deleted, mysterious. I'll add the comment back as it appears in the GitHub notification email I got:

On Fri, Jan 22, 2016 at 6:24 AM, zqwroiez notifications@github.com wrote:

One could use delegate as a workaround to "convert" a socks proxy into a http proxy:

# Start on port 8080 an http proxy connected to a socks proxy on port 9150
delegated -v -P8080 SERVER=http SOCKS=localhost:9150
# Now download via socks
youtube-dl --proxy http://localhost:8080 https://vimeo.com/44297516

I wanted to try this, but unfortunately there's no formula for DeleGate in homebrew, though a PR was opened in the past: Homebrew/legacy-homebrew#30807. Tried asking the author of that PR what happened but haven't heard back yet. Just thought I'd mention here in case anyone reading would be interested in picking that back up.

[nooop3]

+1

[polyzen]

Not sure how these guys got it working back then, but urllib3 1.14 with PySocks>=1.5.6,<2.0 supports SOCKS.

I've got urllib3 1.14 with PySocks master (which is apparently at 1.5.6), but still getting ERROR: Unable to download webpage: <urlopen error unknown url type: socks5> (caused by URLError('unknown url type: socks5',))

[infinity0]

Just noting here that, recent versions of ffmpeg trigger Debian bug 805741 which means that torsocks youtube-dl goes into a busy loop (that eventually ends with "out of memory") when it tries to run ffmpeg -version.

[yan12125]

I've proposed #9287 based on @bluec0re's socks.py. With that patch commands like this should work:

youtube-dl -v cbjMwKLE-RE --proxy "socks5://127.0.0.1/"

Given your SOCKS proxy listens on 127.0.0.1:1080.

[dstftw]

SOCKS proxy support has been added in youtube-dl 2016.05.10.

[ghost]

youtube-dl --proxy "socks5://127.0.0.1/"
works