Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How do add cookies? #661

Closed
harshmandan opened this issue Jul 1, 2020 · 17 comments
Closed

How do add cookies? #661

harshmandan opened this issue Jul 1, 2020 · 17 comments
Labels

Comments

@harshmandan
Copy link

harshmandan commented Jul 1, 2020

How do I add cookies, extracted from chrome (cookies.txt). Also how do I verify that the cookies are working?

@fent
Copy link
Owner

fent commented Jul 1, 2020

ytdl(url, {
  requestOptions: {
    headers: {
      Cookie: 'a=1;b=2'
    }
  }
});

@fent fent closed this as completed Jul 1, 2020
@fent fent added the question label Jul 1, 2020
@harshmandan
Copy link
Author

harshmandan commented Jul 2, 2020

request options is not available for ytdl.getInfo(). I just want the info but without cookies it just keeps getting 429s.
Also this is my cookie file extracted using cookie.txt:

# Netscape HTTP Cookie File
# This file is generated by youtube-dl.  Do not edit.

[redacted]

So, according to your answer my cookie would be like this?

{Cookie: 'APISID: 82OXkGHhjizKHEU788ABYZK3ldt_1iJREGF; SAPISID:XYA2VbHdaaFgfIjRlbG-/AQYu_EU4y1UYVLi ..........'

@fent
Copy link
Owner

fent commented Jul 2, 2020

request options is not available for ytdl.getInfo()

it is available, it takes similar options as calling ytdl()

Also this is my cookie file extracted using cookie.txt:

I've edited your post. be careful about posting sensitive information, even cookies. someone could login to your account with that.

So, according to your answer my cookie would be like this?

use = instead of :. and no spaces between key and value

@harshmandan
Copy link
Author

Okay! Thanks for the info but it still isn't working.
Now it throws this error:

Error: Error parsing info: JSON.parse(...).reduce is not a function
at exports.getBasicInfo (/home/live/node_modules/ytdl-core/lib/info.js:43:11)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async Object.exports.<computed> [as getBasicInfo] (/home/live/node_modules/ytdl-core/lib/info.js:296:18)
at async exports.getFullInfo (/home/live/node_modules/ytdl-core/lib/info.js:187:14)
at async Object.exports.<computed> [as getFullInfo] (/home/live/node_modules/ytdl-core/lib/info.js:296:18)

I tried with this code:

ytdl.getInfo(req.body.vid, { requestOptions: 
                { 
                    headers: {
                        Cookie: 'APISID=secure;LOGIN_INFO=secure;PREF=f1=50000000&hl=en;SAPISID=secure;SID=secure;SIDCC=secure;SSID=secure;VISITOR_INFO1_LIVE=secure;__Secure-3PAPISID=sec;__Secure-3PSID=sec;__Secure-APISID=sec;__Secure-HSID=sec;__Secure-SSID=sec;s_gl=sec'
                    }
                }
            }, (err, info) => {
                      console.log(info);
            });

@fent
Copy link
Owner

fent commented Jul 2, 2020

try adding a space after each ;. all examples i've seen of cookies have them like that

key=value; another_key=another_value; more=stuff;

@harshmandan
Copy link
Author

Still nothing. I logged the body in info.js at 43rd line. The body has this output:

{"reload":"now"}

@fent
Copy link
Owner

fent commented Jul 5, 2020

hmm, I'll have to investigate this. this was also noticed in this issue #632 (comment)

@fent
Copy link
Owner

fent commented Jul 5, 2020

I tried using cookies myself, everything worked, didn't get {"reload":"now"}

@harshmandan
Copy link
Author

harshmandan commented Jul 5, 2020

Alright. I think I can tell you how to replicate this. I generated my cookie file using the cookie.txt extension in chrome.
Cleared cookies, logged in into youtube, extracted the cookie file using extension.

Then I generated this string:

APISID=sec; LOGIN_INFO=sec; PREF=f1=50000000&hl=en; SAPISID=sec; SID=sec; SIDCC=sec; SSID=sec; VISITOR_INFO1_LIVE=sec; __Secure-3PAPISID=sec; __Secure-3PSID=sec; __Secure-APISID=sec; __Secure-HSID=sec; __Secure-SSID=sec; s_gl=sec'

sec is where my secret keys go.

Used these keys in my code as:

ytdl.getInfo(req.body.vid, { requestOptions: 
                { 
                    headers: {
                        Cookie: cookieString
                    }
                }
            }, (err, info) => {
                if (err) {
                    console.log(err);
                    return res.status(422).send({code:422, status:"err", err:err});
                }
                res.json({code:200, status:"success", info});
            });

When I run this code, its throws this error:

Error: Error parsing info: JSON.parse(...).reduce is not a function
at exports.getBasicInfo (/home/live/node_modules/ytdl-core/lib/info.js:43:11)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async Object.exports.<computed> [as getBasicInfo] (/home/live/node_modules/ytdl-core/lib/info.js:296:18)
at async exports.getFullInfo (/home/live/node_modules/ytdl-core/lib/info.js:187:14)
at async Object.exports.<computed> [as getFullInfo] (/home/live/node_modules/ytdl-core/lib/info.js:296:18)

To verify the cookie file. I've used the ytdl command line utility and it works fine.

Can you verify if the cookies you used were obtained using the same process and/or are an youtube account's cookies?

Thanks!


Edit 1:

I tried another method to extract cookies here and add it to header.
Added logs to the info.js file and logged url (line 37), requestOptions (lines 37) and body (line 41).
I got this output:

url: https://www.youtube.com/watch?v=C1uR4vC0Z5g&hl=en&bpctr=1593938137&pbj=1

reqOptions: {
    headers: {
        Cookie: 'String extracted from browser',
        'x-youtube-client-name': '1',
        'x-youtube-client-version': '2.20191008.04.01'
        }
    }

body: {"reload":"now"}

Error: Error parsing info: JSON.parse(...).reduce is not a function
at exports.getBasicInfo (/home/live/node_modules/ytdl-core/lib/info.js:43:11)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async Object.exports.<computed> [as getBasicInfo] (/home/live/node_modules/ytdl-core/lib/info.js:296:18)
at async exports.getFullInfo (/home/live/node_modules/ytdl-core/lib/info.js:187:14)
at async Object.exports.<computed> [as getFullInfo] (/home/live/node_modules/ytdl-core/lib/info.js:296:18)

Edit 2:
Took a look at the issue #632 and the temp fix for it as well. But removing the cookie header is not an option for me as I'm using a server to extract info and have a fixed IP which gets blocked (429s) very quickly if I don't use cookies.

@fent
Copy link
Owner

fent commented Jul 5, 2020

ok, I was able to replicate it by using the cookies.txt extension to extract cookies, and I did get a { "reload": "now" } response. but I think that could have to do with giving a bad cookie header.

if I try the 2nd method you posted, I don't get an error and all the requests go through fine. but, I'm also not sure if I'm really logged in.

@harshmandan
Copy link
Author

Okay! I can understand if the first method doesn't work. But I'm getting error with the second method as well.

Just to confirm if this is the procedure you followed.

  • Open Incognito/Private session.
  • Head to youtube and log in
  • You'll be redirected to youtube homepage.
  • Open dev tools and head to network tab.
  • Now refresh the page again.
  • Copy Cookie from REQUEST Header section from the first request (It should show up as www.youtube.com)
  • Use that header with YTDL

@fent
Copy link
Owner

fent commented Jul 6, 2020

I found a way, it needs the x-youtube-identity-token header. you can find this in the watch page's source if you search for ID_TOKEN. but I'll make a change so that ytdl-core looks for it if the cookie header is set

fent added a commit that referenced this issue Jul 7, 2020
@mh4ck
Copy link

mh4ck commented Jul 14, 2020

Even with the x-youtube-identity-token header i get the parsedBody.reduce is not a function error

I wrote a small and dirty nightmare.js script which extract the values from YouTube:

const Nightmare = require("nightmare");
const fs = require("fs");
const request = require("request");

const nightmare = Nightmare({
    show: true,
    height: 768,
    width: 1024
});

nightmare
.useragent("Chrome")
.goto("https://www.youtube.com/")
.wait('a[href*="accounts.google.com"]')
.click('a[href*="accounts.google.com"]')
.wait(10000)
.type('#Email', '[your email]')
.click('#next')
.wait(10000)
.type('input[type=password]', '[your password]' + '\u000d')
.wait(10000)
.useragent("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36")
.goto("https://www.youtube.com/watch?v=AhaB4KEUGW0")
.wait('#avatar-btn')
.evaluate(() => {
    var ret = {cookie: document.cookie, cv: null};
    var cv_regex = /innertubeContextClientVersion":\"([^\"]+)/gm;
    var str = document.body.innerHTML;
    var m = cv_regex.exec(str);
    if(m && m[1]) {
        ret.cv = m[1];
    }

    var headHTML = document.head.innerHTML;
    var id_regex = /ID_TOKEN":\"([^\"]+)/gm;
    var idM = id_regex.exec(headHTML);
    if(idM && idM[1]) {
        ret.id = idM[1];
    }
    return ret;
})
.end()
.then((ret) => {
    let cookie = ret.cookie;
    console.log(cookie);
    let clientVersion = ret.cv;
    console.log(clientVersion);
    let idToken = ret.id;
    console.log(idToken);

    request.post("[send to your address where you need it]", {
        form: {
            cookie: cookie,
            clientVersion: clientVersion,
            idToken: idToken
        }
    },
    (err, resp, body) => {
        if(err) {
            console.log(err);
            return;
        }

        if(resp.statusCode != 200) {
            console.log("[Error]: Received status code: " + resp.statusCode);
            return;
        }

        console.log("[Info]: Cookie send successfull to Server!");
    });
})
.catch((err) => {
    console.log(err);
});

I'm sending now these headers to the info request

  • cookie
  • x-youtube-client-name
  • x-youtube-client-version
  • x-youtube-identity-token

And get the error "Error parsing info: parsedBody.reduce is not a function"

Am i doing smth. wrong? @fent

@fent
Copy link
Owner

fent commented Jul 14, 2020

I'm sending now these headers to the info request

  • cookie
  • x-youtube-client-name
  • x-youtube-client-version
  • x-youtube-identity-token

are you defining those yourself in the options to getInfo? or letting ytdl-core add them?

@mh4ck
Copy link

mh4ck commented Jul 15, 2020

I'm sending now these headers to the info request

  • cookie
  • x-youtube-client-name
  • x-youtube-client-version
  • x-youtube-identity-token

are you defining those yourself in the options to getInfo? or letting ytdl-core add them?

I define them myself in the headers, if i let it do itself it says that it can't find the identity token.

@fent
Copy link
Owner

fent commented Jul 15, 2020

does the cookie work for you in your browser? are you using a proxy or a custom user-agent?

@mh4ck
Copy link

mh4ck commented Jul 15, 2020

Yes it works in my browser, while login i'm usung the UA 'Chrome' otherway the login is not working.

I'm using the script above to generate/read the header values.

Then i set them in getInfo requestOptions...

Edit: Yes sure i'm using more than 1000 different proxies - for login i'am using no proxy. Then i send the cookie with the getInfo request and a proxy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants