Gcloud::Storage::File#signed_url does not work when using Google Cloud SDK credentials

[premist]

I tried using Gcloud::Storage::File#signed_url to generate object url with query strings, but I can't get it to work. I used the following code:

s = Gcloud.storage "my-project-name"
buk = s.find_bucket "my-bucket"
file = buk.find_file "myfile.txt"
file.signed_url

It returns the following error:

[10] pry(main)> file.signed_url
NoMethodError: undefined method `sign' for nil:NilClass
from /Users/premist/.rbenv/versions/2.2.2/lib/ruby/gems/2.2.0/gems/gcloud-0.1.1/lib/gcloud/storage/file.rb:502:in `signed_url'

I tried to investigate this and found out that signing_key on Gcloud::Credentials is not being set properly.

I used gcloud auth credential from my computer initially, and I tried to explicitly provide keyfile for authentication too, but it didn't work.

[blowmage]

Hey @premist, the File#signed_url method expects that the Signet::OAuth2::Client object used for authenticating returns a valid object when calling it's signing_key method. According to the documentation, signing_key can return a OpenSSL::PKey (which we expect), or a string (which we don't expect yet, but I'm not sure we will see since we create a OpenSSL::PKey object). But in your case nil is being returned, which is very strange.

I am having a hard time reproducing this. Are you available to pair remotely so I can see what you are doing in hopes of creating a reliable reproduction?

[premist]

I will try to replicate this on clean Linux machine soon and let you know. If this works properly on clean environment, there should be some misconfiguration on my machine which causes the issue.

[blowmage]

@premist Any luck reproducing this?

[premist]

Hi @blowmage, sorry for late reply. Will try this later today and I'll let you know how it goes.

[blowmage]

Thanks! Looking forward to it. :)

[blowmage]

Hey @premist, when you provide the keyfile explicitly, are you giving the absolute path to the file? e.g /Users/premist/gcloud.json and not ~/gcloud.json?

[premist]

I created a clean Ubuntu (15.04 x64) VM, and did the following:

• Install Ruby 2.2.2 via rbenv
• Install Google Cloud SDK
• Used $ gcloud auth login to sign in
• Install gcloud gem

I tried the code above again and it failed. Seems like signing_key doesn't work as intended when gcloud gem authenticates using Cloud SDK credentials.

[blowmage]

@premist Thanks! You also said:

I used gcloud auth credential from my computer initially, and I tried to explicitly provide keyfile for authentication too, but it didn't work.

I can't get it to fail when explicitly providing a keyfile, but I assume that the path you provided didn't resolve and so it fell back to the cloud SDK credentials. Any chance you can confirm?

[premist]

I tried providing JSON keyfile explicitly to Gcloud.storage, and it worked. Providing an invalid path like ~/Downloads/keyfile.json raised a proper error, which is the following:

RuntimeError: The keyfile '~/Downloads/keyfile.json' is not a valid file.

I'm not sure why I was unable to get signed url when I provided keyfile explicitly, but seems like it works as intended.

[blowmage]

It looks like that the code that derives credentials from the cloud SDK doesn't return "service account" credentials, which is missing the private key needed to sign the URLs. Right now we're trying to determine how we can convert "authorized user" credentials to "service account" credentials.

[premist]

I found something that might be related to this on gsutil documentation. In order to use gsutil signurl command, you need to explicitly provide private key file regardless of Google Cloud SDK authentication.

I checked gcloud-python and gcloud-node, seems like those libraries don't support authentication by Cloud SDK. Also I couldn't find a code which converts 'authorized user' credentials to 'service account' credentials.

[jgeewax]

I'm pretty sure both node and python will pull the creds set by the cloud SDK. CC @stephenplusplus @dhermes to comment.

[dhermes]

Yes, we have a tight integration with https://github.com/google/oauth2client and don't implement much of the signing ourselves, just proxy it out to the auth library.

[stephenplusplus]

Same as @dhermes said, gcloud-node's auth library is https://github.com/google/google-auth-library-nodejs which respects the cloud sdk session.

[premist]

Ah, thanks for the correction. Then it should be possible to generate signed url using Cloud SDK credentials.

[blowmage]

@dhermes @stephenplusplus Do your auth libraries give you credentials with the private key when authenticating with the cloud SDK? Or do you have a way to exchange the "authorized user" credentials for "service account" credentials?

[blowmage]

The signed url documentation is clear about requiring service account authentication for signing the url. We need the private key to sign the url.

[stephenplusplus]

No, haven't found a way to do this yet: googleapis/google-cloud-node#211

[dhermes]

There is no way to sign with "authorized user" credentials, it must be a service account. Our code paths fail on non-service account credentials (as well as on GCE service account credentials).

[blowmage]

@stephenplusplus Ah, thanks for the link. I had missed than in my searches. Makes sense.

@dhermes Thank you for the confirmation.

@premist I have some ideas. I'll try to get a PR up today.