Weird project issue with Cloud Vision API?

[jgeewax]

For some reason it looks like Gcloud.vision is detecting my project as google.com:cloudsdktool rather than the one set in my environment (jjg-cloud-research).

Any chance this is a bug somewhere?

$ echo $GCLOUD_PROJECT
jjg-cloud-research
$ irb
2.3.0 :002 > require "gcloud"
 => true 
2.3.0 :003 > vision = Gcloud.new.vision
 => #<Gcloud::Vision::Project:0x000000019ec248 @connection=Gcloud::Vision::Connection(jjg-cloud-research)> 
2.3.0 :004 > image = vision.image "gs://jjg-cloud-research/cage.jpg"
 => #<Gcloud::Vision::Image (url: gs://jjg-cloud-research/cage.jpg)> 
2.3.0 :005 > image.face
Gcloud::Vision::ApiError: Google Cloud Vision API has not been used in project google.com:cloudsdktool before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/vision.googleapis.com/overview?project=google.com:cloudsdktool then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
    from /usr/local/google/home/jjg/.rvm/gems/ruby-2.3.0/gems/gcloud-0.10.0/lib/gcloud/vision/project.rb:261:in `annotate'
    from /usr/local/google/home/jjg/.rvm/gems/ruby-2.3.0/gems/gcloud-0.10.0/lib/gcloud/vision/image.rb:131:in `faces'
    from /usr/local/google/home/jjg/.rvm/gems/ruby-2.3.0/gems/gcloud-0.10.0/lib/gcloud/vision/image.rb:142:in `face'
    from (irb):5
    from /usr/local/google/home/jjg/.rvm/rubies/ruby-2.3.0/bin/irb:11:in `<main>'      
2.3.0 :006 > vision.project
 => "jjg-cloud-research" 
2.3.0 :007 > 

[blowmage]

Can you use the other services with those environment variables? Or is this only affecting Vision?

[blowmage]

Oh, I think I know what is going on. That cloudsdk is the default authentication project id when using the gcloud cli credentials. The Vision API calls aren't scoped to the provided project id like every other API, so I bet the API is deriving the project id to charge the API call to from the credentials provided. In this case it is getting the cloudsdk project id.

Do you have a GCLOUD_KEYFILE environment variable set? I bet if you used a JSON keyfile instead of the gcloud cli with it will work.

[jgeewax]

Doesn't seem to be a problem with Datastore, just Vision.

$ irb
2.3.0 :001 > require "gcloud"
 => true 
2.3.0 :002 > gcloud = Gcloud.new
 => #<Object:0x000000024af8d8 @project=nil, @keyfile=nil> 
2.3.0 :003 > gcloud = Gcloud.new "jjg-cloud-research"
 => #<Object:0x000000024934a8 @project="jjg-cloud-research", @keyfile=nil> 
2.3.0 :004 > datastore = gcloud.datastore
 => #<Gcloud::Datastore::Dataset:0x00000002f12c08 @service=Gcloud::Datastore::Service(jjg-cloud-research)> 
2.3.0 :005 > key = datastore.key "Task" 1234
SyntaxError: (irb):5: syntax error, unexpected tINTEGER, expecting end-of-input
    from /usr/local/google/home/jjg/.rvm/rubies/ruby-2.3.0/bin/irb:11:in `<main>'
2.3.0 :006 > key = datastore.key "Task", 1234
 => #<Gcloud::Datastore::Key:0x00000002e87a68 @kind="Task", @id=1234, @project=nil, @namespace=nil> 
2.3.0 :007 > datastore.find key
 => nil 
2.3.0 :008 > vision = gcloud.vision
 => #<Gcloud::Vision::Project:0x00000003005a48 @connection=Gcloud::Vision::Connection(jjg-cloud-research)> 
2.3.0 :009 > image = vision.image "gs://jjg-cloud-research/cage.jpg"
 => #<Gcloud::Vision::Image (url: gs://jjg-cloud-research/cage.jpg)> 
2.3.0 :010 > image.face
Gcloud::Vision::ApiError: Google Cloud Vision API has not been used in project google.com:cloudsdktool before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/vision.googleapis.com/overview?project=google.com:cloudsdktool then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
    from /usr/local/google/home/jjg/.rvm/gems/ruby-2.3.0/gems/gcloud-0.10.0/lib/gcloud/vision/project.rb:261:in `annotate'
    from /usr/local/google/home/jjg/.rvm/gems/ruby-2.3.0/gems/gcloud-0.10.0/lib/gcloud/vision/image.rb:131:in `faces'
    from /usr/local/google/home/jjg/.rvm/gems/ruby-2.3.0/gems/gcloud-0.10.0/lib/gcloud/vision/image.rb:142:in `face'
    from (irb):10
    from /usr/local/google/home/jjg/.rvm/rubies/ruby-2.3.0/bin/irb:11:in `<main>'

[jgeewax]

OK -- so you're saying that this whole "we don't know who to bill" problem is cropping up here as well? Even if I clearly state the project ID, it still uses the object assigned to the service account?

[jgeewax]

If that's the case, we need a much better error than "The Cloud SDK's project isn't activated! Go activate it!"

We should tell people they need to generate a service account and use that.

[blowmage]

We clearly state the project id when using gcloud-ruby, but the actual API call doesn't have that info. Other APIs will post to paths like /datastore/v1beta3/<project_id>/dataset.run_query, but vision posts to /v1/annotate, and the project id is not included in the request payload.

I agree that we should add some documentation that the cli auth credentials won't work for Vision. (Unless we can find a way to add the project id to the API call somehow.)

[jgeewax]

Yep - I'm arguing this out on the API side. In the meantime, if we detect using credentials for google.com:cloudsdk, we should give a good error description.

[ohadperry]

problem solved for me using
export GOOGLE_APPLICATION_CREDENTIALS=personal-gmail-account.json

[loretoparisi]

I have used the GOOGLE_APPLICATION_CREDENTIALS as it follows

$ export GOOGLE_APPLICATION_CREDENTIALS=/Users/admin/.config/gcloud/application_default_credentials.json

But this did not solve my problem

$ curl -s -k -H "Content-Type: application/json" -H "Authorization: Bearer $GCLOUD_ACCESSTOKEN" https://speech.googleapis.com/v1beta1/speech:syncrecognize -d @sync-request.json
{
  "error": {
    "code": 403,
    "message": "Google Cloud Speech API has not been used in project google.com:cloudsdktool before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/speech.googleapis.com/overview?project=google.com:cloudsdktool then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.",
    "status": "PERMISSION_DENIED",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.Help",
        "links": [
          {
            "description": "Google developers console API activation",
            "url": "https://console.developers.google.com/apis/api/speech.googleapis.com/overview?project=google.com:cloudsdktool"
          }
        ]
      }
    ]
  }
}

It still hangs on the default project, but I have set the project env too:

$ export GCLOUD_PROJECT=$MYPROJECT
$ gcloud config set project "$MYPROJECT"

[blowmage]

I suspect #975 may help this issue as well.

[BSalita]

Note to myself when using Natural Language API. Make sure project IS_ACTIVE:

gcloud config configurations list

If not, set project:

gcloud config set project MY_PROJECT

Obtain Bearer token:

gcloud auth application-default print-access-token

No need to use environment variable GOOGLE_APPLICATION_CREDENTIALS

[orcaman]

Had the same issue with Speech API just now.

export GOOGLE_APPLICATION_CREDENTIALS=my_creds.json

solved it for me as well.

[BSalita]

Windows batch file which obtains a token.

call gcloud auth application-default print-access-token > gcloudAuthAccessToken.txt
echo on
set /p AccessToken=<gcloudAuthAccessToken.txt
curl -s -k -H "Content-Type: application/json" -H "Authorization: Bearer %AccessToken%"
https://language.googleapis.com/v1/documents:analyzeSyntax -d "@analyzeSyntax.json"

[swcloud]

@jgeewax Is this still an issue?

[dorinionescu]

for simple command line projects use

set GOOGLE_APPLICATION_CREDENTIALS=E:\key\Apia_Bucuresti-e471c5971f29.json ( or "export" on linux)

For complex projects use direct java control

	GoogleCredential credential = GoogleCredential
	          .fromStream(new FileInputStream( "key/Apia_Bucuresti-e471c5971f29.json"))
	          .createScoped( VisionScopes.all());

where json creation and API activation are described here Google Vision API Doc

Enjoy !

[blowmage]

I've verified that this is still an issue when using the gcloud SDK when authenticating with a user's personal account. The ruby Vision code detects the project_id from the environment variable, but the credentials that API calls use are recognizing the project that the personal user account authenticated with, which is google.com:cloudsdktool. The google-cloud-resource-prefix header is sent with the project_id detected from the environment variable, but the API doesn't use the header value for billing.

Even specifically setting the project in the gcloud SDK doesn't affect the project the user's credentials are configured to.

$ gcloud config set project nodal-almanac-725

Nor does setting the gcloud SDK authentication using a service account appear to help matters:

$ gcloud auth activate-service-account ...

IMO the google-cloud-ruby authentication recommendations should remain unchanged. Users should configure the GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_KEYFILE environment variables with service account credentials.

The google-cloud-ruby code does not alter the credentials in any way, so I don't see how google-cloud-ruby can alter this behavior.

[jamiegalbreath]

I am getting this Issue anyone get it solved?

[blowmage]

This was solved by setting a default header in all API calls, but the Vision client has changed recently and this was one of the things that was removed. To do this now you need to specify the header values in the call options:

require "google/cloud/vision"

image_annotator_client = Google::Cloud::Vision::ImageAnnotator.new

image = {
  source: { 
    gcs_image_uri: "gs://gapic-toolkit/President_Barack_Obama.jpg" 
  }
}
features = [{ type: :FACE_DETECTION }]
requests = [{ image: image, features: features }]

options = Google::Gax::CallOptions.new(
  kwargs: { 
    "google-cloud-resource-prefix" => "projects/my-project-id"
  }
)

response = image_annotator_client.batch_annotate_images(
  requests,
  options: options
)