Google Storage XML head_object doesn't pass idempotent flag

[simonschellaert]

I'm developing an application where we make heavy use of the Fog Google Storage XML API (fog-google 1.7.1). In particular, we are issuing HEAD requests for thousands of objects. While running this application, I'm seeing intermittent Excon::Error::Socket: end of file reached (EOFError) errors.

I was able to reproduce this issue in my Rails console as follows:

1. Create a new connection.

connection = Fog::Storage.new(
  provider: 'Google',
  google_project: 'XXX',
  google_storage_access_key_id: 'YYY',
  google_storage_secret_access_key: 'ZZZ,
)

2. Issue a HEAD request as follows: connection.head_object(bucket_name, file_name). This request should succeed.

3. Wait ten minutes or longer. (A shorter interval may also work but I didn't test that.)

4. Issue the same HEAD request again: connection.head_object(bucket_name, file_name).

5. I now receive the aforementioned Excon::Error::Socket: end of file reached (EOFError).

I was able to resolve the issue by passing the option persistent: false when creating the connection. While analyzing the code of fog-google, I noticed that the option :idempotent => true is not passed in head_object, while that option is passed in get_object, put_object and delete_object (see below). Since that option effectively signals that Excon may retrieve the request (if the connection timed out for example), I'm guessing that might be the cause of the failures I'm seeing. I personally see no reason why head_object shouldn't be idempotent, so I think this flag could safely be added. Is this correct? Thanks in advance for the feedback.

fog-google/lib/fog/storage/google_xml/requests/get_object.rb

Line 47 in 6ca3d33

:idempotent => true,

fog-google/lib/fog/storage/google_xml/requests/put_object.rb

Line 32 in 6ca3d33

:idempotent => true,

fog-google/lib/fog/storage/google_xml/requests/delete_object.rb

Line 18 in 6ca3d33

:idempotent => true,

[RigoOnRails]

@Temikus Hi! What do you think about this? Would love to see this working.

[Temikus]

@RigoTheDev I've staged the change in: #498

Can you or @simonschellaert check if it resolves your issue? Just bundle it from git or rake build for a local gem.

[RigoOnRails]

@Temikus Seems to be working! Thank you!

[Temikus]

@RigoTheDev Sweet. Thanks! I'll merge shortly.