Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fixes #180] Allow for root-level arrays to be querystring-ed in Modem.prototype.buildQuerystring #181

Merged
merged 5 commits into from
Jan 13, 2025

Conversation

Clovel
Copy link
Contributor

@Clovel Clovel commented Jan 8, 2025

This fixes the behaviour of Modem.prototype.buildQuerystring for root-level arrays.

It fixes these issues :

I have launched the tests on docker-modem and dockerode (by setting my fork as the source for NPM) and they all pass. Versions are :

Windows :

  • Docker version 27.3.1, build ce12230
  • Docker Compose version v2.29.2-desktop.2
  • Tested using NodeJS 14

Let me know if anything more needs to be checked.

@Clovel Clovel changed the title Allow for root-level arrays to be querystring-ed in Modem.prototype.buildQuerystring [Fixes #180] Allow for root-level arrays to be querystring-ed in Modem.prototype.buildQuerystring Jan 8, 2025
Signed-off-by: Clovis Durand <clovis.durand@loginline.com>
@Clovel
Copy link
Contributor Author

Clovel commented Jan 8, 2025

I tried running the GitHub Actions on my fork and there seems to be an issue with Node 18 & Node 22 because of sshfs. This seems unrelated to my changes.

Node 14 passed.

@Clovel
Copy link
Contributor Author

Clovel commented Jan 11, 2025

It seems that all GitHub Actions have passed over on this repo ! that's good news

@apocas apocas merged commit f1331a0 into apocas:master Jan 13, 2025
@Clovel
Copy link
Contributor Author

Clovel commented Jan 15, 2025

Hi @apocas,

Now that this is merged, when can we expect a release tag to be published to NPM with this fix ? Is a release scheduled soon ?

@apocas
Copy link
Owner

apocas commented Jan 16, 2025

Published v5.0.6

if (
opts[key] &&
typeof opts[key] === 'object' &&
!Array.isArray(opts[key])
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Clovel @apocas This change is quite aggressive, it goes for using query serialization for only t and extrahosts to using it for any Array option.

This doesn't seem to be universally true. At least I can see this breaks the cachefrom on Image.build that expects a JSON serialized array and otherwise it errors with

(HTTP code 400) unexpected - error reading cache-from: invalid character 'a' looking for beginning of value

I confirmed the issue does not happen with docker-modem v5.0.3, prior to this change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would seem that the serialization needs to be more standard indeed. Not all routes behave the same (or at least I haven't found a common denominator).

It does fix the usage of ENV variables and other parameters of the /images/create route though, which was my issue.

Also unit tests pass, so a case seems to be missing for your issue.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the Moby code, t and extrahosts are read using Request.Form (see lines 55-56), while cachefrom uses Request.FormValue (see line 143). I don't know enough about Go to understand why the data is interpreted differently, but that could give us a clue.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the docker API is not very consistent with how it accepts array parameters.

  • cachefrom in ImageBuild expects a stringified "JSON array of images"
  • changes in ImageCreate expects an array of strings.

I'll open up an issue and possibly a PR

pipex added a commit to pipex/dockerode that referenced this pull request Feb 24, 2025
Docker [ImageBuild](https://docs.docker.com/reference/api/engine/version/v1.48/#tag/Image/operation/ImageBuild) endpoint requires
that the `cachefrom` option is a "JSON serialized array". This
requirement seems to be an exception to the way arrays are consumed by
the API (usually querystring serialized).

This change is needed after apocas/docker-modem#181 that makes querystring serialized arrays the default.

This commit adds an exception for this option on `Docker.buildImage`.

Change-type: patch
pipex added a commit to pipex/dockerode that referenced this pull request Feb 24, 2025
Docker [ImageBuild](https://docs.docker.com/reference/api/engine/version/v1.48/#tag/Image/operation/ImageBuild) endpoint requires
that the `cachefrom` option is a "JSON serialized array". This
requirement seems to be an exception to the way arrays are consumed by
the API (usually querystring serialized).

This change is needed after apocas/docker-modem#181 that makes querystring serialized arrays the default.

This commit adds an exception for this option on `Docker.buildImage`.

Change-type: patch
Closes: apocas#792
pipex added a commit to pipex/dockerode that referenced this pull request Feb 24, 2025
Docker [ImageBuild](https://docs.docker.com/reference/api/engine/version/v1.48/#tag/Image/operation/ImageBuild) endpoint requires
that the `cachefrom` option is a "JSON serialized array". This
requirement seems to be an exception to the way arrays are consumed by
the API (usually querystring serialized).

This change is needed after apocas/docker-modem#181 that makes querystring serialized arrays the default.

This commit adds an exception for this option on `Docker.buildImage`.

Change-type: patch
Closes: apocas#792
pipex added a commit to balena-io-modules/balena-compose that referenced this pull request Feb 24, 2025
A change the way options are serialized in docker-modem causes an issue
with builds using the `cachefrom` option and library tests to fail.
This change can be reverted if apocas/dockerode#793 is merged.

Change-type: patch
Relates-to: apocas/docker-modem#181
Relates-to: apocas/dockerode#792
pipex added a commit to balena-io-modules/balena-compose that referenced this pull request Feb 25, 2025
A change the way options are serialized in docker-modem causes an issue
with builds using the `cachefrom` option and library tests to fail.
This change can be reverted if apocas/dockerode#793 is merged.

Change-type: patch
Relates-to: apocas/docker-modem#181
Relates-to: apocas/dockerode#792
pipex added a commit to balena-io-modules/mocha-pod that referenced this pull request Feb 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants