Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

http-client dag.put failed with invalid byte #3914

Closed
jtsmedley opened this issue Oct 7, 2021 · 4 comments · Fixed by #3917
Closed

http-client dag.put failed with invalid byte #3914

jtsmedley opened this issue Oct 7, 2021 · 4 comments · Fixed by #3917
Assignees
Labels
kind/maintenance Work required to avoid breaking changes or harm to project's status quo P1 High: Likely tackled by core team if no one steps up

Comments

@jtsmedley
Copy link

  • Version: ipfs-http-client 53.0.1, go-ipfs v0.10.0
  • Platform: Windows
  • Subsystem:

Severity: Medium

Description: Running await ipfsClient.dag.put({"hello": "world") fails with an error of [Uncaught HTTPError: Invalid byte while expecting start of value: 0x5a]. Issue was not present before IPFS Desktop updated to 0.10.0

Steps to reproduce the error: Connect to local ipfs-go instance and attempt to put a DAG of {"hello": "world"}.

@jtsmedley jtsmedley added the need/triage Needs initial labeling and prioritization label Oct 7, 2021
@welcome
Copy link

welcome bot commented Oct 7, 2021

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review.
In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment.
Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

  • "Priority" labels will show how urgent this is for the team.
  • "Status" labels will show if this is ready to be worked on, blocked, or in progress.
  • "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

@rvagg
Copy link
Member

rvagg commented Oct 8, 2021

@achingbrain the DAG API is now out of sync between go-ipfs and js-ipfs-http-client and errors like this are going to start popping up. This particular error is going to be about the codec defaults, and what's more, the arguments have changed. format and input-enc are now replaced with store-codec and input-codec and both the defaults and behaviour are different, in subtle (but arguably much more clear, now) ways.

The default input-enc here is raw and format is dag-cbor and there's pre-processing done here before it's sent off to the server. So I think what's happening for this error is the JS object {hello: "world"} is encoded with dag-cbor and sent as bytes to the server. But the default input-codec is dag-json on the server and its receiving cbor bytes, so it's getting 0x5a when it really wants a valid JSON value character.

Defaults and options need to be updated here to match 0.10. I just don't know how you deal with the version mismatch if someone's using 0.9, but maybe you just don't?

https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md#v0100-2021-09-30

@achingbrain
Copy link
Member

Defaults and options need to be updated here to match 0.10. I just don't know how you deal with the version mismatch if someone's using 0.9, but maybe you just don't?

We've not addressed this before - if people are using the older version of go-ipfs they should use the older version of ipfs-http-client.

It could probably use a table in the README saying which versions of the client are compatible with which versions of go-ipfs and js-ipfs.

@jtsmedley
Copy link
Author

This is the latest version of js-ipfs and go-ipfs. A chart would be great. Everything works fine before 0.10.

Thanks for checking into this!

@lidel lidel added kind/maintenance Work required to avoid breaking changes or harm to project's status quo P1 High: Likely tackled by core team if no one steps up and removed need/triage Needs initial labeling and prioritization labels Oct 8, 2021
rvagg added a commit that referenced this issue Oct 11, 2021
Fixes: #3914
Ref: https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md#v0100-2021-09-30

--format and --input-enc have been replaced with --input-codec and
--store-codec and mean something a little different. You now supply
raw input and instruct the server which --input-codec that data is
in which it will decode, then re-encode with --store-codec before
storing it and providing you with the CID.

We accept plain JavaScript objects to encode with --store-codec via
the API here, defaulting to dag-cbor, and send that to the server as
encoded bytes using that codec, to be stored using that codec.

If you supply an --input-codec then we assume you're supplying raw,
encoded bytes using that codec and we pass that directly on to the
server to handle.
rvagg added a commit that referenced this issue Oct 11, 2021
Fixes: #3914
Ref: https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md#v0100-2021-09-30

--format and --input-enc have been replaced with --input-codec and
--store-codec and mean something a little different. You now supply
raw input and instruct the server which --input-codec that data is
in which it will decode, then re-encode with --store-codec before
storing it and providing you with the CID.

We accept plain JavaScript objects to encode with --store-codec via
the API here, defaulting to dag-cbor, and send that to the server as
encoded bytes using that codec, to be stored using that codec.

If you supply an --input-codec then we assume you're supplying raw,
encoded bytes using that codec and we pass that directly on to the
server to handle.
rvagg added a commit that referenced this issue Oct 11, 2021
Fixes: #3914
Ref: https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md#v0100-2021-09-30

--format and --input-enc have been replaced with --input-codec and
--store-codec and mean something a little different. You now supply
raw input and instruct the server which --input-codec that data is
in which it will decode, then re-encode with --store-codec before
storing it and providing you with the CID.

We accept plain JavaScript objects to encode with --store-codec via
the API here, defaulting to dag-cbor, and send that to the server as
encoded bytes using that codec, to be stored using that codec.

If you supply an --input-codec then we assume you're supplying raw,
encoded bytes using that codec and we pass that directly on to the
server to handle.
rvagg added a commit that referenced this issue Oct 12, 2021
Fixes: #3914
Ref: https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md#v0100-2021-09-30

--format and --input-enc have been replaced with --input-codec and
--store-codec and mean something a little different. You now supply
raw input and instruct the server which --input-codec that data is
in which it will decode, then re-encode with --store-codec before
storing it and providing you with the CID.

We accept plain JavaScript objects to encode with --store-codec via
the API here, defaulting to dag-cbor, and send that to the server as
encoded bytes using that codec, to be stored using that codec.

If you supply an --input-codec then we assume you're supplying raw,
encoded bytes using that codec and we pass that directly on to the
server to handle.
rvagg added a commit that referenced this issue Oct 16, 2021
Fixes: #3914
Ref: https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md#v0100-2021-09-30

--format and --input-enc have been replaced with --input-codec and
--store-codec and mean something a little different. You now supply
raw input and instruct the server which --input-codec that data is
in which it will decode, then re-encode with --store-codec before
storing it and providing you with the CID.

We accept plain JavaScript objects to encode with --store-codec via
the API here, defaulting to dag-cbor, and send that to the server as
encoded bytes using that codec, to be stored using that codec.

If you supply an --input-codec then we assume you're supplying raw,
encoded bytes using that codec and we pass that directly on to the
server to handle.
rvagg added a commit that referenced this issue Nov 15, 2021
Fixes: #3914
Ref: https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md#v0100-2021-09-30

--format and --input-enc have been replaced with --input-codec and
--store-codec and mean something a little different. You now supply
raw input and instruct the server which --input-codec that data is
in which it will decode, then re-encode with --store-codec before
storing it and providing you with the CID.

We accept plain JavaScript objects to encode with --store-codec via
the API here, defaulting to dag-cbor, and send that to the server as
encoded bytes using that codec, to be stored using that codec.

If you supply an --input-codec then we assume you're supplying raw,
encoded bytes using that codec and we pass that directly on to the
server to handle.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/maintenance Work required to avoid breaking changes or harm to project's status quo P1 High: Likely tackled by core team if no one steps up
Projects
No open projects
Archived in project
Development

Successfully merging a pull request may close this issue.

4 participants