ipfs.dag.put HTTPError: missing an unmarshaller for tag 1 when using ipfs-http-client

[stanbar]

• Version:
  "ipfs-http-client": "^44.0.3",
  ipfs: 0.5.1

• Platform:
  Docker (Linux b566b5fa7508 4.19.76-linuxkit Add to cli: ipfs pin [-r] <ipfs-path> #1 SMP Fri Apr 3 15:53:26 UTC 2020 x86_64 GNU/Linux)

Severity:

High - The main functionality of the application does not work, API breakage, repo format

Description:

I want to add to dag the object that looks like this:

const voting = {
    "title": "asdfasd",
    "polls": [
        {
            "_id": "5ec46e322d7e49029617fa2e",
            "question": "question",
            "options": [
                {
                    "_id": "5ec46e322d7e49029617fa2f",
                    "name": "one",
                    "code": 1
                },
                {
                    "_id": "5ec46e322d7e49029617fa30",
                    "name": "two",
                    "code": 2
                }
            ]
        }
    ],
    "issueAccountId": "GCENXCYETHIROKE22XBUPVGRW7FAQJVN32VCH4XKXEIFJMYFA532HSVI",
    "assetCode": "Newvoting",
    "distributionAccountId": "GAWSMPQUG3YK4ED6YVVQKWJ4FJQ5FBS3TTCI3TWSQVSML7RWEOUJB7U6",
    "ballotBoxAccountId": "GBOFKKWIEBFOWQ23UOUSYZOW6TFMGHFJD6PXKWU725TAX724AE5KBJV6",
    "authorization": "open",
    "visibility": "public",
    "votesCap": 100,
    "encryption": {
        "encryptionKey": "MDgCEQDPE5eb9/3Qz7cJqUQuiilnAhBE+ZH2pLxJOAii2tzKbZyIAhEAn7UjCy+yLapZs6Nr0GxRgQ==",
        "encryptedUntil": "2020-05-27T23:16:27.858Z"
    },
    "startDate": "2020-05-19T23:16:26.689Z",
    "endDate": "2020-05-27T23:16:27.858Z",
    "slug": "voting-vgmxmt",
    "createdAt": "2020-05-19T23:39:30.912Z",
    "updatedAt": "2020-05-19T23:39:30.912Z",
    "id": "5ec46e322d7e49029617fa2d"
}

when i call the function

const cid = (await ipfs.dag.put(voting, { pin: true, format: 'dag-cbor', hashAlg: 'sha2-256' })).toString()

it throws

 HTTPError: missing an unmarshaller for tag 1
     at Object.errorHandler [as handleError] (/server/node_modules/ipfs-http-client/src/lib/core.js:67:15)
     at processTicksAndRejections (internal/process/task_queues.js:97:5)
     at Client.fetch (/server/node_modules/ipfs-utils/src/http.js:155:9)
     at Object.put (/server/node_modules/ipfs-http-client/src/dag/put.js:46:17)

I'm using ipfs-http-client and the actual node is running in docker container.

I looked at the implementation and the serializer seems to be working fine

import * as dagCBOR from 'ipld-dag-cbor'

const voting = // { ... }
const serialized = dagCBOR.util.serialize(voting)

Is working fine.

[achingbrain]

I'm having problems replicating this. I've tried creating a minimal project:

package.json:

{
  "dependencies": {
    "go-ipfs-dep": "^0.5.1",
    "ipfs-http-client": "^44.1.0",
    "ipfsd-ctl": "^4.1.1"
  }
}

index.js:

const { createFactory } = require('ipfsd-ctl')

async function main () {
  const factory = createFactory({
    type: 'go',
    test: true,
    ipfsHttpModule: require('ipfs-http-client'),
    ipfsBin: require('go-ipfs-dep').path()
  })
  const node = await factory.spawn()
  const voting = {
    // copied from above
  }
  const cid = await node.api.dag.put(voting, { pin: true, format: 'dag-cbor', hashAlg: 'sha2-256' })
  console.info(`put ${cid}`)
  await node.stop()
}

main()
  .catch(err => {
    console.error(err)
    process.exit(1)
  })

console:

$ node index.js
put bafyreidccdo62shnbnjvcbmqunrvqglckydhjp7zzu3n4csfeagwqbfv3q

ipfs-http-client runs it's interface tests against go-ipfs - could you add a test to the dag.put suite that fails with the same error you are seeing?

To run the tests:

$ git clone https://github.com/ipfs/js-ipfs.git
$ cd js-ipfs
$ npm i
// now edit packages/interface-ipfs-core/src/dag/put.js and add your test
$ cd packages/ipfs-http-client
$ npm run test:node

When it's failing, please create a PR to this repo with your changes.

[stanbar]

I wrote the test, and it's passing. So I guess the problem is somewhere else.
The error throws from HTTP request, so maybe the go-ipfs node (running in docker), doesn't understand the serialized content.
The thing is, when I put something else like:

await node.api.dag.put({ hello: 'world' }, { pin: true, format: 'dag-cbor', hashAlg: 'sha2-256' })

it is working fine.

I've found the workaround, if I first stringify and parse the object it gets added successfully.

const normalizedObject = JSON.parse(JSON.stringify(voting))
await ipfs.dag.put(normalizedObject, { pin: true, format: 'dag-cbor', hashAlg: 'sha2-256' }

I guess there is some problem with the voting object itself (I'm fetching it from mongodb). So you can close the issue, if you don't know why is that happening. Thank you for your time.

[achingbrain]

Cbor tag 1 is for dates so I wonder if something weird is going on there. This error is coming from the remote node though - any ideas @Stebalien?

[achingbrain]

I’m guessing startDate/endDate etc are actually Date objects in memory and not strings as above.

[achingbrain]

It's definitely that, if I change:

"createdAt": "2020-05-19T23:39:30.912Z",
"updatedAt": "2020-05-19T23:39:30.912Z",

to:

"createdAt": new Date("2020-05-19T23:39:30.912Z"),
"updatedAt": new Date("2020-05-19T23:39:30.912Z"),

I get:

$ node index.js
HTTPError: missing an unmarshaller for tag 1
    at Object.errorHandler [as handleError] (/Users/alex/test/bar/node_modules/ipfs-http-client/src/lib/core.js:67:15)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async Client.fetch (/Users/alex/test/bar/node_modules/ipfs-utils/src/http.js:155:9)
    at async Object.put (/Users/alex/test/bar/node_modules/ipfs-http-client/src/dag/put.js:52:17)
    at async main (/Users/alex/test/bar/index.js:49:15) {
  name: 'HTTPError',
  response: Response {
    size: 0,
    timeout: 0,
    [Symbol(Body internals)]: { body: [PassThrough], disturbed: true, error: null },
    [Symbol(Response internals)]: {
      url: 'http://127.0.0.1:54141/api/v0/dag/put?format=dag-cbor&input-enc=raw&pin=true&hash=sha2-256',
      status: 500,
      statusText: 'Internal Server Error',
      headers: [Headers],
      counter: 0
    }
  }
}

js-IPFS doesn't throw the same error, so I suppose there's only a subset of CBOR tags that go-IPFS supports?

[achingbrain]

@Stebalien any thoughts on the above? Looks like go-ipfs doesn't support unmarshalling cbor with tags that aren't CIDs or simple strings/numbers/maybe?

[achingbrain]

js-ipfs is being deprecated in favor of Helia. You can #4336 and read the migration guide.

You can swap ipfs-http-client for kubo-rpc-client - it's a drop-in replacement and where compatibility fixes with Kubo will get added in future.