Implementing the new interfaces #1086
Conversation
ghost
assigned
src/core/components/files.js
Outdated
| callback(null, toStream.source(files[files.length - 1].content)) | ||
| }) | ||
| p, | ||
| pull.concat(callback) | ||
| ) | ||
| }), |
There was a problem hiding this comment.
@pgte the .cat for a large file (15Mb) is failing with:
Seems that some of the streams is giving up too early. Could you help me figuring this one out?
There was a problem hiding this comment.
Think I got it, I can't use concat to concat buffers https://github.com/pull-stream/pull-stream/blob/master/sinks/concat.js
|
What is missing in this PR:
|
|
Now the example is using pull-streams directly (though ipfs.files.add) but I'm being unsuccessful to add and fetch files (although tests are 100% passing). Something might have changed in the browser APIs that is messing with the data blobs. @dignifiedquire @victorbjelkholm does this ring a bell? |
| node.on('ready', () => { | ||
| console.log('IPFS node is ready') | ||
| }) | ||
| node.on('ready', () => console.log('IPFS node is ready')) |
There was a problem hiding this comment.
What about using .once instead of `.on´?
|
|
||
| node.start(() => { | ||
| console.log('Online status: ', node.isOnline() ? 'online' : 'offline') | ||
| node.on('ready', () => { |
| node = new IPFS({ | ||
| repo: String(Math.random() + Date.now()) | ||
| }) | ||
| node = new IPFS({ repo: String(Math.random() + Date.now()) }) | ||
|
|
||
| node.on('ready', () => { |
There was a problem hiding this comment.
once would be nicer, I think.
| // EXAMPLE | ||
| // node = new self.Ipfs({ repo: 'ipfs-' + Math.random() }) | ||
|
|
||
| node.on('start', () => node.id((err, id) => { |
test/core/kad-dht.node.js
Outdated
| const parallel = require('async/parallel') | ||
| const IPFSFactory = require('../utils/ipfs-factory-instance') | ||
|
|
||
| describe('verify that kad-dht is doing its thing', () => { | ||
| describe.skip('verify that kad-dht is doing its thing', () => { |
There was a problem hiding this comment.
This tests needs to be unskiped and passing before merging
Update, I'm down to:HTTP
|
Were those ever enabled? I can't trace them to being enabled either in |
|
@dryajov yes, they were enabled through the node.js file that was present in the test folder (like it is now) |
| console.log('Fetched file:', cid, file.content.length) | ||
| // calling createFileBlob makes the Chrome go "Oh Snap" | ||
| // const listItem = createFileBlob(file.content, cid) | ||
| // $filesList.insertBefore(listItem, $filesList.firstChild) |
There was a problem hiding this comment.
Ok, narrowed down to the createFileBlob func. Apparently it always makes the Chrome go "oh Snap" and in Firefox it creates a wrong file (3x the size)
@victorbjelkholm thoughts on this one?
Good news everyone!I was successfully able to upload a file with more than 750Mb (didn't test higher) and was able to fetch it through the gateway (slowly but it went through). |
src/http/api/resources/files.js
Outdated
| console.log('packed a directory') | ||
| cb() | ||
| }) | ||
| } |
There was a problem hiding this comment.
The final issue is here, for some reason, tar-stream blocks on the second dir that is added:
Fixed it 🌟The issue was introduced when I adopted the getPullStream for the http-api (instead of the old with ReadableStreams) and although both implemented just fine, the end of a tl;dr;, Readable Streams are dangerous, mixing them with pull-streams doesn't make it better, gotta be extra cautious. |
License: MIT Signed-off-by: Alan Shaw <alan.shaw@protocol.ai>
ref: ipfs-inactive/interface-js-ipfs-core#162