The context in CtxReadFull is not propagated to the child nodes being read

[schomatis]

The context used during read operations is always passed to the child PB node and "bifurcated" with a context.WithCancel(ctx) call,

go-unixfs/io/pbdagreader.go

Lines 50 to 58 in 76afbe2

	func NewPBFileReader(ctx context.Context, n *mdag.ProtoNode, file *ft.FSNode, serv ipld.NodeGetter) *PBDagReader {
	fctx, cancel := context.WithCancel(ctx)
	curLinks := getLinkCids(n)
	return &PBDagReader{
	serv: serv,
	buf: NewBufDagReader(file.Data()),
	promises: make([]*ipld.NodePromise, len(curLinks)),
	links: curLinks,
	ctx: fctx,

I understand that the rationale behind this is that if I cancel a context at a certain depth in the DAG I'm reading, that is, if I cancel a fetch operation of a node (promise) in the DAG, it will cancel all the operations below that depth that are happening in their children.

That context being "bifurcated" always comes from the internal PBDagReader's context:

go-unixfs/io/pbdagreader.go

Lines 128 to 139 in 76afbe2

	func (dr *PBDagReader) loadBufNode(node ipld.Node) error {
	switch node := node.(type) {
	case *mdag.ProtoNode:
	fsNode, err := ft.FSNodeFromBytes(node.Data())
	if err != nil {
	return fmt.Errorf("incorrectly formatted protobuf: %s", err)
	}
	switch fsNode.Type() {
	case ftpb.Data_File:
	dr.buf = NewPBFileReader(dr.ctx, node, fsNode, dr.serv)
	return nil

If we're doing a Read call (that calls CtxReadFull with the internal context) the behavior is consistent with the rationale described above, but if the user manually calls CtxReadFull with a custom context it will be applied only to the fetch operations (precalcNextBuf) of the root node but it won't be used for the child nodes at deeper depths,

go-unixfs/io/pbdagreader.go

Lines 169 to 176 in 76afbe2

	func (dr *PBDagReader) Read(b []byte) (int, error) {
	return dr.CtxReadFull(dr.ctx, b)
	}
	// CtxReadFull reads data from the DAG structured file
	func (dr *PBDagReader) CtxReadFull(ctx context.Context, b []byte) (int, error) {
	if dr.buf == nil {
	if err := dr.precalcNextBuf(ctx); err != nil {

The result is that a cancel of the custom context used in CtxReadFull won't cancel fetch operations (promises) at deeper levels, is that the intended behavior?

[schomatis]

@Stebalien Could you check this please, I need to understand the context use here to apply it in the DAG walker in ipfs/go-ipld-format#39 and replace this logic without breaking the API.

[Stebalien]

The result is that a cancel of the custom context used in CtxReadFull won't cancel fetch operations (promises) at deeper levels, is that the intended behavior?

Yes, that looks like a bug. It looks like we should:

1. Make sure dr.buf has a CtxReadFull function.
2. Use it to thread the context through.

Does that sound right to you? That is, we should repalce the following io.ReadFull with a dr.buf.CtxReadFull:

go-unixfs/io/pbdagreader.go

Line 185 in 76afbe2

n, err := io.ReadFull(dr.buf, b[total:])

[schomatis]

Thanks for taking a look at this, I'm not sure if CtxReadFull can provide all the functionality of ReadFull, but it doesn't matter, I don't plan to fix this code, I just wanted to make sure I understood the general spirit behind the cascading contexts to replicate it in ipfs/go-ipld-format#39 (I'll fix it there).

[schomatis]

Fixed in #60.