Add a new bit mask for copy + Update Python array API spec

[leofang]

Close #116. Close #134. Close #132.

This PR is an accompanying PR for data-apis/array-api#602 and data-apis/array-api#741. It does a few things to prepare for the release of Python array API standard v2023:

• Add a new DLPACK_FLAG_BITMASK_IS_COPIED bit mask to indicate if a copy has been made by the producer (Add view/copy informational flag #134)
• Add the missing bit mask docs to the C API reference
• Update the Python API specs:
  • max_version is added to __dlpack__ to support the transition from DLPack 0.x to 1.x (Next steps for the Python API? #116)
  • copy is added to from_dlpack and __dlpack__ to allow explicit copy requests (Add view/copy informational flag #134)
  • device (dl_device) is added to from_dlpack (__dlpack__) to allow cross-device copies (Question about expectation on cross-hardware #132). However, currently we only mandate the support of device-to-host copies.
  • Mention that the new versioned tensor should be used and the proper treatment for DLPack 1.0+

[leofang]

cc: @rgommers @seberg @tqchen @fcharras @oleksandr-pavlyk

[seberg]

Thanks, Leo! Needs the data-api commit to be full explanatory for the Python exchange. But that seems fine (it is linked at the top).

So this LGTM, modulo the "must". I would like if we can make it a must flag (also suggested brief mention earlier).

If anyone worries about it, adding the flag could be a minor version bump, but not sure it matters at this point.

[rgommers]

My main comment is conceptual: the ..._IS_COPIED name and "must make a copy" aren't entirely accurate I think. The actual requirement for copy=True is that the producer guarantees that it is not making use of the memory, and hence the consumer can do with it what it wants. E.g, update values in-place safely. In most cases that is going to require a memory copy, but not always. Simple example:

import numpy as np
import torch

# Here NumPy can know that the array created by `np.ones(3)` isn't used (same trick as for
# temporary elision), so it can simply return that memory rather than copy it first. For a 
# library with more context (e.g., in a compute graph) there may be more ways to guarantee
# that the memory isn't used on the producer side.
torch.from_dlpack(np.ones(3), copy=True)

This case could be either addressed in the text, or by changing IS_COPIED to something like GUARANTEED_UNUSED.

[seberg]

This case could be either addressed in the text

Right this is about guaranteeing sole ownership (after hand-off nobody else will access the buffer ever again). Leo was menioning "unique" as a name (from unique pointer), not sure what is a better name/text though.

EDIT: but I guess it is more than "exclusive" ownership, which might also mean that ownership is handed back after freeing the buffer. Tricky!

[leofang]

@rgommers when @seberg and I discussed offline, we wanted something that makes the ownership clear by drawing analogy from C++ smart pointers, so

• a.__dlpack__(copy=True): The producer copies a and returns a unique pointer to the data. The IS_COPIED flag is set. The consumer knows it’s the sole owner (@seberg: particularly, it doesn’t need to make an additional copy to avoid mutating a). When the consumer is done, the memory is still returned to the producer through th DLPack destructor.
• a.__dlpack__(copy=False): The producer does not copy and just returns a shared pointer to the data. The IS_COPIED flag is not set. Whether or not the consumer can mutate the data would depend on the READ_ONLY flag (and if the consumer honors it).

Currently, copy/IS_COPIED are used as a proxy to decide the ownership across the library boundary. I do not think your example could/should work without an actual copy (i.e. copy elision can't possibly happen), because generally two libraries' graph systems (if both have one) don't communicate with (or even be aware of) each other. If the day comes, we'll redesign DLPack and bump the major version.

[seberg]

I think I would lean towards just sticking with the copy name. But if we can think of a note that in theory "copy" can also be flagged if unique ownership is passed off (and can never be returned) would seem fine.

I do not think your example could/should work without an actual copy (i.e. copy elision can't possibly happen)

I agree it's probably impossible to elide. But, adding a note that "copy" could have a slightly more broad meaning would seem fine (maybe someone can actually invalidate their object instead).

Although, I would also be happy to defer that note at this point.

[rgommers]

The unique pointer analogy sounds good to me.

I think I would lean towards just sticking with the copy name. But if we can think of a note that in theory "copy" can also be flagged if unique ownership is passed off (and can never be returned) would seem fine.

Sticking with the copy name seems fine to me as long as the note for uniqueness is added.

I agree it's probably impossible to elide.

Even if that's true for the example I gave (which I'm not 100% sure about), with more context like when an array is created within a code block that's JIT-compiled (torch.compile for example does understand calls to the NumPy API), it should be possible to know that an array is not used.

[leofang]

Sounds good.

Sticking with the copy name seems fine to me as long as the note for uniqueness is added.

Will do.

Even if that's true for the example I gave (which I'm not 100% sure about), with more context like when an array is created within a code block that's JIT-compiled (torch.compile for example does understand calls to the NumPy API), it should be possible to know that an array is not used.

I personally think it's a "foul" that torch.compile supports NumPy APIs and it should not be used for consideration here 🙂 (I assume we're referring to pytorch/rfcs#54.) When a library does not mind its own business and tries to decide what another library should behave within a certain context (in this case, JIT), all things discussed regarding cross-library interoperability is moot, because that library is the sole decision maker here (potentially without the other library authors' consent!). Then, anything is possible.

[leofang]

Sticking with the copy name seems fine to me as long as the note for uniqueness is added.

Will do.

Done in commit a1eecba.

[leofang]

@tqchen this is ready 🙂

[tqchen]

maybe @seberg @rgommers can explicitly chime in and approve if you argee?

[seberg]

LGTM. If anyone cares, we could flag the addition as 1.1 (rather than 1.0), but with __dlpack__ never being bumped to actual versioning, I doubt it matters.

[rgommers]

My one main comment was addressed, and everything else LGTM. So seems ready to go - thanks a lot!

[leofang]

Thanks all for pushing this across the finish line!

@tqchen @rgommers I suppose we can make a release for DLPack 1.0 now? 🙂

[tqchen]

https://github.com/dmlc/dlpack/releases/tag/v1.0rc