From d55ca46c4a48fa03f1a8e1d823f74122a9498c8b Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 9 Mar 2023 21:45:03 -0800 Subject: [PATCH 1/6] Document PEP 688 --- Doc/library/collections.abc.rst | 8 +++++++ Doc/library/functions.rst | 2 +- Doc/library/inspect.rst | 35 +++++++++++++++++++++++++++++ Doc/library/stdtypes.rst | 14 +++++++++++- Doc/reference/datamodel.rst | 39 +++++++++++++++++++++++++++++++++ Doc/whatsnew/3.12.rst | 12 ++++++++++ 6 files changed, 108 insertions(+), 2 deletions(-) diff --git a/Doc/library/collections.abc.rst b/Doc/library/collections.abc.rst index 1ada0d352a0cc6..06e3904a5bc578 100644 --- a/Doc/library/collections.abc.rst +++ b/Doc/library/collections.abc.rst @@ -177,6 +177,7 @@ ABC Inherits from Abstract Methods Mi :class:`AsyncIterable` [1]_ ``__aiter__`` :class:`AsyncIterator` [1]_ :class:`AsyncIterable` ``__anext__`` ``__aiter__`` :class:`AsyncGenerator` [1]_ :class:`AsyncIterator` ``asend``, ``athrow`` ``aclose``, ``__aiter__``, ``__anext__`` +:class:`Buffer` [1]_ ``__buffer__`` ============================== ====================== ======================= ==================================================== @@ -346,6 +347,13 @@ Collections Abstract Base Classes -- Detailed Descriptions .. versionadded:: 3.6 +.. class:: Buffer + + ABC for classes that provide the :meth:`__buffer__` method, + implementing the :ref:`buffer protocol `. See :pep:`688`. + + .. versionadded:: 3.12 + Examples and Recipes -------------------- diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index f0f374771b0cf1..a53677c67338d6 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1070,7 +1070,7 @@ are always available. They are listed here in alphabetical order. .. _func-memoryview: -.. class:: memoryview(object) +.. class:: memoryview(object, *, flags=BufferFlags.FULL_RO) :noindex: Return a "memory view" object created from the given argument. See diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst index 789e9839d22f71..e027b0d97fd60d 100644 --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -1575,6 +1575,41 @@ the following flags: for any introspection needs. +Buffer flags +------------ + +The :mod:`inspect` module provides an :class:`enum.IntFlag` + +.. class:: BufferFlags + + This is an :class:`enum.IntFlag` that represents the flags that + can be passed to the :meth:`__buffer__` method of objects + implementing the :ref:`buffer protocol `. + + The meaning of the flags is explained at :ref:`buffer-request-types`. + + .. attribute:: SIMPLE + .. attribute:: WRITABLE + .. attribute:: FORMAT + .. attribute:: ND + .. attribute:: STRIDES + .. attribute:: C_CONTIGUOUS + .. attribute:: F_CONTIGUOUS + .. attribute:: ANY_CONTIGUOUS + .. attribute:: INDIRECT + .. attribute:: CONTIG + .. attribute:: CONTIG_RO + .. attribute:: STRIDED + .. attribute:: STRIDED_RO + .. attribute:: RECORDS + .. attribute:: RECORDS_RO + .. attribute:: FULL + .. attribute:: FULL_RO + .. attribute:: READ + .. attribute:: WRITE + + .. versionadded:: 3.12 + .. _inspect-module-cli: Command Line Interface diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 550f808a16dfaa..788e19f056bb8b 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -3704,7 +3704,7 @@ Memory Views of an object that supports the :ref:`buffer protocol ` without copying. -.. class:: memoryview(object) +.. class:: memoryview(object, *, flags=BufferFlags.FULL_RO) Create a :class:`memoryview` that references *object*. *object* must support the buffer protocol. Built-in objects that support the buffer @@ -3789,6 +3789,15 @@ copying. >>> hash(v[::-2]) == hash(b'abcefg'[::-2]) True + The optional ``flags`` can be used to control the :class:`inspect.BufferFlags ` + passed to the underlying buffer. For example:: + + >>> import inspect + >>> v = memoryview(b"x", flags=inspect.BufferFlags.WRITABLE) + ... + BufferError: Object is not writable. + >>> v = memoryview(b"x", flags=inspect.BufferFlags.SIMPLE) + .. versionchanged:: 3.3 One-dimensional memoryviews can now be sliced. One-dimensional memoryviews with formats 'B', 'b' or 'c' are now :term:`hashable`. @@ -3800,6 +3809,9 @@ copying. .. versionchanged:: 3.5 memoryviews can now be indexed with tuple of integers. + .. versionchanged:: 3.12 + Added the ``flags`` argument. + :class:`memoryview` has several methods: .. method:: __eq__(exporter) diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index f447bbb1216d52..4c74bacbb9ca7e 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2849,6 +2849,45 @@ a :exc:`TypeError`. The specification for the Python ``match`` statement. +.. _python-buffer-protocol: + +Emulating buffer types +---------------------- + +The :ref:`buffer protocol ` provides a way for Python +objects to expose efficient access a low-level memory array. This protocol +is implemented by builtin types such as :class:`bytes` and :class:`memoryview`, +and third-party libraries may define additional buffer types. + +While buffer types are usually implemented in C, it is also possible to +implement the protocol in Python. + +.. method:: object.__buffer__(self, flags) + + Called when a buffer is requested from ``self`` (for example, by the + :class:`memoryview` constructor). The ``flags`` argument is an integer + representing the kind of buffer requested, affecting for example whether + the returned buffer is read-only or writable. :class:`inspect.BufferFlags` + provides a convenient way to interpret the flags. The method must return + a :class:`memoryview` object. + +.. method:: object.__release_buffer__(self, buffer) + + Called when a buffer is no longer needed. The ``buffer`` argument is a + :class:`memoryview` object that was previously returned by + :meth:`__buffer__`. The method must release any resources associated + with the buffer. This method should return ``None``. + +.. versionadded:: 3.12 + +.. seealso:: + + :pep:`688` - Making the buffer protocol accessible in Python + Introduces the Python ``__buffer__`` and ``__release_buffer__`` methods. + + :class:`collections.abc.Buffer` + ABC for buffer types. + .. _special-lookup: Special method lookup diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index d982cb62ec2f4e..d4c6c77dc5aa89 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -138,6 +138,18 @@ New Features with contributions from Gregory P. Smith [Google] and Mark Shannon in :gh:`96123`.) +PEP 688: Making the buffer protocol accessible in Python +-------------------------------------------------------- + +:pep:`688` introduces a way to use the :ref:`buffer protocol ` +from Python code. Classes that implement the :meth:`__buffer__` method +are now usable as buffer types. + +The new :class:`collections.abc.Buffer` ABC provides a standard +way to represent buffer objects, for example in type annotations. +:class:`memoryview` has a new ``flags`` argument to control the +:class:`buffer flags ` passed to the underlying buffer. +(Contributed by Jelle Zijlstra in :gh:`102500`.) Other Language Changes ====================== From c64c54653bfe31b3bebf6a0a667f86aa0af1af7c Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 9 Mar 2023 21:53:01 -0800 Subject: [PATCH 2/6] Improvements --- Doc/library/collections.abc.rst | 2 +- Doc/library/inspect.rst | 40 ++++++++++++++++----------------- Doc/reference/datamodel.rst | 4 +++- Doc/whatsnew/3.12.rst | 6 ++--- 4 files changed, 27 insertions(+), 25 deletions(-) diff --git a/Doc/library/collections.abc.rst b/Doc/library/collections.abc.rst index 06e3904a5bc578..669b7345499a78 100644 --- a/Doc/library/collections.abc.rst +++ b/Doc/library/collections.abc.rst @@ -349,7 +349,7 @@ Collections Abstract Base Classes -- Detailed Descriptions .. class:: Buffer - ABC for classes that provide the :meth:`__buffer__` method, + ABC for classes that provide the :meth:`~object.__buffer__` method, implementing the :ref:`buffer protocol `. See :pep:`688`. .. versionadded:: 3.12 diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst index e027b0d97fd60d..e2531e6a9dd3f6 100644 --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -1583,30 +1583,30 @@ The :mod:`inspect` module provides an :class:`enum.IntFlag` .. class:: BufferFlags This is an :class:`enum.IntFlag` that represents the flags that - can be passed to the :meth:`__buffer__` method of objects + can be passed to the :meth:`~object.__buffer__` method of objects implementing the :ref:`buffer protocol `. The meaning of the flags is explained at :ref:`buffer-request-types`. - .. attribute:: SIMPLE - .. attribute:: WRITABLE - .. attribute:: FORMAT - .. attribute:: ND - .. attribute:: STRIDES - .. attribute:: C_CONTIGUOUS - .. attribute:: F_CONTIGUOUS - .. attribute:: ANY_CONTIGUOUS - .. attribute:: INDIRECT - .. attribute:: CONTIG - .. attribute:: CONTIG_RO - .. attribute:: STRIDED - .. attribute:: STRIDED_RO - .. attribute:: RECORDS - .. attribute:: RECORDS_RO - .. attribute:: FULL - .. attribute:: FULL_RO - .. attribute:: READ - .. attribute:: WRITE + .. attribute:: BufferFlags.SIMPLE + .. attribute:: BufferFlags.WRITABLE + .. attribute:: BufferFlags.FORMAT + .. attribute:: BufferFlags.ND + .. attribute:: BufferFlags.STRIDES + .. attribute:: BufferFlags.C_CONTIGUOUS + .. attribute:: BufferFlags.F_CONTIGUOUS + .. attribute:: BufferFlags.ANY_CONTIGUOUS + .. attribute:: BufferFlags.INDIRECT + .. attribute:: BufferFlags.CONTIG + .. attribute:: BufferFlags.CONTIG_RO + .. attribute:: BufferFlags.STRIDED + .. attribute:: BufferFlags.STRIDED_RO + .. attribute:: BufferFlags.RECORDS + .. attribute:: BufferFlags.RECORDS_RO + .. attribute:: BufferFlags.FULL + .. attribute:: BufferFlags.FULL_RO + .. attribute:: BufferFlags.READ + .. attribute:: BufferFlags.WRITE .. versionadded:: 3.12 diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 4c74bacbb9ca7e..bd8d6fd4bdebf2 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2875,8 +2875,10 @@ implement the protocol in Python. Called when a buffer is no longer needed. The ``buffer`` argument is a :class:`memoryview` object that was previously returned by - :meth:`__buffer__`. The method must release any resources associated + :meth:`~object.__buffer__`. The method must release any resources associated with the buffer. This method should return ``None``. + Buffer objects that do not need to perform any cleanup are not required + to implement this method. .. versionadded:: 3.12 diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index d4c6c77dc5aa89..d8be7fb8039f0e 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -142,13 +142,13 @@ PEP 688: Making the buffer protocol accessible in Python -------------------------------------------------------- :pep:`688` introduces a way to use the :ref:`buffer protocol ` -from Python code. Classes that implement the :meth:`__buffer__` method +from Python code. Classes that implement the :meth:`~object.__buffer__` method are now usable as buffer types. The new :class:`collections.abc.Buffer` ABC provides a standard way to represent buffer objects, for example in type annotations. -:class:`memoryview` has a new ``flags`` argument to control the -:class:`buffer flags ` passed to the underlying buffer. +:class:`memoryview` has a new ``flags`` argument to control the flags +(:class:`inspect.BufferFlags`) passed to the underlying buffer. (Contributed by Jelle Zijlstra in :gh:`102500`.) Other Language Changes From f561ddf8871391ca2d161434213787f6463faa02 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Fri, 10 Mar 2023 16:58:25 -0800 Subject: [PATCH 3/6] No flags argument to memoryview --- Doc/library/functions.rst | 2 +- Doc/library/stdtypes.rst | 14 +------------- Doc/whatsnew/3.12.rst | 4 ++-- 3 files changed, 4 insertions(+), 16 deletions(-) diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index a53677c67338d6..f0f374771b0cf1 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -1070,7 +1070,7 @@ are always available. They are listed here in alphabetical order. .. _func-memoryview: -.. class:: memoryview(object, *, flags=BufferFlags.FULL_RO) +.. class:: memoryview(object) :noindex: Return a "memory view" object created from the given argument. See diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 788e19f056bb8b..550f808a16dfaa 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -3704,7 +3704,7 @@ Memory Views of an object that supports the :ref:`buffer protocol ` without copying. -.. class:: memoryview(object, *, flags=BufferFlags.FULL_RO) +.. class:: memoryview(object) Create a :class:`memoryview` that references *object*. *object* must support the buffer protocol. Built-in objects that support the buffer @@ -3789,15 +3789,6 @@ copying. >>> hash(v[::-2]) == hash(b'abcefg'[::-2]) True - The optional ``flags`` can be used to control the :class:`inspect.BufferFlags ` - passed to the underlying buffer. For example:: - - >>> import inspect - >>> v = memoryview(b"x", flags=inspect.BufferFlags.WRITABLE) - ... - BufferError: Object is not writable. - >>> v = memoryview(b"x", flags=inspect.BufferFlags.SIMPLE) - .. versionchanged:: 3.3 One-dimensional memoryviews can now be sliced. One-dimensional memoryviews with formats 'B', 'b' or 'c' are now :term:`hashable`. @@ -3809,9 +3800,6 @@ copying. .. versionchanged:: 3.5 memoryviews can now be indexed with tuple of integers. - .. versionchanged:: 3.12 - Added the ``flags`` argument. - :class:`memoryview` has several methods: .. method:: __eq__(exporter) diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index d8be7fb8039f0e..56a875038e166d 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -147,8 +147,8 @@ are now usable as buffer types. The new :class:`collections.abc.Buffer` ABC provides a standard way to represent buffer objects, for example in type annotations. -:class:`memoryview` has a new ``flags`` argument to control the flags -(:class:`inspect.BufferFlags`) passed to the underlying buffer. +The new :class:`inspect.BufferFlags` enum represents the flags that +can be used to customize buffer creation. (Contributed by Jelle Zijlstra in :gh:`102500`.) Other Language Changes From 14f9c432c8662d84ca6c15f6f83078576b44bd58 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Wed, 15 Mar 2023 00:39:44 -0700 Subject: [PATCH 4/6] Apply suggestions by Hugo Co-authored-by: Hugo van Kemenade --- Doc/reference/datamodel.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index bd8d6fd4bdebf2..c46deea4a48ada 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2855,7 +2855,7 @@ Emulating buffer types ---------------------- The :ref:`buffer protocol ` provides a way for Python -objects to expose efficient access a low-level memory array. This protocol +objects to expose efficient access to a low-level memory array. This protocol is implemented by builtin types such as :class:`bytes` and :class:`memoryview`, and third-party libraries may define additional buffer types. @@ -2864,8 +2864,8 @@ implement the protocol in Python. .. method:: object.__buffer__(self, flags) - Called when a buffer is requested from ``self`` (for example, by the - :class:`memoryview` constructor). The ``flags`` argument is an integer + Called when a buffer is requested from *self* (for example, by the + :class:`memoryview` constructor). The *flags* argument is an integer representing the kind of buffer requested, affecting for example whether the returned buffer is read-only or writable. :class:`inspect.BufferFlags` provides a convenient way to interpret the flags. The method must return @@ -2873,7 +2873,7 @@ implement the protocol in Python. .. method:: object.__release_buffer__(self, buffer) - Called when a buffer is no longer needed. The ``buffer`` argument is a + Called when a buffer is no longer needed. The *buffer* argument is a :class:`memoryview` object that was previously returned by :meth:`~object.__buffer__`. The method must release any resources associated with the buffer. This method should return ``None``. From ebe51b55b68b09aa3b5ff4385a247dd6226f1241 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Wed, 15 Mar 2023 07:34:41 -0700 Subject: [PATCH 5/6] Update Doc/library/inspect.rst Co-authored-by: Hugo van Kemenade --- Doc/library/inspect.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst index e2531e6a9dd3f6..a95676e6e945d0 100644 --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -1578,7 +1578,7 @@ the following flags: Buffer flags ------------ -The :mod:`inspect` module provides an :class:`enum.IntFlag` +The :mod:`inspect` module provides an :class:`enum.IntFlag`. .. class:: BufferFlags From 647aa3587e3a01d41aa62d00327962ed5aab72ad Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 4 May 2023 08:01:52 -0700 Subject: [PATCH 6/6] remove weird sentence --- Doc/library/inspect.rst | 2 -- 1 file changed, 2 deletions(-) diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst index 6005e46d3a50a4..7884308a333020 100644 --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -1606,8 +1606,6 @@ the following flags: Buffer flags ------------ -The :mod:`inspect` module provides an :class:`enum.IntFlag`. - .. class:: BufferFlags This is an :class:`enum.IntFlag` that represents the flags that