Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-107432: Rework the style of Doc/howto/functional.rst #107449

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
77 changes: 25 additions & 52 deletions Doc/howto/functional.rst
Original file line number Diff line number Diff line change
@@ -1,34 +1,24 @@
********************************
Functional Programming HOWTO
Functional Programming in Python
********************************

:Author: A. M. Kuchling
:Release: 0.32

In this document, we'll take a tour of Python's features suitable for
implementing programs in a functional style. After an introduction to the
concepts of functional programming, we'll look at language features such as
:term:`iterator`\s and :term:`generator`\s and relevant library modules such as
:mod:`itertools` and :mod:`functools`.
Comment on lines -8 to -12
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why remove this paragraph? It seems useful for readers to have a summary at the beginning of what the article's going to cover.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A user's eyes will skip over this. They will just bounce off the new reader, for whom they make little connection. They will not add value for the reader who is returning to the page. The links don't take the reader to the topics in the page, but take them away from it, so they don't serve as useful signposts to what's in the page.

If anyone wants to see what's in the page, they can see that immediately from the page's table of contents, where they will find the topics in the context and order of other topics, and can see their place in the hierarchy of concepts.

In general, summaries of what something is going to discuss (and summaries of what has been discussed) add nothing of value to writing. (A summary of conclusions is a quite different matter.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The links don't take the reader to the topics in the page, but take them away from it, so they don't serve as useful signposts to what's in the page.

I think it's useful to mention functools and itertools early-on as concrete examples of standard-library modules that allow you to use the functional programming style in Python, especially since the opening section is so abstract. I wouldn't mind it if you changed the links so that they pointed to the passages later on in this document where it explores the itertools and functools modules.



Introduction
============

This section explains the basic concept of functional programming; if
you're just interested in learning about Python language features,
skip to the next section on :ref:`functional-howto-iterators`.
The basics of functional programming
====================================

Programming languages support decomposing problems in several different ways:

* Most programming languages are **procedural**: programs are lists of
* Many programming languages are **procedural**: programs are lists of
instructions that tell the computer what to do with the program's input. C,
Pascal, and even Unix shells are procedural languages.

* In **declarative** languages, you write a specification that describes the
problem to be solved, and the language implementation figures out how to
perform the computation efficiently. SQL is the declarative language you're
most likely to be familiar with; a SQL query describes the data set you want
perform the computation efficiently. SQL is an example of a declarative
language; a SQL query describes the data set you want
to retrieve, and the SQL engine decides whether to scan tables or use indexes,
which subclauses should be performed first, etc.

Expand Down Expand Up @@ -57,7 +47,7 @@ functional, for example.

In a functional program, input flows through a set of functions. Each function
operates on its input and produces some output. Functional style discourages
functions with side effects that modify internal state or make other changes
functions with *side effects* that modify internal state or make other changes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This document mostly uses strong-emphasis to indicate a term being defined. To make it consistent:

Suggested change
functions with *side effects* that modify internal state or make other changes
functions with **side effects** that modify internal state or make other changes

that aren't visible in the function's return value. Functions that have no side
effects at all are called **purely functional**. Avoiding side effects means
not using data structures that get updated as a program runs; every function's
Expand Down Expand Up @@ -177,7 +167,7 @@ a few functions specialized for the current task.
Iterators
=========

I'll start by looking at a Python language feature that's an important
Let's start by considering a Python language feature that's an important
foundation for writing functional-style programs: iterators.

An iterator is an object representing a stream of data; this object returns the
Expand Down Expand Up @@ -259,7 +249,6 @@ consume all of the iterator's output, and if you need to do something different
with the same stream, you'll have to create a new iterator.



Data Types That Support Iterators
---------------------------------

Expand Down Expand Up @@ -326,7 +315,6 @@ elements::
13



Generator expressions and list comprehensions
=============================================

Expand Down Expand Up @@ -440,7 +428,7 @@ Generators are a special class of functions that simplify the task of writing
iterators. Regular functions compute a value and return it, but generators
return an iterator that returns a stream of values.

You're doubtless familiar with how regular function calls work in Python or C.
You may be familiar with how regular function calls work in Python or C.
When you call a function, it gets a private namespace where its local variables
are created. When the function reaches a ``return`` statement, the local
variables are destroyed and the value is returned to the caller. A later call
Expand Down Expand Up @@ -528,19 +516,16 @@ Passing values into a generator
-------------------------------

In Python 2.4 and earlier, generators only produced output. Once a generator's
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could drop the versions here and just describe the state of the language in Python 3.x. Maybe drop the first paragraph with "there was no elegant way..." entirely, and the new first paragraph would be something like:

It's also possible to pass values into a generator. The previous examples have used `yield x` alone on a line, as if :keyword:`yield` is a statement, but in fact it's an expression that returns a value that can be assigned to ...

code was invoked to create an iterator, there was no way to pass any new
information into the function when its execution is resumed. You could hack
together this ability by making the generator look at a global variable or by
passing in some mutable object that callers then modify, but these approaches
are messy.
code was invoked to create an iterator, there was no elegant way to pass any new
information into the function when its execution is resumed.

In Python 2.5 there's a simple way to pass values into a generator.
:keyword:`yield` became an expression, returning a value that can be assigned to
a variable or otherwise operated on::
In Python 2.5, :keyword:`yield` became an expression, returning a value that
can be assigned to a variable or otherwise operated on, providing a simple
way to pass values into a generator::

val = (yield i)

I recommend that you **always** put parentheses around a ``yield`` expression
It's recommended that you **always** put parentheses around a ``yield`` expression
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current wording is more concise. It's also more precise -- with your wording, I'm left wondering who exactly is recommending this? Lots of things in this article are just one person's opinion, and never claim to be anything else, but with your rewording here, it makes it sound like it's the official pronouncement from on high about the proper programming style for Python. Removing the usage of the first person throughout the article really muddies the water here about the purpose of this article, in my opinion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If they are just one person's opinion, they don't belong in the documentation. The documentation's contract with the user is exactly that: that it is the authoritative pronouncement from on-high about programming in Python. We would never allow the code to be so permissive and lax, to represent merely the way someone feels it's good to do it, and we should hold documentation to the same standards.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If they are just one person's opinion, they don't belong in the documentation.

Says who? Again, that's not how the CPython project has traditionally thought of HOWTO documents, and it's not what the Doc/howto/ directory in CPython has historically been used for. If you want to substantially rewrite a large portion of the articles in Doc/howto/ based on a new, much stricter idea of what it means for a document to be a HOWTO guide, then I honestly think you should try to achieve a broader consensus among the community before going ahead with something like this. I haven't seen any discussions on Discourse about this, for example.

Personally, I think it's great that we have a space in the CPython documentation for more subjective, personal reflections on programming practices. I'd much prefer we say "it's not a how-to" and move it to some other directory than try to turn the document into something it's not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If they are just one person's opinion, they don't belong in the documentation.

Says who? Again, that's not how the CPython project has traditionally thought of HOWTO documents, and it's not what the Doc/howto/ directory in CPython has historically been used for. If you want to substantially rewrite a large portion of the articles in Doc/howto/ based on a new, much stricter idea of what it means for a document to be a HOWTO guide, then I honestly think you should try to achieve a broader consensus among the community before going ahead with something like this. I haven't seen any discussions on Discourse about this, for example.

I understood - I believe - that there is consensus on adopting Diátaxis for Python's documentation. There have been discussions, I ran a couple of workshops with the documentation team, and so on.

It's a big project, and it's going to take a long time. My goal is to help set new standards of quality in Python's documentation. It is going to change a lot, and set new expectations. There isn't any other way around this.

Personally, I think it's great that we have a space in the CPython documentation for more subjective, personal reflections on programming practices. I'd much prefer we say "it's not a how-to" and move it to some other directory than try to turn the document into something it's not.

Yes, that is precisely what I described as the goal in the original commit for this pull request. These are steps along the way. But even explanation-type documentation, which is what this particular document is, needs to have authority. Otherwise it is simply not documentation.

Copy link
Member

@AlexWaygood AlexWaygood Aug 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understood - I believe - that there is consensus on adopting Diátaxis for Python's documentation.

I've liked a lot of what I've seen from the Diataxis work on the CPython docs -- I think it's resulted in some very worthwhile improvements to the docs, and I'm excited to see it continue! We need to consider each change individually, however. There may be broad consensus about using Diataxis as a framework for the CPython docs overall, but I don't really feel like I've seen any kind of consensus-building about what to do with these older HOWTO documents specifically. Endorsement of a general principle/framework is a pretty different thing to endorsing a specific application of that framework that arguably involves redefining the purpose of an entire directory.

I understand that the pace of the change here might be frustrating for you, and I'm sorry for that, but careful review of each change really is necessary, in my opinion.

I ran a couple of workshops with the documentation team

(Super minor nitpick, but there is no "documentation team". There's a docs community -- a group of people who are interested and enthusiastic about improving the CPython docs. But docs contributions come from a wide variety of people, many of whom aren't part of that community, and there are lots of core developers interested in documentation who aren't part of that community. I think it would be a mistake to assume that the people in that community are necessarily representative of the Python community as a whole 🙂)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that is precisely what I described as the goal in the original commit for this pull request.

I don't understand -- if the goal is to move this document out of the Doc/howto/ directory, then why do you keep on making arguments based on what a HOWTO is "supposed to be"? If the hope is to reclassify this document altogether, then surely we should do that first, so that we have a clear idea of what the document is meant to be while we're editing it?

when you're doing something with the returned value, as in the above example.
The parentheses aren't always necessary, but it's easier to always add them
instead of having to remember when they're needed.
Expand Down Expand Up @@ -608,8 +593,8 @@ generators:
will also be called by Python's garbage collector when the generator is
garbage-collected.

If you need to run cleanup code when a :exc:`GeneratorExit` occurs, I suggest
using a ``try: ... finally:`` suite instead of catching :exc:`GeneratorExit`.
If you need to run cleanup code when a :exc:`GeneratorExit` occurs, it's recommended
to use a ``try: ... finally:`` suite instead of catching :exc:`GeneratorExit`.
Comment on lines +596 to +597
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer the current wording, which is more concise, and more upfront about the fact that this is one person's opinion.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I doubt I had a personal opinion about catching vs. try...finally, & vaguely think I got the suggestion from somewhere. Could it have been from python-dev discussion when the feature was being added? Or maybe it was just general concern about the complexity of dealing with the GeneratorExit; do you need to re-raise it once your cleanup is done? Maybe this should say "it's simpler to use try...finally" instead of 'recommended'? (And do 3.x async features change any of this?)


The cumulative effect of these changes is to turn generators from one-way
producers of information into both producers and consumers.
Expand Down Expand Up @@ -639,7 +624,7 @@ features of generator expressions:
>>> [upper(s) for s in ['sentence', 'fragment']]
['SENTENCE', 'FRAGMENT']

You can of course achieve the same effect with a list comprehension.
(You can achieve the same effect with a list comprehension.)

:func:`filter(predicate, iter) <filter>` returns an iterator over all the
sequence elements that meet a certain condition, and is similarly duplicated by
Expand Down Expand Up @@ -1131,20 +1116,21 @@ usual way::
def print_assign(name, value):
return name + '=' + str(value)

Which alternative is preferable? That's a style question; my usual course is to
avoid using ``lambda``.
Which alternative is preferable? That's mostly a question of style.
Comment on lines -1134 to +1119
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to me to be an unnecessary edit


One reason for my preference is that ``lambda`` is quite limited in the
You may wish to avoid using ``lambda``, as there are limits to the
Comment on lines -1137 to +1121
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't find this necessary

functions it can define. The result has to be computable as a single
expression, which means you can't have multiway ``if... elif... else``
comparisons or ``try... except`` statements. If you try to do too much in a
``lambda`` statement, you'll end up with an overly complicated expression that's
hard to read. Quick, what's the following code doing? ::
hard to read.

Consider::
Comment on lines -1142 to +1128
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of a rhetorical question aimed at the reader has the effect of shaking up the pace of the prose by suddenly throwing in a different kind of sentence construction. I think it's an enjoyable effect, and don't think it needs to be removed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, most of the changes above to remove 'I suggest' or 'I think' are fine with me, but here the sentence is trying to ask a somewhat punchy question.


import functools
total = functools.reduce(lambda a, b: (0, a[1] + b[1]), items)[1]

You can figure it out, but it takes time to disentangle the expression to figure
It takes some mental effort to disentangle the expression to figure
out what's going on. Using a short nested ``def`` statements makes things a
little bit better::

Expand All @@ -1154,7 +1140,7 @@ little bit better::

total = functools.reduce(combine, items)[1]

But it would be best of all if I had simply used a ``for`` loop::
But best of all would have been to use a ``for`` loop::

total = 0
for a, b in items:
Expand All @@ -1166,19 +1152,6 @@ Or the :func:`sum` built-in and a generator expression::

Many uses of :func:`functools.reduce` are clearer when written as ``for`` loops.

Fredrik Lundh once suggested the following set of rules for refactoring uses of
``lambda``:

1. Write a lambda function.
2. Write a comment explaining what the heck that lambda does.
3. Study the comment for a while, and think of a name that captures the essence
of the comment.
4. Convert the lambda to a def statement, using that name.
5. Remove the comment.

I really like these rules, but you're free to disagree
about whether this lambda-free style is better.

Comment on lines -1169 to -1181
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why this section needs to be deleted. It seems to me to be a useful way of thinking about the problem.

It's true that it's written in a slightly more informal style, and a slightly more humorous style, than a lot of the rest of Python's documentation. But Python's Doc/howto/ directory has historically been home to articles that can be written in a freer, more conversational tone than the rest of Python's documentation.

I disagree with retrofitting a new, stricter definition of what it means for a document to be a HOWTO guide onto Python's Doc/howto/ directory, which has historically had a different concept of what how-to guides are and should be.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A key problem with this page of documentation is that it is not a how-to guide. Whatever else needs to be done with it, it's in the wrong place. If a it does something other than guide the user through a task or problem, it's literally not a how-to guide. Names do matter.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A key problem with this page of documentation is that it is not a how-to guide.

It is a guide that explains how to apply the functional programming paradigm to Python code

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of these HOWTOs, like this one and the Unicode one, were intros/justifications at the time for newly introduced language features, but now could use editing to just describe the current feature set, with less emphasis on their newness.

I was also hoping that this would encourage others to write HOWTOs on particular topics such as secure Python programming, but that didn't work and sites like Stack Overflow largely took over that function. They could be called something like 'Learning paths' or 'topic guides', but no better term ever became clear.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, here's a more detailed reference for Fredrik Lundh's comment, dated April 1 2001, if you wanted to link it.
https://groups.google.com/g/comp.lang.python/c/qdGhqsmmieo/m/_piGPkADYTgJ

Should it describe Fredrik somehow, as "long-time Python contributor"? Should it specify "the late"?


Revision History and Acknowledgements
Copy link
Contributor Author

@evildmp evildmp Jul 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it necessary to keep this "Revision History and Acknowledgements" section?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the history can be removed, Acknowledgements could be moved to the end (after References) or removed. The references section should also probably have the 'General' subhead removed (I can't add in-line comments there).

Copy link
Member

@akuchling akuchling Sep 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed; we can drop it.

We could also drop the version number at the top of the file.

=====================================
Expand Down
Loading