Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Infra: generate machine-readable PEP index #2475

Merged
merged 5 commits into from
Apr 1, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions pep_sphinx_extensions/pep_zero_generator/parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,20 @@ def __init__(self, filename: Path, authors_overrides: dict):
# Parse PEP authors
self.authors: list[Author] = _parse_authors(self, metadata["Author"], authors_overrides)

# Other headers
self.created = metadata["Created"]
self.discussions_to = metadata["Discussions-To"]
self.python_version = metadata["Python-Version"]
self.replaces = metadata["Replaces"]
self.requires = metadata["Requires"]
self.resolution = metadata["Resolution"]
self.superseded_by = metadata["Superseded-By"]
if metadata["Post-History"]:
# Squash duplicate whitespace
self.post_history = " ".join(metadata["Post-History"].split())
else:
self.post_history = None

def __repr__(self) -> str:
return f"<PEP {self.number:0>4} - {self.title}>"

Expand Down
33 changes: 30 additions & 3 deletions pep_sphinx_extensions/pep_zero_generator/pep_index_generator.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
from __future__ import annotations

import csv
import json
from pathlib import Path
import re
from typing import TYPE_CHECKING
Expand All @@ -30,9 +31,29 @@
from sphinx.environment import BuildEnvironment


def create_pep_zero(_: Sphinx, env: BuildEnvironment, docnames: list[str]) -> None:
# Sphinx app object is unneeded by this function

def create_pep_json(peps: list[parser.PEP]) -> str:
pep_dict = {
pep.number: {
"title": pep.title,
"authors": ", ".join(pep.authors.nick for pep.authors in pep.authors),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to make this an array of nicks, so users don't need to split the string again?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, all of the fields are strings (or null), and match the literal value of the headers (minus cleaned-up whitespace). if we're going to do this, it would be inconsistent if we didn't process the other headers as well into more structured formats, which would be a good idea but better left to a future PR. With #2484 , tools can rely on each of the headers to match a certain format, so they are easier and more consistent to work with as strings and can be split with just .split(","), without having to worry much about edge cases.

As mentioned above,

In the future, if there's any interest, we could actually parse the fields into more structured formats, especially once my PRs land that fully validate their format for much easier and trouble-free internal and external use.

"discussions_to": pep.discussions_to,
"status": pep.status,
"type": pep.pep_type,
"created": pep.created,
"python_version": pep.python_version,
"post_history": pep.post_history,
"resolution": pep.resolution,
"requires": pep.requires,
"replaces": pep.replaces,
"superseded_by": pep.superseded_by,
"url": f"https://peps.python.org/pep-{pep.number:0>4}/",
}
for pep in sorted(peps)
}
return json.dumps(pep_dict, indent=1)


def create_pep_zero(app: Sphinx, env: BuildEnvironment, docnames: list[str]) -> None:
# Read from root directory
path = Path(".")

Expand Down Expand Up @@ -63,3 +84,9 @@ def create_pep_zero(_: Sphinx, env: BuildEnvironment, docnames: list[str]) -> No
docnames.insert(1, pep_zero_filename)
# Add to files for writer
env.found_docs.add(pep_zero_filename)

# Create peps.json
pep0_json = create_pep_json(peps)
out_dir = Path(app.outdir) / "api"
out_dir.mkdir(exist_ok=True)
Path(out_dir, "peps.json").write_text(pep0_json, encoding="utf-8")