Add deploy preview for docs

[epicfaace]

BPO	37860
Nosy	@willingc, @ewdurbin, @Mariatta, @isidentical, @epicfaace, @aeros
PRs	bpo-37860: Add netlify deploy preview for docs #15288 bpo-37860: Update netlify.toml #18242 bpo-37860: Enable docs deploy previews through GitHub Actions #19025

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-08-14.21:14:46.493>
labels = ['3.9', 'docs']
title = 'Add deploy preview for docs'
updated_at = <Date 2021-03-22.01:23:26.366>
user = 'https://github.com/epicfaace'

bugs.python.org fields:

activity = <Date 2021-03-22.01:23:26.366>
actor = 'epicfaace'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2019-08-14.21:14:46.493>
creator = 'epicfaace'
dependencies = []
files = []
hgrepos = []
issue_num = 37860
keywords = ['patch']
message_count = 7.0
messages = ['349752', '350084', '364264', '364266', '364271', '368201', '389269']
nosy_count = 7.0
nosy_names = ['docs@python', 'willingc', 'EWDurbin', 'Mariatta', 'BTaskaya', 'epicfaace', 'aeros']
pr_nums = ['15288', '18242', '19025']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue37860'
versions = ['Python 3.9']

[epicfaace]

It would be good to preview the cpython documentation on PRs using Netlify.

See python/core-workflow#348

[willingc]

New changeset 87bc3b7 by Carol Willing (Ashwin Ramaswami) in branch 'master':
bpo-37860: Add netlify deploy preview for docs (GH-15288)
87bc3b7

[epicfaace]

Given Ernest's comment in #15288 (comment) about the limitations of Netlify (1000 build minutes per month), it doesn't seem like Netlify will be a viable alternative.

Instead, I've added a PR (#19025) that takes an alternative approach. It uses GitHub Actions to deploy the docs to the gh-pages branch, then adds a comment to the PR with a link to the deploy preview.

The way this works is that the gh-pages branch has subfolders with the SHA of each commit, and each deploy preview is deployed to each subfolder.

I've enabled this on my fork of cpython -- here's an example PR on my fork: epicfaace#7

[epicfaace]

GitHub Pages has no limit on "build limits" like Netlify does.

One limitation of this approach, though, is that GitHub Pages sites are limited to 1 GB in size. Each doc build is 50 MB, meaning that we could host at a maximum 20 builds. If we needed to, we could resolve this by doing one (or multiple) of the following:

• check with GitHub to see if they will increase the limit for this case
• only build PRs that have been tagged "documentation"
• add a workflow to automatically remove deploy previews after a certain amount of time has passed or after a PR has been merged
• don't deploy on GitHub Pages, but use something like https://surge.sh/, which apparently doesn't have a hard limit on size of a static site

[aeros]

only build PRs that have been tagged "documentation"

My preference would be to have a "preview-with-netlify" label that could be manually added to use the netlify preview on a case-by-case basis. There are a substantial number of trivial PRs (such as typo fixes and similar minor changes) that are automatically labelled as "documentation" which wouldn't particularly benefit from the netlify preview and would needlessly occupy part of the limit.

Personally, I feel that it most largely benefits more significant documentation changes that involve modification of existing markup or the addition of new markup, rather than all documentation PRs. So a manually added label to trigger it makes the most sense to me. This would be very similar to the recently added "test-with-buildbots" label.

[isidentical]

Personally, I feel that it most largely benefits more significant documentation changes that involve modification of existing markup or the addition of new markup, rather than all documentation PRs. So a manually added label to trigger it makes the most sense to me. This would be very similar to the recently added "test-with-buildbots" label.

+1, It would be so much useful in cases like GH 19928.

[epicfaace]

I realized there's another alternative which is probably simpler. We can use Netlify to host the deploy previews, but we can use GitHub Actions to build the documentation. This way, we won't run into the build time limits for Netlify, and we can still use Netlify to host deploy previews.

We can use this GitHub Actions action: https://github.com/marketplace/actions/netlify-actions, and just add it as an additional step to our existing doc GitHub Actions workflow (https://github.com/python/cpython/blob/master/.github/workflows/doc.yml).

What do you think about this approach?

[hugovk]

@epicfaace Hi again! Building on GitHub Actions and deploying to Netlify definitely sounds promising and definitely worth a try!

Like you say, we're already building the docs in our own workflow and so it would be good to see how long the upload takes.

[hugovk]

I looked into the Netlify action, but there's a blocker; it requires a Netlify personal access token / NETLIFY_AUTH_TOKEN as a secret:

https://github.com/nwtgck/actions-netlify#required-inputs-and-env

And GitHub doesn't make secrets available to PRs from forks, which unfortunately would limit us to uploads only for core devs and only when they created a branch in this repo.

I guess we could hardcode the token without using secrets, but that doesn't sound like a great idea.

---

Another option, Read the Docs can now build previews for PRs:

https://docs.readthedocs.io/en/stable/pull-requests.html

We're using for PR previews in the PEPs repo, even though the main site is hosted elsewhere (on GitHub Pages).

For example:

• PR PEP 691: Draft PEP for Simple JSON API peps#2578
• created a preview at https://pep-previews--2578.org.readthedocs.build/
• allowing that new PEP to be previewed at https://pep-previews--2578.org.readthedocs.build/pep-0691/

As a demo, I've set it up for my fork:

• https://readthedocs.org/projects/hugovk-cpython/builds/
• Demo PR: RTD PR preview demo hugovk/cpython#31
• Demo docs build: https://readthedocs.org/projects/hugovk-cpython/builds/16854291/
• Demo docs: https://hugovk-cpython--31.org.readthedocs.build/
• Demo changed page: https://hugovk-cpython--31.org.readthedocs.build/whatsnew/3.11.html

I would recommend naming it something like cpython-docs-previews to make it clear it's not the official docs home, similar to PEPs using pep-previews.

(See also python/docs-community#5 for hosting the live docs on RTD, which is another but related matter. This could be a first step towards that.)

Thoughts on using RTD for PR previews?

[pradyunsg]

Why do you want to build on GitHub Actions? Netlify provides its own building compute (similar to RTD) and has an open source plan that CPython would qualify for: https://www.netlify.com/legal/open-source-policy/. The build is configured by specifying a build command and a publish directory -- https://docs.netlify.com/configure-builds/overview/?#basic-build-settings.

Netlify has deploy previews functionality baked into their offering: https://docs.netlify.com/site-deploys/deploy-previews/

[hugovk]

@pradyunsg See #82041 (comment) and #15288 (comment):

We have a Pro level open source plan on Netlify, but the limits of 1,000 build minutes per month likely aren't enough.

But wait!

That was in January 2000 and it now looks like there's 25k minutes per month!

If the estimates in #15288 (comment) are still accurate, that would give us something like 4,250 builds per month.

There were 660 PRs created last month, a few would have multiple builds (and some older PRs would have builds too), but I expect 4k builds should be plenty.

[epicfaace]

That was in January 2000 and it now looks like there's 25k minutes per month!

Let's do it then!

[epicfaace]

@hugovk @pradyunsg I added a PR in #92852.