What to do about old versions of the docs on readthedocs?

[nicoddemus]

Yeah recently we had a few cases of misunderstandings because of old links.

Usually we only really keep two docs up-to-date, which is stable and latest.

What should we do with old documentation pages?

Btw the docs mentioned in the original issue are quite old, from the reorganize-docs branch, back in 2016. I don't think we will really pick it up again at this point, perhaps we can just delete them to avoid confusion? @pfctdayelise @hackebrot?

Originally posted by @nicoddemus in #8094 (comment)

[pfctdayelise]

Good idea to avoid confusion, thanks @nicoddemus !

[RonnyPfannschmidt]

btw, do we still have the notes from back then, i believe we should try to make what we set out back in the sprint possible in a more iterative small pieced fashion

[pfctdayelise]

Hmm...the broad idea I believe was to rearrange the docs to distinguish between

• Tutorial
• How-to guide
• Discussions
• Reference

This is described by Daniel Procida in a talk https://pyvideo.org/pycon-au-2017/what-nobody-tells-you-about-documentation.html (although apparently that was 2017, so maybe my brain just made this up....)

Our notes about it are here https://github.com/pytest-dev/pytest/wiki/Docs-refactor , where we didn't mention that, so maybe we were just planning to organise a bit and categorise according to level of user. I do think the docs are still a bit overwhelming and there is not that much navigation help. The reference material has improved a bit though :) not sure if it is complete yet!

There is also a picture of our sticky notes for planned organisation, in the blog post - https://blog.pytest.org/2016/pytest-development-sprint/

[aklajnert]

Since the users are redirected to the old docs from Google, maybe instead of removing them, it would be a good idea to redirect to the new one or display a huge banner, that these docs are for the old version with a link to the new one?

[nicoddemus]

Since the users are redirected to the old docs from Google, maybe instead of removing them, it would be a good idea to redirect to the new one or display a huge banner, that these docs are for the old version with a link to the new one?

Indeed that's ideal, however that setting is already configured:

Perhaps the banner is not showing up because of our theme?

EDIT:

Hold on, actually the banner shows up: https://docs.pytest.org/en/6.0.0/reference.html

however it doesn't show for the link in the original issue: https://docs.pytest.org/en/reorganize-docs/new-docs/user/naming_conventions.html. I also can't find a reorganize-docs version in readthedocs at all:

https://readthedocs.org/projects/pytest/versions/

I suspect the docs for the reorganize-docs branch are still present in the old docs server... does that make sense?

[Zac-HD]

For Hypothesis, we only host "latest", specificially so that stale docs can't show up in search results etc., and I'd support "only stable, latest, and 4.x" for pytest for the same reasons.

(we're also hosting the last py2 docs for now, but free support only applies to the latest release and nobody pays, so 🤷‍♂️)

[nicoddemus]

I'd support "only stable, latest, and 4.x" for pytest for the same reasons.

👍 from me. Does anybody see a good reason to host other versions?

[asottile]

I'd support "only stable, latest, and 4.x" for pytest for the same reasons.

👍 from me. Does anybody see a good reason to host other versions?

sg2m!

[Zac-HD]

Sounds like all we need is for someone with edit access (e.g. @nicoddemus, @pfctdayelise, @RonnyPfannschmidt, @asottile) to go to https://readthedocs.org/projects/pytest/versions/ while logged in, then edit -> uncheck active -> save for all the older versions. I'd be happy to but don't have write access.

[asottile]

whew that was a lot of clicking -- but I think it's done

[Zac-HD]

🎉

[PhilippSelenium]

Imagine the following scenario:
I have a working sphinx and CI build where pytest is pinned to i.e. 6.2.0
Now pytest does a major version upgrade (i.e. 7.0.0) (and some interfaces were changed/removed/...)
Despite having pytest pinned my sphinx build will break :(
And I have no option but to update to the newest version asap ...

[Zac-HD]

I'm sorry that this is a potentially breaking change for you 😢

However we do think that the reduced risk of confusion for less experienced users outweighs the potential breaking change for your docs. You can also work around this by saving a local copy, building Pytest docs from a git submodule, etc., whereas there's no equivalent workaround for the problem of outdated documentation appearing in search results.

[PhilippSelenium]

@Zac-HD ... the needs of the many ... ;)
Thanks for the hint about the workaround we are already doing that.

[RonnyPfannschmidt]

Read the docs could perhaps use a add on to help referring to current docs and making searches ignore outdated docs

[The-Compiler]

It looks like the reorganize-docs branch is still around on RTD and apparently turns up in Google search results (I just had a coworker ask why the snippet from the docs wouldn't work). Is there something we can do about this?

[RonnyPfannschmidt]

Let's take a look at dropping, once I get back to the computer

[Zac-HD]

Just checking in on this - it's gone from readthedocs, but still available on pytest.org - and causing trouble in e.g. #8655.

Possibly stale files on a server somewhere, since the readthedocs version selector is missing too?

[The-Compiler]

Well, docs.pytest.org points to readthedocs.io:

$ dig docs.pytest.org A     
[...]

;; ANSWER SECTION:
docs.pytest.org.	262	IN	CNAME	readthedocs.io.
readthedocs.io.		262	IN	A	104.17.32.82
readthedocs.io.		262	IN	A	104.17.33.82

so either we're missing something from those settings, or it's a bug on their side...

[The-Compiler]

I've tried adding a prefix redirect /en/reorganize-docs/ -> /en/6.2.x/ in RTD as a workaround, but looks like that didn't change anything either (so I now removed it again)...

[The-Compiler]

FYI, I opened readthedocs/readthedocs.org#8264 for this now (cc @kini)

[RonnyPfannschmidt]

@The-Compiler if all fails, how about recreating the branch once from main and dropping it once regendoc can actually see it and regenerated it again (only if they cant fix the issue timely)

[The-Compiler]

I assume you mean rtd, not regendoc? Either way, seems like a very temporary solution, but perhaps worth a try. On the other hand, we've been living with this for years, so it's not like another week or two makes a difference, as long as it gets fixed.

[The-Compiler]

The good news: I got things sorted out, the old docs are gone, and going to e.g. reorganize-docs/index.html will redirect to 6.2.x/index.html.

The bad news: It falls apart for pages which have a redirect due to #8471 (comment) - for example, reorganize-docs/mark.html will incorrectly redirect to 6.2.x/how-to/mark.html which is still a 404.

However, this should fix itself once we got 7.0 released, as at that point we'll likely switch back to /stable/ / /latest/ for the docs, and the docs will exist at the new path (with how-to).

Still I've reported this to RTD here: readthedocs/readthedocs.org#8309