Skip to content
This repository has been archived by the owner on Aug 19, 2023. It is now read-only.

Add doc8 run to CI and lint checks #52

Merged
merged 7 commits into from
Feb 21, 2019

Conversation

mtreinish
Copy link
Member

Summary

This commit adds running doc8 to our normal CI and lint workflows. Right
now we don't have any doc verification (aside from what pylint enforces
on docstrings in the tests, which doesn't get published anywhere) and when
it comes time to publish a new release there is a scramble to fix all the
warnings (and potentially errors) in the documentation. By running doc8 we
can verify that our documentation is formatted properly for sphinx to
compile the output without issue (and without any warnings).

Details and comments

The ipython requirement added in the tox.ini is necessary for doc8/pygments to be able to parse the jupyter noteboook code examples. If ipython isn't installed it's not able to parse the code blocks.

jaygambetta
jaygambetta previously approved these changes Jan 31, 2019
@mtreinish
Copy link
Member Author

@jaygambetta ugh, the aqua docs introduced 1938 lint issues: https://travis-ci.com/Qiskit/qiskit/jobs/174263223#L2538 I'll clean things up so we can merge this and prevent this in the future.

@jaygambetta
Copy link
Member

great. I figure get the content there and then we can clean up

Copy link
Member

@diego-plan9 diego-plan9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we regroup before introducing this check, and we see how it fits in the automation plans? While getting some consistency in the .rst files is a good thing, there are several factors involved:

  • doc8 might not be comprehensive enough - a quick make doc seems to result in ~80 sphinx warnings of different nature and severity. If we want to be comprehensive, we could use the output of make doc directly.
  • actually related to the previous topic - linting of .rst files in terra was informally considered at some point, but the impression was that the barrier of entry for working with .rst files can be actually higher than working with .py files (as in, fixing Sphinx warnings is usually more tricky and hard to decipher than fixing a Python style warning).
  • depending on the policy for storing the content coming from the tutorials in the repo, linting for those files would either need to be "replicated", or just discarded (mostly a practical concern).

@mtreinish
Copy link
Member Author

Doc8 should be fine, it's what I use locally to verify all my rst files that I've pushed to terra and I never had any issues with warnings on doc builds after. The only things doc8 wouldn't catch is any issues in dynamically generated content (like autodocs and tutorials after #42 ). But that tradeoff is fine, because there won't be code submissions for that content to this repo (since they are made to other repos).

I picked doc8 here to be lightweight syntax and lint checker so we didn't have to run sphinx build on every proposed commit. If we want to run make doc and fail on warnings that will accomplish the same goal (albeit much slower) and I'm fine with that. Although, it will also likely introduce external dependency issues when docstring or tutorial builds fail because of changes in those other repos.

This patch just isn't done yet I pushed up my work before lunch to save it. The aqua docs had a ton of issues that I needed to cleanup to make this pass.

Also for tutorials from nbconvert there isn't normally any formatting issues aside from the occasional line length failure (but that's normally from when I run locally with 80 chars, the true line length :) ).

@mtreinish
Copy link
Member Author

@jaygambetta this should be good to go now, I fixed all the issues reported by doc8 on the current master

@diego-plan9
Copy link
Member

Adding some numbers to better aid the discussion, comparing the make doc output of master and the changes in this PR - mostly using grep WARNING over the full documentation:

master PR 52
total warnings 333 309
docstring warnings 245 245
non-docstring warnings 88 64
tox lint time 33s 83s

The numbers are just for reference and might vary a bit, but I think shed some light about the tradeoffs: our main source of warnings is the autodocs (ie. the docstrings), which are not tackled by the automatic linting - and for the non-docstring warnings, the linter is able to catch 22 of them, but most of them (64) still remain. Looking at the big picture and at the current PR and versions, we are arguably trading off ~50s in each build plus some strain on each contributor for solving ~7% of the warnings, which I personally think is not a good balance.

I think we basically will not be able to escape doing a round of manual fixing for those remaining warnings at one point or another, as the linter would not be able to pick a number of them, and would be more efficient to take advantage of that. What about:

  • have a plan for revising the warnings periodically (for example, before a major release) and do some style fixes locally if needed
  • place some effort in collaborating with each component's team for fixing the docstrings
  • for practical purposes, turn this PR into the first of those periodical ones (ie. keep the .rst changes themselves, to avoid losing the work and also aim for merging soon to avoid conflict solving, which I assume has not been too fun)

(For master, a341df0 was used; for the PR column, 2727dff was used (rendering both with the full stable autodocs, to avoid missing file warnings and replicate what finally gets pushed to the landing site). The lint time is based on the travis builds, although it probably has a wide variance.)

@jaygambetta
Copy link
Member

jaygambetta commented Feb 15, 2019

I see @diego-plan9 point but for this repo, it should not be run too many times. I am good with this addition as it picks up a few more errors. If we start to see this take too much time of travis we should reconsider this change.

@mtreinish can you fix the conflicts and then i approve so you can merge

@mtreinish mtreinish force-pushed the run-doc8-for-docs branch 2 times, most recently from 3f39c39 to 38e5153 Compare February 19, 2019 20:01
This commit adds running doc8 to our normal CI and lint workflows. Right
now we don't have any doc verification (aside from what pylint enforces
on docstrings in the tests, which doesn't get published anywhere) and when
it comes time to publish a new release there is a scramble to fix all the
warnings (and potentially errors) in the documentation. By running doc8 we
can verify that our documentation is formatted properly for sphinx to
compile the output without issue (and without any warnings).
@mtreinish
Copy link
Member Author

@jaygambetta This can merge now

@jaygambetta jaygambetta dismissed diego-plan9’s stale review February 21, 2019 13:19

diego and I discuss and we will see how performance goes

@jaygambetta jaygambetta merged commit 8626dd4 into Qiskit:master Feb 21, 2019
@mtreinish mtreinish deleted the run-doc8-for-docs branch February 21, 2019 13:20
jakelishman pushed a commit to jakelishman/qiskit-terra that referenced this pull request Aug 11, 2023
* Add doc8 run to CI and lint checks

This commit adds running doc8 to our normal CI and lint workflows. Right
now we don't have any doc verification (aside from what pylint enforces
on docstrings in the tests, which doesn't get published anywhere) and when
it comes time to publish a new release there is a scramble to fix all the
warnings (and potentially errors) in the documentation. By running doc8 we
can verify that our documentation is formatted properly for sphinx to
compile the output without issue (and without any warnings).

* Fix lint issues with aqua docs

* Fix most of the aqua docs lint issues

* Fix last lint issues

* Fix lint from rebase

* More rebase cleanups for new docs

* More rebase fixes
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants