DOC: ongoing fixing of doctests

[jorisvandenbossche]

Starting with #19952, we run (some) doctests on Travis to ensure the example are kept up to date.
Currently, we only test the docstrings for Series and DataFrame, and reshape-related top level functions, and we also skip a lot of the tests.

So PRs to incrementally test more of the docstrings are very welcome, and can be good bite-sized newbie issues.

The workflow would be like this:

• Take a certain function or a certain module

• Fix the examples in the docstring(s):

  • To run the doctest for a single method, you can either use the validation script, eg:

    python scripts/validate_docstrings.py pandas.DataFrame.mean
    

    to run a full module or a subset of them, you can use something like:

    pytest --doctest-modules pandas/core/frame.py -k DataFrame
    

  • It is certainly welcome to also fix other problems with the docstring (all other errors and warnings that the validation script gives), or to actually improve the example. But I think it is also fine to do PRs just fixing the doctests as they are, so we at least can test them (so if that then somebody else improves the examples, we are sure those new examples are actually tested).

  • Some more details can be found here: http://pandas-docs.github.io/pandas-docs-travis/contributing.html#updating-a-pandas-docstring. And specifically, some tips on getting doctests to pass: http://pandas-docs.github.io/pandas-docs-travis/contributing_docstring.html#tips-for-getting-your-examples-pass-the-doctests

• When the doctests pass for a certain function or module, add those (or remove the skip) to the ones that are tested on Travis.
  For that, you need to edit the ci/code_checks.sh file. For example, for the Series tests, it looks like:

  pandas/ci/code_checks.sh

  Lines 125 to 126 in cf11f71

  	pytest --doctest-modules -v pandas/core/series.py \
  	-k"-nonzero -reindex -searchsorted -to_dict"

  Here, you could remove one of the skipped ones (the -k"-agg" syntax is to skip the doctests for Series.agg and Series.aggregate). Or add such a new line for a new module.

Good docstrings to start with are the ones for Series and DataFrame (the ones located in pandas/core/series.py, pandas/core/frame.py and pandas/core/generic.py).
After that, adding new modules like the indexing ones would be a possibility.

[Roald87]

I was looking into improving the docstrings of Series.max() and have some questions about adding the examples. Currently the method _make_stat_function is used to generate the docstring. However, this method doesn’t allow for an example to be added (as far as I can tell). Is it needed to create a new method for the min, max, std, mean etc. methods, similar to _make_cum_function for the cum methods?

Also should there be separate examples/docstrings for Series.max and DataFrame.max?

[TomAugspurger]

I think you need to have _make_stat_function take an examples argument and pass that through. See how _make_cum_function and cumsum do it.

Also should there be separate examples/docstrings for Series.max and DataFrame.max?

No, we're ok with mixing those.

[Moisan]

Note that since f6da1f1, ci/doctests.sh content (for skipping or adding a doctests to the CI) has been moved to ci/code_checks.sh.

[jorisvandenbossche]

@Moisan Thanks for noticing, updated the text

[stijnvanhoey]

[europandas sprint 2019] I'm working on the doctests of frame.py DataFrame

[jbrockmendel]

i see a lot of merged PRs, is this done?

[datapythonista]

i see a lot of merged PRs, is this done?

Looks like there are still some docstrings with errors:

FAILED pandas/_config/config.py::pandas._config.config.option_context
FAILED pandas/_testing/__init__.py::pandas._testing._make_timeseries
FAILED pandas/_testing/_io.py::pandas._testing._io.network
FAILED pandas/_testing/asserters.py::pandas._testing.asserters._get_tol_from_less_precise
FAILED pandas/_testing/contexts.py::pandas._testing.contexts.set_timezone
FAILED pandas/core/flags.py::pandas.core.flags.Flags.allows_duplicate_labels
FAILED pandas/core/resample.py::pandas.core.resample.Resampler.aggregate
FAILED pandas/core/resample.py::pandas.core.resample.Resampler.transform
FAILED pandas/core/arrays/datetimelike.py::pandas.core.arrays.datetimelike.DatelikeOps.strftime
FAILED pandas/core/internals/construction.py::pandas.core.internals.construction.dataclasses_to_dicts
FAILED pandas/io/stata.py::pandas.io.stata.read_stata
FAILED pandas/io/formats/format.py::pandas.io.formats.format.EngFormatter.__call__
FAILED pandas/io/formats/style.py::pandas.io.formats.style.Styler.apply
FAILED pandas/io/formats/style.py::pandas.io.formats.style.Styler.applymap
FAILED pandas/io/formats/style.py::pandas.io.formats.style.Styler.background_gradient
FAILED pandas/io/formats/style.py::pandas.io.formats.style.Styler.hide_columns
FAILED pandas/io/formats/style.py::pandas.io.formats.style.Styler.hide_index
FAILED pandas/io/formats/style.py::pandas.io.formats.style.Styler.highlight_between
FAILED pandas/io/formats/style.py::pandas.io.formats.style.Styler.highlight_quantile
FAILED pandas/io/formats/style.py::pandas.io.formats.style.Styler.pipe
FAILED pandas/io/formats/style.py::pandas.io.formats.style.Styler.set_properties
FAILED pandas/io/formats/style.py::pandas.io.formats.style.Styler.set_table_attributes
FAILED pandas/io/formats/style.py::pandas.io.formats.style.Styler.set_table_styles
FAILED pandas/io/formats/style.py::pandas.io.formats.style.Styler.set_td_classes
FAILED pandas/io/formats/style.py::pandas.io.formats.style.Styler.set_tooltips
FAILED pandas/io/formats/style.py::pandas.io.formats.style.Styler.text_gradient
FAILED pandas/io/formats/style.py::pandas.io.formats.style.Styler.to_latex
FAILED pandas/io/formats/style.py::pandas.io.formats.style.Styler.where
FAILED pandas/io/formats/style_render.py::pandas.io.formats.style_render.StylerRenderer.format
FAILED pandas/io/formats/style_render.py::pandas.io.formats.style_render._parse_latex_header_span
FAILED pandas/io/formats/style_render.py::pandas.io.formats.style_render._parse_latex_table_styles
FAILED pandas/plotting/_core.py::pandas.plotting._core.PlotAccessor.line
FAILED pandas/plotting/_core.py::pandas.plotting._core.boxplot
FAILED pandas/plotting/_core.py::pandas.plotting._core.boxplot_frame
FAILED pandas/plotting/_core.py::pandas.plotting._core.boxplot_frame_groupby
FAILED pandas/plotting/_misc.py::pandas.plotting._misc.andrews_curves
FAILED pandas/plotting/_misc.py::pandas.plotting._misc.autocorrelation_plot
FAILED pandas/plotting/_misc.py::pandas.plotting._misc.bootstrap_plot
FAILED pandas/plotting/_misc.py::pandas.plotting._misc.lag_plot
FAILED pandas/plotting/_misc.py::pandas.plotting._misc.parallel_coordinates
FAILED pandas/plotting/_misc.py::pandas.plotting._misc.radviz
FAILED pandas/plotting/_misc.py::pandas.plotting._misc.scatter_matrix

Not many, I'll see if I can create specific issues for some of them.

[datapythonista]

Reopening, I think we still have more doctests that are failing.

We should be able to close this when our call to the doctests can simply be python -m pytest --doctest-modules pandas.