validate-modules: add support for semantic markup #80243

felixfontein · 2023-03-16T15:34:51Z

SUMMARY

Subset of #74937 which adds semantic markup support to validate-modules.

ISSUE TYPE

Feature Pull Request

COMPONENT NAME

validate-modules

felixfontein · 2023-03-16T15:51:37Z

#80244 is the subset of this PR that only handles the seealso schema change.

…te-modules.

felixfontein · 2023-03-26T12:17:00Z

Note that this is essentially doing the same as ansible-community/antsibull-docs#112.

felixfontein · 2023-03-26T14:53:55Z

I'm wondering whether it isn't better to not actually add this here, but instead have antsibull-docs lint-collection-docs do such linting. Obviously one thing that isn't covered by that are the modules and plugins coming with ansible-core, though.

mattclay · 2023-03-30T21:03:45Z

test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/schema.py

+    for m in _UNESCAPE.finditer(content):
+        if m.group(1) not in ('\\', ')'):
+            raise _add_ansible_error_code(
+                Invalid('Directive "%s" contains unnecessarily quoted "%s"' % (directive, m.group(1))),


I believe it would be more accurate to refer to the use of \ as "escaped" instead of "quoted", as is done in the Python documentation. It should be updated in various places in this PR and in #80240.

Fixed in #80383, also in be68627.

mattclay · 2023-03-30T21:21:18Z

test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/schema.py

+_CONTENT_LINK_SPLITTER_RE = re.compile(r'(?:\[[^\]]*\])?\.')
+_CONTENT_LINK_END_STUB_RE = re.compile(r'\[[^\]]*\]$')


These contain unnecessary escapes:

Suggested change

_CONTENT_LINK_SPLITTER_RE = re.compile(r'(?:\[[^\]]*\])?\.')

_CONTENT_LINK_END_STUB_RE = re.compile(r'\[[^\]]*\]$')

_CONTENT_LINK_SPLITTER_RE = re.compile(r'(?:\[[^]]*])?\.')

_CONTENT_LINK_END_STUB_RE = re.compile(r'\[[^]]*]$')

mattclay · 2023-03-30T21:46:18Z

...e-test-sanity-validate-modules/ansible_collections/ns/col/plugins/modules/semantic_markup.py

+        a7=dict(),
+        a8=dict(),
+        a9=dict(),
+        a10=dict(),


There are no tests for suboptions here.

I added suboption tests in #80383.

mattclay · 2023-03-30T22:22:26Z

test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/schema.py

+_IGNORE_MARKER = 'ignore:'
+_IGNORE_STRING = '(ignore)'
+
+_VALID_PLUGIN_TYPES = set(DOCUMENTABLE_PLUGINS)


Does this intentionally omit support for the role type as described in #80240?

No, this PR hasn't been updated for roles support yet (that's why I marked it as a draft).

#80383 also supports roles.

mattclay · 2023-03-30T22:30:08Z

I'm wondering whether it isn't better to not actually add this here, but instead have antsibull-docs lint-collection-docs do such linting. Obviously one thing that isn't covered by that are the modules and plugins coming with ansible-core, though.

I think it makes sense to have this validation in core, to avoid issues with invalid semantic markup in core plugins.

felixfontein · 2023-03-31T04:33:46Z

I've been thinking about removing the explicit markup validation code from this PR and instead of using the antsibull-docs-parser (https://github.com/ansible-community/antsibull-docs-parser) library. (It has zero dependencies, the idea is that other projects can use it for markup processing.)

mattclay · 2023-03-31T04:53:30Z

I've been thinking about removing the explicit markup validation code from this PR and instead of using the antsibull-docs-parser (https://github.com/ansible-community/antsibull-docs-parser) library. (It has zero dependencies, the idea is that other projects can use it for markup processing.)

So you'd maintain the same functionality as already exists in this PR, but offload some of the implementation to that library?

felixfontein · 2023-03-31T06:09:27Z

Yes; only the markup parsing / validation itself will be offloaded. Right now that part is basically implemented five times (here, ansible-doc in a simplified version, antsibull-docs, antsibull-docs-parser, and antsibull-docs-ts) - my goal is to trim this down to at most three different implementations (ansible-doc probably needs its own to not depend on another library; antsibull-docs-parser for the generic Python implementation; and antsibull-docs-ts for the generic TypeScript implementation).

mattclay · 2023-03-31T16:04:06Z

@felixfontein Since ansible-doc and validate-modules will both need it, why not just vendor the library in the ansible repo so they can both use the same code?

mattclay · 2023-03-31T16:11:53Z

@felixfontein Do you think it would be reasonable to merge this as-is before freeze, and then follow-up with the vendoring as a bug fix before the release? That would also take care of the missing role support.

felixfontein · 2023-03-31T20:32:30Z

@mattclay IMO that would be fine! I won't have time to do anything on this today or tomorrow, but I can create a follow-up 'bugfix' next week.

mattclay · 2023-03-31T20:38:16Z

@felixfontein That works for me. If you're ready for this to be merged, just mark it as ready for review. The minor issues I mentioned in my review comments can be addressed in your follow-up PR.

felixfontein · 2023-03-31T21:12:17Z

@mattclay done

felixfontein · 2023-04-02T20:16:36Z

#80383 contains the follow-up PR which replaces the parsing/rendering code with code from antsibull-docs-parser.

felixfontein mentioned this pull request Mar 16, 2023

Add new semantic markup support to ansible-doc text output and to validate-modules #74937

Closed

ansibot added affects_2.15 feature This issue/PR relates to a feature request. needs_triage Needs a first human triage before being processed. new_plugin This PR includes a new plugin. test This PR relates to tests. labels Mar 16, 2023

felixfontein mentioned this pull request Mar 16, 2023

validate-modules: support plugin see-also #80244

Merged

felixfontein mentioned this pull request Mar 18, 2023

Move markup code out ansible-community/antsibull-docs#108

Merged

bcoca removed the needs_triage Needs a first human triage before being processed. label Mar 21, 2023

felixfontein mentioned this pull request Mar 21, 2023

[Associated vote ended on 2023-03-09] Include semantic markup in plugin DOCUMENTATION strings ansible-community/community-topics#53

Closed

Implement semantic markup support for Ansible documentation in valida…

5e942d9

…te-modules.

felixfontein force-pushed the semantic-markup-ansible-test branch from e3014c6 to 5e942d9 Compare March 23, 2023 18:37

ansibot added the has_issue label Mar 23, 2023

felixfontein mentioned this pull request Mar 26, 2023

Allow to validate Ansible markup ansible-community/antsibull-docs#112

Merged

felixfontein marked this pull request as draft March 27, 2023 20:09

ansibot added the WIP This issue/PR is a work in progress. Nevertheless it was shared for getting input from peers. label Mar 27, 2023

mattclay mentioned this pull request Mar 30, 2023

Semantic markup docs #80240

Merged

mattclay reviewed Mar 30, 2023

View reviewed changes

ansibot added the stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. label Mar 31, 2023

felixfontein marked this pull request as ready for review March 31, 2023 21:12

mattclay merged commit 2f647e9 into ansible:devel Mar 31, 2023

felixfontein deleted the semantic-markup-ansible-test branch March 31, 2023 21:17

This was referenced Apr 2, 2023

Add strict mode for parsing (linting) ansible-community/antsibull-docs-parser#15

Merged

Vendor antsibull-docs-parser code to process and render Ansible markup in ansible-doc #80383

Closed

ansible locked and limited conversation to collaborators Apr 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

validate-modules: add support for semantic markup #80243

validate-modules: add support for semantic markup #80243

felixfontein commented Mar 16, 2023

felixfontein commented Mar 16, 2023

felixfontein commented Mar 26, 2023

felixfontein commented Mar 26, 2023

mattclay Mar 30, 2023

felixfontein Apr 2, 2023

mattclay Mar 30, 2023

mattclay Mar 30, 2023

felixfontein Apr 2, 2023

mattclay Mar 30, 2023

felixfontein Mar 31, 2023

felixfontein Apr 2, 2023

mattclay commented Mar 30, 2023

felixfontein commented Mar 31, 2023

mattclay commented Mar 31, 2023

felixfontein commented Mar 31, 2023

mattclay commented Mar 31, 2023

mattclay commented Mar 31, 2023

felixfontein commented Mar 31, 2023

mattclay commented Mar 31, 2023

felixfontein commented Mar 31, 2023

felixfontein commented Apr 2, 2023

		_CONTENT_LINK_SPLITTER_RE = re.compile(r'(?:\[[^\]]*\])?\.')
		_CONTENT_LINK_END_STUB_RE = re.compile(r'\[[^\]]*\]$')

validate-modules: add support for semantic markup #80243

validate-modules: add support for semantic markup #80243

Conversation

felixfontein commented Mar 16, 2023

SUMMARY

ISSUE TYPE

COMPONENT NAME

felixfontein commented Mar 16, 2023

felixfontein commented Mar 26, 2023

felixfontein commented Mar 26, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattclay commented Mar 30, 2023

felixfontein commented Mar 31, 2023

mattclay commented Mar 31, 2023

felixfontein commented Mar 31, 2023

mattclay commented Mar 31, 2023

mattclay commented Mar 31, 2023

felixfontein commented Mar 31, 2023

mattclay commented Mar 31, 2023

felixfontein commented Mar 31, 2023

felixfontein commented Apr 2, 2023