Implemented Nested Docs. #3597

jonathan343 · 2023-02-13T19:02:34Z

Note: This is the boto3 version of the PR #2866 opened up in botocore.

Issue

Our current documentation structure creates one file for each service along with its client methods, exceptions, paginators, and waiters. Having all this content on one page has cased some performance issues on some browsers for large services. This will continue to be an issue as services continue to grow and we need a way to break up service documentation into multiple pages.

Summary

This PR implements the changes that adapt boto3 docs to changes that were made to the botocore doc structure. It also implements boto3 specific nested docs for service resources. In other words, the main page for each service (<service_name>.html) will now have the main client information along with sub-sections for each of the categories (if applicable e.g. a service may not have waiters), each which contain a list of links to the corresponding sub-pages. A new section called "Resources" has been added to the top-level service page and looks similar to the others such as "Client". It simply lists to the nested resource pages.

Note: <service_name>.html files remain in their original location to preserve pre-existing pages. There will also be new folders for each service containing the corresponding sub-pages as shown below.

Old Structure:

├── services
│   ├── ec2.html
│   ├── dynamodb.html
│   ├── s3.html
│   ├── ...

New Structure:

├── services
│   ├── ec2.html
│   ├── dynamodb.html
│   ├── s3.html
│   ├── ...
│   ├── s3
│   │   ├── client
│   │   |   ├── <method_name>.html
│   │   |   ├── ...
│   │   ├── paginator
│   │   |   ├── <paginator_name>.html
│   │   |   ├── ...
│   │   ├── waiter
│   │   |   ├── <waiter_name>.html
│   │   |   ├── ...
│   │   ├── <resource_name> or service-resource
│   │   |   ├── index.html
│   │   |   ├── <identifier_name>.html
│   │   |   ├── <attribute_name>.html
│   │   |   ├── <reference_name>.html
│   │   |   ├── <action_name>.html
│   │   |   ├── <sub_resource_name>.html
│   │   |   ├── <collection_name>.html
│   │   |   ├── <waiter_name>.html
│   │   |   ├── ...
│   ├── ...

Redirects

In order to allow previously existing links to continue to work, a client side fix has been added which parses the URL fragment using window.location.hash.substring(1) and reroutes to the corresponding sup page.
For Example:

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.delete_bucket

will redirect to:

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s2/client/delete_bucket.html

Preview Changes

I've attached a zip file (boto3_docs.zip) which includes a preview for the described changes. The best way to view this is using a local server to host the files. Here are some quick steps to get view the changes:

Download the attached zip
Open a new Terminal
cd <path_to_zip>
unzip -d boto3_docs boto3_docs.zip
cd boto3_docs/html
python -m http.server
Open the provided link in your preferred browser

…om js.

nateprewitt · 2023-02-17T15:37:41Z

boto3/docs/base.py

+        self._resource_sub_path = self._resource_name.lower()
        if self._resource_name == self._service_name:
-            self._resource_sub_path = 'ServiceResource'
+            self._resource_sub_path = 'service-resource'


What prompted the name changes here?

For the reasons in my previous response, this had to be 'serviceresource' or 'service-resource'. I thought the latter looked better and also correlated with how the Service Resource section fragment looks #service-resource. Either one can be supported.

nateprewitt · 2023-02-17T15:39:04Z

boto3/docs/resource.py

+    def _add_feature_freeze_note(self, section):
+        section = section.add_new_section('feature-freeze')
+        section.style.start_note()
+        section.write(
+            ("The AWS Python SDK team does not intend to add new features to the resources "
+            "interface in boto3. Existing interfaces will continue to operate during "
+            "boto3's lifecycle. Customers can find access to newer service features through "
+            f"the :doc:`client interface <../../{self._service_name}>`.")
+        )
+        section.style.end_note()
+


Is this generated for every resource? I've forgotten if we've discussed this previously. We may want to link back to the main resource page with that message rather than adding it everywhere.

Yes, this is generated at the top of every resource page. Not sure what you mean by:

link back to the main resource page with that message rather than adding it everywhere.

Would we alter the message saying something like "Please see for the current guidance on using boto3 Resources"?

Yep, something like that would be good.

nateprewitt · 2023-02-17T15:41:00Z

boto3/docs/service.py

    def _document_service_resource(self, section):
+        # Create a new DocumentStructure for each Service Resource and add contents.
+        service_resource_doc = DocumentStructure('ServiceResource', target='html')


It looks like we've converted ServiceResource to service-resource everywhere else. Is there a reason it's still CamelCase here?

I could be wrong but dont think the value of DocumentStructure name or path affects much. I can update for consistency once we come to a conclusion on how paths should be handled.

Yep, I think that's fine with the explanation above. Let's make it all consistent with what you have and we can call it good.

nateprewitt · 2023-02-17T15:45:40Z

tests/functional/docs/test_s3.py

        self.assert_contains_lines_in_order(
            [
                '.. py:class:: S3.Client',
-                '  s3/Client/download_file',


Some of the naming changes are losing the casing that distinguishes classes from methods. Was that change prompted by a technical requirement or just to enforce path consistency?

The lowercasing of sub-paths was prompted by the fact that resources are now in their own sub-pages, the fragments of section headers are lowercased, and CloudFront path patterns are case-sensitive. If a user was previously linking to the following resource section headers .../s3.html#service-resource or .../s3.html#bucketlifecycleconfiguration, we would (without the lowercasing) want to reroute to .../s3/ServiceResource/index.html which is mappable but we'd have to do the same for .../s3/BucketLifeCycleConfiguration/index.html and all other resources (93 across 10 services). However, we can redirect to .../s3/bucketlifecycleconfiguration/index.html without a mapping.

These are the options I see:

Leave as is and sacrifice casing.

Create a huge mapping of lowercased resource names to correctly cased names.

Don't support service resource section header redirections. Note: Resource class/method links will still be rerouted as normal since their fragments provide the correct casing.

I think that's probably ok, as long as you'll reach the same location with the old link, I don't think casing is the end of the world.

nateprewitt

Left a couple small notes we can try to address in a follow up. Otherwise, I think we're looking good. Thanks @jonathan343!

nateprewitt · 2023-02-28T02:15:52Z

boto3/docs/resource.py

+    def _add_overview_of_member_type(self, section, resource_member_type):
+        section.style.new_line()
+        section.write(
+            f'These are the resource\'s available {resource_member_type}:'


nit; we can fix this in a follow up, but I think this is supposed to be "resources" plural rather than possessive?

nateprewitt · 2023-02-28T02:17:25Z

boto3/docs/resource.py

+        section.style.start_note()
+        section.write(
+            "Before using anything on this page, please refer to the resources "
+            ":doc:`user guide <../../../../guide/resources>` for the most recent "


nit; can we do this as an absolute path relative than relative to the resource page?

jonemo · 2023-02-28T15:45:39Z

Merging this now to make boto3 develop compatible with botocore develop again after boto/botocore#2866.

* release-1.26.81: Bumping version to 1.26.81 Add changelog entries from botocore Added changelog for nested docs Implemented Nested Docs. (#3597)

jonathan343 added 7 commits February 13, 2023 11:00

Implemented Nested Docs.

d6d4594

Forgot to run linter on 1 file

4b65a5d

Forgot to run linter on 1 file: Part 2

4ff9398

Manually included js/custom.js since Sphinx < 1.8 doesnt support cust…

ce76e5d

…om js.

Added a new Resource section; Improved redirecting from old links.

1704cca

Forgot linter :(

2d9319d

Ok I found the issue with my linter. Last one

793b7aa

nateprewitt reviewed Feb 17, 2023

View reviewed changes

jonathan343 added 3 commits February 22, 2023 12:44

Updated Resources notes & redirects; Updated copyright year.

6661128

ran pre-commit

dfa742f

Ran js file throught linter. Added copyright notice

3e6546b

nateprewitt approved these changes Feb 28, 2023

View reviewed changes

jonemo merged commit 102a74b into boto:develop Feb 28, 2023

jonathan343 mentioned this pull request Feb 28, 2023

Added changelog for nested docs #3607

Merged

aws-sdk-python-automation added a commit that referenced this pull request Feb 28, 2023

Merge branch 'release-1.26.81'

3d48469

* release-1.26.81: Bumping version to 1.26.81 Add changelog entries from botocore Added changelog for nested docs Implemented Nested Docs. (#3597)

tim-finnigan mentioned this pull request Mar 1, 2023

Boto3 Docs Website Crashes on large Documentation pages #3160

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implemented Nested Docs. #3597

Implemented Nested Docs. #3597

jonathan343 commented Feb 13, 2023 •

edited

Loading

nateprewitt Feb 17, 2023

jonathan343 Feb 17, 2023

nateprewitt Feb 17, 2023

jonathan343 Feb 20, 2023

nateprewitt Feb 21, 2023

nateprewitt Feb 17, 2023

jonathan343 Feb 17, 2023 •

edited

Loading

nateprewitt Feb 21, 2023

nateprewitt Feb 17, 2023

jonathan343 Feb 17, 2023 •

edited

Loading

nateprewitt Feb 21, 2023

nateprewitt left a comment

nateprewitt Feb 28, 2023

nateprewitt Feb 28, 2023

jonemo commented Feb 28, 2023

Implemented Nested Docs. #3597

Implemented Nested Docs. #3597

Conversation

jonathan343 commented Feb 13, 2023 • edited Loading

Issue

Summary

Redirects

Preview Changes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonathan343 Feb 17, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonathan343 Feb 17, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nateprewitt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonemo commented Feb 28, 2023

jonathan343 commented Feb 13, 2023 •

edited

Loading

jonathan343 Feb 17, 2023 •

edited

Loading

jonathan343 Feb 17, 2023 •

edited

Loading