DOC-950 Add a CLI command for generating Cloud tier limits #133
Conversation
✅ Deploy Preview for docs-extensions-and-macros ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the 📝 WalkthroughWalkthroughThis PR adds a new “cloud-docs” CLI group to bin/doc-tools.js with regions, tiers, and discrepancy subcommands; extends the legacy cloud-regions command with a --tabs option and deprecation notice; introduces tabbed rendering support in tools/cloud-regions (new processCloudRegionsForTabs, updated generateCloudRegions and renderCloudRegions, new adoc-tabs template, and tweaks to existing adoc table template). It adds a full cloud tier table generator (tools/cloud-tier-table) with HTML/MD/ADOC/CSV outputs, an interactive HTML template, a discrepancy report generator, tests and mock YAML data. Documentation pages/partials for regions and cloud limits are added. .gitignore narrows modules ignore to only root-level. Sequence Diagram(s)sequenceDiagram
autonumber
actor User
participant CLI as doc-tools (cloud-docs)
participant GH as GitHub API / FS
participant Regions as generateCloudRegions
participant Render as renderCloudRegions
participant Tpl as Handlebars Templates
participant FS as File System
User->>CLI: cloud-docs regions [--tabs] [--format|--template]
CLI->>Regions: generateCloudRegions({owner,repo,path,ref,format,token,template,tabs})
Regions->>GH: fetch YAML (regions data)
alt tabs mode
Regions->>Regions: processCloudRegionsForTabs()
Regions-->>CLI: {clusterTypes,...}
CLI->>Render: renderCloudRegions({clusterTypes,tabs:true,format,template})
else non-tabs
Regions->>Regions: processCloudRegions()
Regions-->>CLI: {providers,...}
CLI->>Render: renderCloudRegions({providers,tabs:false,format,template})
end
Render->>Tpl: select/compile template (adoc-tabs if tabs)
Render-->>CLI: rendered output(s)
CLI->>FS: write output(s) or stdout (dry-run)
sequenceDiagram
autonumber
actor User
participant CLI as doc-tools (cloud-docs)
participant TierGen as generateCloudTierTable
participant GH as GitHub API / FS
participant Tpl as Handlebars (HTML)
participant FS as File System
User->>CLI: cloud-docs tiers --input <yaml> --master-data <url|file> --format (html|md|adoc|csv)
CLI->>TierGen: generateCloudTierTable({input,masterData,format,template,limits})
par Load inputs
TierGen->>GH: parseYaml(input)
TierGen->>GH: fetchPublicTiers(masterData)
end
TierGen->>TierGen: match public tiers to config profiles (highest -vN)
TierGen->>TierGen: build rows (LIMIT_KEYS)
alt html
TierGen->>Tpl: render HTML with cloud-tier-table-html.hbs
else md/adoc/csv
TierGen-->>CLI: formatted table string
end
CLI->>FS: write or stdout
sequenceDiagram
autonumber
actor User
participant CLI as doc-tools (cloud-docs)
participant Disc as generateDiscrepancyReport
participant TierGen as generateCloudTierTable
participant Parse as Papaparse
participant FS as File System
User->>CLI: cloud-docs discrepancy --input --master-data --format (md|json)
CLI->>Disc: generateDiscrepancyReport(options)
Disc->>TierGen: generateCloudTierTable(..., format: csv)
TierGen-->>Disc: CSV string
Disc->>Parse: parse CSV
Disc->>Disc: analyze per tier (ingress/egress/partitions/clients)
alt json
Disc-->>CLI: JSON report
else md
Disc-->>CLI: Markdown report
end
CLI->>FS: write or stdout
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Possibly related PRs
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
JakeSCahill
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (2)
__tests__/docs-data/mock-master-data.yaml (2)
6-9: Consider consistent typing for advertised limit fields.The advertised throughput fields use string values (
"1000000","2000000"), while partition and client count fields use integers (100,50). This type inconsistency may cause issues during parsing or comparison operations.Consider standardizing all advertised limit values to the same type (either all strings or all integers) for consistency:
cloudProvider: "testcloud" - advertisedMaxIngress: "1000000" - advertisedMaxEgress: "2000000" - advertisedMaxPartitionCount: 100 - advertisedMaxClientCount: 50 + advertisedMaxIngress: 1000000 + advertisedMaxEgress: 2000000 + advertisedMaxPartitionCount: 100 + advertisedMaxClientCount: 50
25-25: Add trailing newline to YAML file.The file is missing a trailing newline, which is a POSIX standard and may cause issues with certain parsers or version control tools.
Add a newline at the end of the file.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (18)
.gitignore(1 hunks)__tests__/docs-data/mock-master-data.yaml(1 hunks)__tests__/docs-data/mock-tier.yml(1 hunks)__tests__/tools/cloud-tier-table.spec.js(1 hunks)bin/doc-tools.js(3 hunks)preview/extensions-and-macros/modules/ROOT/nav.adoc(1 hunks)preview/extensions-and-macros/modules/ROOT/pages/cloud-limits.adoc(1 hunks)preview/extensions-and-macros/modules/ROOT/partials/regions-byoc.adoc(1 hunks)preview/extensions-and-macros/modules/ROOT/partials/regions-dedicated.adoc(1 hunks)tier-discrepancy-report.json(1 hunks)tier-discrepancy-report.md(1 hunks)tools/cloud-regions/cloud-regions-table-adoc-tabs.hbs(1 hunks)tools/cloud-regions/cloud-regions-table-adoc.hbs(1 hunks)tools/cloud-regions/generate-cloud-regions.js(2 hunks)tools/cloud-regions/render-cloud-regions.js(1 hunks)tools/cloud-tier-table/cloud-tier-table-html.hbs(1 hunks)tools/cloud-tier-table/generate-cloud-tier-table.js(1 hunks)tools/cloud-tier-table/generate-discrepancy-report.js(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (6)
tools/cloud-tier-table/generate-discrepancy-report.js (3)
bin/doc-tools.js (14)
require(3-3)require(5-5)require(10-10)require(12-12)require(13-13)require(15-15)require(16-19)require(858-858)require(1358-1358)require(1444-1444)require(1510-1510)require(1560-1560)parsed(724-724)report(1523-1527)__tests__/tools/cloud-tier-table.spec.js (1)
require(3-3)tools/cloud-tier-table/generate-cloud-tier-table.js (3)
parsed(302-302)rows(409-504)rows(595-595)
tools/cloud-tier-table/generate-cloud-tier-table.js (3)
bin/doc-tools.js (29)
fs(8-8)require(3-3)require(5-5)require(10-10)require(12-12)require(13-13)require(15-15)require(16-19)require(858-858)require(1358-1358)require(1444-1444)require(1510-1510)require(1560-1560)f(998-998)out(110-110)out(1374-1383)out(1477-1484)out(1576-1585)templatePath(1366-1366)templatePath(1446-1446)templatePath(1568-1568)r(486-486)r(720-720)r(1289-1289)limits(1456-1456)fmt(1365-1365)fmt(1472-1472)fmt(1517-1517)fmt(1567-1567)__tests__/tools/cloud-tier-table.spec.js (2)
fs(2-2)require(3-3)tools/cloud-tier-table/generate-discrepancy-report.js (2)
require(3-3)i(333-333)
__tests__/tools/cloud-tier-table.spec.js (3)
tools/cloud-tier-table/generate-cloud-tier-table.js (4)
path(2-2)fs(1-1)fs(289-289)fs(559-559)bin/doc-tools.js (14)
path(6-6)require(3-3)require(5-5)require(10-10)require(12-12)require(13-13)require(15-15)require(16-19)require(858-858)require(1358-1358)require(1444-1444)require(1510-1510)require(1560-1560)fs(8-8)tools/cloud-tier-table/generate-discrepancy-report.js (1)
require(3-3)
tools/cloud-regions/render-cloud-regions.js (1)
tools/cloud-regions/generate-cloud-regions.js (4)
renderCloudRegions(35-35)clusterTypes(269-269)providers(150-188)providers(272-272)
bin/doc-tools.js (3)
tools/cloud-tier-table/generate-discrepancy-report.js (4)
options(184-188)options(331-331)require(3-3)report(217-217)tools/cloud-tier-table/generate-cloud-tier-table.js (6)
fmt(596-596)templatePath(599-599)path(2-2)fs(1-1)fs(289-289)fs(559-559)tools/cloud-regions/generate-cloud-regions.js (2)
path(32-32)fs(33-33)
tools/cloud-regions/generate-cloud-regions.js (1)
__tests__/tools/render-cloud-regions.test.js (1)
renderCloudRegions(1-1)
🔇 Additional comments (7)
preview/extensions-and-macros/modules/ROOT/partials/regions-byoc.adoc (1)
1-89: LGTM! Well-structured generated content.The AsciiDoc partial is correctly formatted with proper tabs syntax and consistent table structure across all cloud providers. The auto-generation header provides clear regeneration instructions.
__tests__/docs-data/mock-tier.yml (1)
1-119: LGTM! Comprehensive test tier configurations.The test tier configurations are well-structured with consistent field naming and appropriate value progression across basic, basic-v2, and advanced tiers. The cluster_config sections contain comprehensive limit parameters that align well with the cloud tier table generation requirements.
tools/cloud-regions/cloud-regions-table-adoc-tabs.hbs (1)
1-34: LGTM! Well-structured Handlebars template.The template correctly uses nested iterations to generate tabbed AsciiDoc content with proper syntax for tabs and tables. The structure aligns with the expected output format.
tier-discrepancy-report.json (1)
1-1977: LGTM! Comprehensive discrepancy analysis data.The JSON structure is well-formed and consistent across all 41 tier analyses. The data includes proper severity classification, formatted values, and percentage differences for all four tracked metrics.
__tests__/tools/cloud-tier-table.spec.js (1)
5-70: LGTM! Comprehensive test coverage for core functionality.The test suite effectively validates the key requirements:
- Public tier filtering (excluding internal tiers)
- Display name mapping from master-data
- Inclusion of advertised limits in the output
The tests use appropriate assertions to verify both positive (expected content) and negative (excluded content) cases.
tier-discrepancy-report.md (1)
1-710: LGTM! Well-structured and comprehensive discrepancy report.The Markdown report is properly formatted with:
- Clear executive summary with severity breakdown
- Consistent detailed analysis sections for each cloud provider
- Properly formatted tables with appropriate emoji indicators
- Actionable recommendations for addressing discrepancies
The document effectively communicates the discrepancy analysis findings in a user-friendly format.
preview/extensions-and-macros/modules/ROOT/pages/cloud-limits.adoc (1)
1-6: Approve AsciiDoc inclusionReferenced file exists at preview/extensions-and-macros/modules/ROOT/examples/cloud-limits-by-tier.html. LGTM.
| <thead> | ||
| <tr> | ||
| <th class="sticky-col sticky-header sticky-col-0" onclick="sortCloudTierTable(0)" scope="col">Tier</th> | ||
| <th class="limit-col limit-col-0" onclick="sortCloudTierTable(1)" scope="col">Cloud Provider</th> |
There was a problem hiding this comment.
@JakeSCahill Could these column names be sentence cap; e.g., Cloud provider, Machine type, etc.?
There was a problem hiding this comment.
And could there be a horizonal scroll on the table?
There was a problem hiding this comment.
Add a scroll bar? It already scrolls.
There was a problem hiding this comment.
I see a vertical scroll to see all the tiers, but I mean a horizontal scroll to see all the limits
There was a problem hiding this comment.
@micheleRP could you check again? I made some changes.
|
@coderabbitai generate docstrings |
✅ Actions performedDocstrings generation started. |
Docstrings generation was requested by @JakeSCahill. * #133 (comment) The following files were modified: * `tools/cloud-regions/generate-cloud-regions.js` * `tools/cloud-tier-table/generate-cloud-tier-table.js` * `tools/cloud-tier-table/generate-discrepancy-report.js`
|
Note Generated docstrings for this pull request at #136 |
Docstrings generation was requested by @JakeSCahill. * #133 (comment) The following files were modified: * `tools/cloud-regions/generate-cloud-regions.js` * `tools/cloud-tier-table/generate-cloud-tier-table.js` * `tools/cloud-tier-table/generate-discrepancy-report.js` Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Resolves https://redpandadata.atlassian.net/browse/DOC-950
Cloud throughput tiers have multiple types of limits and quotas that directly affect customer workloads and expectations. Previously, these limits were not well-documented, leading to customer surprises when hitting unexpected constraints (see referenced Slack threads where customers were caught off-guard by undocumented limits).
This follows the industry standard approach used by Confluent Cloud's Kafka Cluster Types documentation, providing transparency around tier capabilities and limitations.
New CLI command:
cloud-docs tiersNew CLI command:
cloud-docs discrepancyInteractive HTML table features:
Data sources
The tool pulls from authoritative configuration files in private repositories:
1. Tier configuration files
Repository:
redpanda-data/cloudv2Path:
contents/install-pack/API URL:
https://api.github.com/repos/redpanda-data/cloudv2/contents/install-packContains version-specific YAML files with detailed tier specifications:
24.3.yml,25.1.yml,25.2.yml, etc.max_throughput_in/max_throughput_out(ingress/egress limits)max_partitions(partition count limits)kafka_connections_max(connection limits)topic_partitions_per_shard(partitioning constraints)2. Master data configuration
Repository:
redpanda-data/cloudv2-infraPath:
apps/master-data-reconciler/manifests/overlays/production/master-data.yamlAPI URL:
https://api.github.com/repos/redpanda-data/cloudv2-infra/contents/apps/master-data-reconciler/manifests/overlays/production/master-data.yaml?ref=integrationUsage examples
Testing
You can see an example of the generated table here: https://deploy-preview-133--docs-extensions-and-macros.netlify.app/preview/cloud-limits/?providers=AWS%2CGCP%2CAzure&tiers=1%2C2%2C3%2C4%2C5%2C6%2C7%2C8%2C9&limits=0%2C1%2C2%2C3%2C4%2C5%2C6%2C7%2C8%2C9%2C10%2C11%2C12%2C13%2C14%2C15%2C16%2C17%2C18%2C19%2C20%2C21%2C22%2C23%2C24%2C25
Set GitHub token with
reposcope:Install dependencies:
Link the package for local testing:
Test table generation:
2025-10-03_15-52-53.mp4