Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce PR Standards #5941

Closed
wants to merge 6 commits into from
Closed

Enforce PR Standards #5941

wants to merge 6 commits into from

Conversation

sharadregoti
Copy link
Contributor

@sharadregoti sharadregoti commented Jan 31, 2025

User description

For internal users - Please add a Jira DX PR ticket to the subject!



Preview Link


Description


Screenshots (if appropriate)


Checklist

  • I have added a preview link to the PR description.
  • I have reviewed the suggestions made by our AI (PR Agent) and updated them accordingly (spelling errors, rephrasing, etc.)
  • I have reviewed the guidelines for contributing to this repository.
  • I have read the technical guidelines for contributing to this repository.
  • Make sure you have started your change off our latest master.
  • I labeled the PR

PR Type

Enhancement, Configuration changes


Description

  • Introduced a GitHub bot to enforce PR standards.

  • Added a workflow to automate PR checklist validation.

  • Enhanced PR template with detailed labeling instructions.

  • Integrated dependencies for GitHub API and markdown parsing.


Changes walkthrough 📝

Relevant files
Enhancement
github.js
Implement GitHub API operations for PR validation               

scripts/pr-review-bot/github.js

  • Added functions to fetch PR body and comment on PRs.
  • Integrated GitHub API for PR operations using Octokit.
  • Implemented logic to close PRs failing checklist validation.
  • +71/-0   
    index.js
    Automate PR checklist validation and handling                       

    scripts/pr-review-bot/index.js

  • Added logic to parse PR template and validate checklist items.
  • Automated commenting and closing of PRs with incomplete checklists.
  • Integrated markdown parsing using marked library.
  • +51/-0   
    Documentation
    pull_request_template.md
    Enhance PR template with labeling guidelines                         

    .github/pull_request_template.md

  • Updated PR template with detailed labeling instructions.
  • Added checklist item for Jira DX PR ticket inclusion.
  • +5/-1     
    Configuration changes
    enforce-pr-standards.yaml
    Add GitHub Actions workflow for PR standards enforcement 

    .github/workflows/enforce-pr-standards.yaml

  • Added a GitHub Actions workflow to enforce PR standards.
  • Configured workflow to trigger on PR events like opening or editing.
  • Integrated Node.js setup and script execution for checklist
    validation.
  • +30/-0   
    package.json
    Configure package.json for PR bot dependencies                     

    scripts/pr-review-bot/package.json

  • Defined project metadata and dependencies.
  • Included @octokit/rest and marked libraries.
  • +17/-0   
    Dependencies
    package-lock.json
    Add dependency lock file for PR bot                                           

    scripts/pr-review-bot/package-lock.json

  • Added dependencies for GitHub API and markdown parsing.
  • Locked package versions for consistent builds.
  • +194/-0 

    Need help?
  • Type /help how to ... in the comments thread for any questions about PR-Agent usage.
  • Check out the documentation for more information.
  • Copy link
    Contributor

    github-actions bot commented Jan 31, 2025

    PR Reviewer Guide 🔍

    (Review updated until commit 8e6dd58)

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
    🧪 No relevant tests
    🔒 Security concerns

    Sensitive information exposure:
    The GITHUB_TOKEN is being used in the workflow. Ensure that it is securely stored and not exposed in logs or error messages.

    ⚡ Recommended focus areas for review

    Possible Issue

    The pull_number variable is being directly assigned from process.env.PR_NUMBER without parsing it as an integer. This could lead to unexpected behavior if the environment variable is not a valid number.

    const pull_number =  process.env.PR_NUMBER; // Convert to integer
    Code Duplication

    The npm install command is duplicated in the workflow file. This redundancy should be removed to streamline the workflow.

    const checklistItems = lexer
        .filter(item => item.type === 'list') // Filter out non-list items and comments
    Workflow Logic

    The workflow enforces PR standards but does not handle cases where the checklist is incomplete but the PR is still valid. Consider adding a mechanism to allow exceptions or approvals for such cases.

    jobs:
      check-pr:
        runs-on: ubuntu-latest
        if: ${{ !github.event.pull_request.draft }}
        steps:
          - name: Checkout code
            uses: actions/checkout@v2
    
          - name: Set up Node.js
            uses: actions/setup-node@v2
            with:
              node-version: '22' # Use your desired Node.js version
    
          - name: Install dependencies
            run: cd scripts/pr-review-bot/ && npm install
    
          - name: Install dependencies
            run: cd scripts/pr-review-bot/ && npm install
    
          - name: Run PR checklist script
            run: node scripts/pr-review-bot/index.js
            env:
              GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Use GitHub's automatically generated token
              PR_NUMBER: ${{ github.event.pull_request.number }}

    Copy link
    Contributor

    github-actions bot commented Jan 31, 2025

    PR Code Suggestions ✨

    Latest suggestions up to 8e6dd58

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Score
    Possible issue
    Correctly set PR to draft status

    Avoid closing the pull request when updating its status to draft, as the state:
    'closed' parameter will permanently close the PR instead of marking it as a draft.

    scripts/pr-review-bot/github.js [58-62]

     await octokit.rest.pulls.update({
         owner,
         repo,
         pull_number,
    -    state: 'closed', // Set draft to true
    +    draft: true, // Correctly set draft status
     });
    Suggestion importance[1-10]: 10

    Why: The current implementation closes the PR instead of marking it as a draft, which is a significant functional issue. This suggestion directly addresses and resolves the problem, ensuring the intended behavior.

    10
    Parse pull_number as an integer

    Ensure that pull_number is parsed as an integer before being used in API calls, as
    it is currently being passed directly from an environment variable which may result
    in unexpected behavior if it is not a valid integer.

    scripts/pr-review-bot/github.js [7]

    -const pull_number =  process.env.PR_NUMBER; // Convert to integer
    +const pull_number = parseInt(process.env.PR_NUMBER, 10); // Convert to integer
    Suggestion importance[1-10]: 9

    Why: Parsing pull_number as an integer ensures that it is correctly interpreted in API calls, preventing potential errors or unexpected behavior. This is a critical fix for the functionality of the script.

    9
    General
    Add error handling for file reading

    Add error handling for the fs.readFileSync call to prevent the application from
    crashing if the .github/pull_request_template.md file is missing or unreadable.

    scripts/pr-review-bot/index.js [9]

    -const fileContent = fs.readFileSync(filePath, 'utf8');
    +let fileContent;
    +try {
    +    fileContent = fs.readFileSync(filePath, 'utf8');
    +} catch (error) {
    +    console.error(`Error reading file ${filePath}:`, error);
    +    process.exit(1); // Exit with error code
    +}
    Suggestion importance[1-10]: 8

    Why: Adding error handling for file reading improves the robustness of the script by preventing crashes if the file is missing or unreadable. This is a valuable enhancement for reliability.

    8
    Remove duplicate dependency installation step

    Remove the duplicate "Install dependencies" step to streamline the workflow and
    avoid unnecessary redundancy.

    .github/workflows/enforce-pr-standards.yaml [20-24]

     - name: Install dependencies
       run: cd scripts/pr-review-bot/ && npm install
     
    -- name: Install dependencies
    -  run: cd scripts/pr-review-bot/ && npm install
    -
    Suggestion importance[1-10]: 7

    Why: Removing the duplicate step streamlines the workflow, reducing redundancy and improving efficiency. While not critical, it is a useful optimization.

    7

    Previous suggestions

    Suggestions up to commit 8e6dd58
    CategorySuggestion                                                                                                                                    Score
    Possible issue
    Correctly mark PR as draft

    Ensure the commentOnPullRequest function does not attempt to close the pull request
    by setting its state to 'closed' when the intention is to mark it as a draft.

    scripts/pr-review-bot/github.js [62]

    -state: 'closed', // Set draft to true
    +draft: true, // Correctly set draft to true
    Suggestion importance[1-10]: 10

    Why: The current code incorrectly sets the PR state to 'closed' instead of marking it as a draft. This suggestion fixes a critical bug that could lead to unintended behavior, making it highly impactful.

    10
    Convert pull_number to an integer

    Ensure that pull_number is converted to an integer before using it in API calls to
    avoid potential type-related issues.

    scripts/pr-review-bot/github.js [7]

    -const pull_number =  process.env.PR_NUMBER; // Convert to integer
    +const pull_number = parseInt(process.env.PR_NUMBER, 10); // Convert to integer
    Suggestion importance[1-10]: 9

    Why: Converting pull_number to an integer is crucial to avoid potential type-related issues when using it in API calls. This suggestion directly addresses a potential bug and improves the robustness of the code.

    9
    Validate GITHUB_TOKEN before usage

    Handle cases where process.env.GITHUB_TOKEN is undefined or invalid to prevent
    runtime errors when creating the Octokit instance.

    scripts/pr-review-bot/github.js [8]

    -const github_token = process.env.GITHUB_TOKEN
    +const github_token = process.env.GITHUB_TOKEN;
    +if (!github_token) {
    +    throw new Error("GITHUB_TOKEN is not defined");
    +}
    Suggestion importance[1-10]: 8

    Why: Adding validation for GITHUB_TOKEN ensures that the application fails gracefully with a clear error message if the token is undefined or invalid. This improves error handling and prevents runtime issues.

    8
    General
    Add error handling for file reading

    Add error handling for the fs.readFileSync call to gracefully handle cases where the
    file does not exist or cannot be read.

    scripts/pr-review-bot/index.js [9]

    -const fileContent = fs.readFileSync(filePath, 'utf8');
    +let fileContent;
    +try {
    +    fileContent = fs.readFileSync(filePath, 'utf8');
    +} catch (error) {
    +    console.error(`Error reading file at ${filePath}:`, error);
    +    process.exit(1);
    +}
    Suggestion importance[1-10]: 7

    Why: Adding error handling for fs.readFileSync ensures that the application can handle cases where the file does not exist or cannot be read, improving the robustness and reliability of the script.

    7
    Suggestions up to commit 542373e
    CategorySuggestion                                                                                                                                    Score
    Possible issue
    Fix incorrect draft status update

    Avoid closing the pull request when updating its status to draft, as the state:
    'closed' parameter is incorrect for marking it as a draft.

    scripts/pr-review-bot/github.js [58-62]

     await octokit.rest.pulls.update({
         owner,
         repo,
         pull_number,
    -    state: 'closed', // Set draft to true
    +    draft: true, // Correctly mark as draft
     });
    Suggestion importance[1-10]: 10

    Why: The suggestion corrects a critical mistake where the pull request is being closed instead of being marked as a draft. This fix directly addresses a functional error in the script.

    10
    Ensure pull_number is an integer

    Validate that pull_number is correctly parsed as an integer before using it in API
    calls to prevent potential runtime errors.

    scripts/pr-review-bot/github.js [7]

    -const pull_number =  process.env.PR_NUMBER; // Convert to integer
    +const pull_number = parseInt(process.env.PR_NUMBER, 10); // Ensure it is an integer
    Suggestion importance[1-10]: 9

    Why: Parsing pull_number as an integer ensures that it is correctly formatted for API calls, preventing potential runtime errors. This is a critical fix for the functionality of the script.

    9
    Handle missing GitHub token error

    Add error handling for cases where process.env.GITHUB_TOKEN is undefined to avoid
    authentication failures.

    scripts/pr-review-bot/github.js [8]

    -const github_token = process.env.GITHUB_TOKEN
    +const github_token = process.env.GITHUB_TOKEN;
    +if (!github_token) {
    +    throw new Error("GITHUB_TOKEN is not defined in the environment variables.");
    +}
    Suggestion importance[1-10]: 8

    Why: Adding error handling for a missing GITHUB_TOKEN ensures the script fails gracefully and provides clear feedback, which is essential for debugging and avoiding authentication failures.

    8
    General
    Handle file read errors gracefully

    Ensure fs.readFileSync is wrapped in a try-catch block to handle potential file read
    errors gracefully.

    scripts/pr-review-bot/index.js [9]

    -const fileContent = fs.readFileSync(filePath, 'utf8');
    +let fileContent;
    +try {
    +    fileContent = fs.readFileSync(filePath, 'utf8');
    +} catch (error) {
    +    console.error(`Error reading file at ${filePath}:`, error);
    +    throw error;
    +}
    Suggestion importance[1-10]: 7

    Why: Wrapping fs.readFileSync in a try-catch block improves the script's robustness by handling potential file read errors, which is a good practice for error management.

    7
    Suggestions up to commit a5e9a0a
    CategorySuggestion                                                                                                                                    Score
    Possible issue
    Fix incorrect PR state update logic

    Avoid closing the pull request when updating its state to draft, as the state:
    'closed' parameter will permanently close the PR instead of marking it as draft.

    scripts/pr-review-bot/github.js [58-62]

     await octokit.rest.pulls.update({
         owner,
         repo,
         pull_number,
    -    state: 'closed', // Set draft to true
    +    draft: true, // Correctly set draft to true
     });
    Suggestion importance[1-10]: 10

    Why: The suggestion resolves a major issue where the PR is being permanently closed instead of being marked as a draft. Correcting this logic is essential for the intended functionality of the script.

    10
    Convert pull_number to an integer

    Ensure that pull_number is explicitly converted to an integer before being used in
    API calls, as it is currently being passed as a string from the environment
    variable, which may lead to unexpected behavior.

    scripts/pr-review-bot/github.js [7]

    -const pull_number =  process.env.PR_NUMBER; // Convert to integer
    +const pull_number = parseInt(process.env.PR_NUMBER, 10); // Convert to integer
    Suggestion importance[1-10]: 9

    Why: The suggestion addresses a critical issue where pull_number is used as a string, which could lead to unexpected behavior in API calls. Explicitly converting it to an integer ensures correctness and prevents potential bugs.

    9
    General
    Add error handling for file reading

    Add error handling for the fs.readFileSync call to gracefully handle cases where the
    file path is incorrect or the file does not exist, preventing the script from
    crashing.

    scripts/pr-review-bot/index.js [9]

    -const fileContent = fs.readFileSync(filePath, 'utf8');
    +let fileContent;
    +try {
    +    fileContent = fs.readFileSync(filePath, 'utf8');
    +} catch (error) {
    +    console.error(`Error reading file at ${filePath}:`, error);
    +    process.exit(1);
    +}
    Suggestion importance[1-10]: 8

    Why: Adding error handling for the fs.readFileSync call improves the robustness of the script by preventing crashes when the file path is incorrect or the file does not exist.

    8
    Remove duplicate dependency installation step

    Remove the duplicate "Install dependencies" step in the workflow to avoid
    unnecessary redundancy and improve efficiency.

    .github/workflows/enforce-pr-standards.yaml [21-24]

     - name: Install dependencies
       run: cd scripts/pr-review-bot/ && npm install
     
    -- name: Install dependencies
    -  run: cd scripts/pr-review-bot/ && npm install
    -
    Suggestion importance[1-10]: 7

    Why: Removing the duplicate "Install dependencies" step eliminates redundancy, improving the efficiency of the workflow without affecting functionality.

    7
    Suggestions up to commit a5e9a0a
    CategorySuggestion                                                                                                                                    Score
    Possible issue
    Correct PR state update logic

    Avoid setting the pull request state to 'closed' when intending to mark it as a
    draft, as this can cause unintended closure of the PR.

    scripts/pr-review-bot/github.js [62]

    -state: 'closed', // Set draft to true
    +draft: true, // Correctly mark as draft
    Suggestion importance[1-10]: 10

    Why: The current code incorrectly sets the PR state to 'closed' instead of marking it as a draft. This suggestion fixes a critical logic error that could unintentionally close PRs, disrupting workflows.

    10
    Convert pull_number to integer

    Ensure that pull_number is converted to an integer before being used in API calls to
    avoid potential type-related issues.

    scripts/pr-review-bot/github.js [7]

    -const pull_number =  process.env.PR_NUMBER; // Convert to integer
    +const pull_number = parseInt(process.env.PR_NUMBER, 10); // Convert to integer
    Suggestion importance[1-10]: 9

    Why: Converting pull_number to an integer ensures type consistency and prevents potential issues when using it in API calls. This is a critical fix as incorrect types could lead to runtime errors.

    9
    Validate GITHUB_TOKEN environment variable

    Handle cases where process.env.GITHUB_TOKEN is undefined or empty to prevent
    authentication errors.

    scripts/pr-review-bot/github.js [8]

    -const github_token = process.env.GITHUB_TOKEN
    +const github_token = process.env.GITHUB_TOKEN;
    +if (!github_token) {
    +    throw new Error("GITHUB_TOKEN is not defined");
    +}
    Suggestion importance[1-10]: 8

    Why: Adding validation for GITHUB_TOKEN ensures that the script fails early with a clear error message if the token is missing, preventing authentication errors during runtime.

    8
    General
    Add error handling for file reading

    Add error handling for the fs.readFileSync call to gracefully handle cases where the
    file does not exist or cannot be read.

    scripts/pr-review-bot/index.js [9]

    -const fileContent = fs.readFileSync(filePath, 'utf8');
    +let fileContent;
    +try {
    +    fileContent = fs.readFileSync(filePath, 'utf8');
    +} catch (error) {
    +    console.error(`Error reading file at ${filePath}:`, error);
    +    process.exit(1);
    +}
    Suggestion importance[1-10]: 7

    Why: Adding error handling for fs.readFileSync improves the robustness of the script by gracefully handling cases where the file does not exist or cannot be read, preventing unexpected crashes.

    7

    Copy link

    netlify bot commented Jan 31, 2025

    PS. Pls add /docs/nightly to the end of url

    Name Link
    🔨 Latest commit 79bfc83
    🔍 Latest deploy log https://app.netlify.com/sites/tyk-docs/deploys/679ccc6841f1290008191661
    😎 Deploy Preview https://deploy-preview-5941--tyk-docs.netlify.app
    📱 Preview on mobile
    Toggle QR Code...

    QR Code

    Use your smartphone camera to open QR code link.

    To edit notification comments on pull requests, go to your Netlify site configuration.

    Copy link
    Contributor

    Persistent review updated to latest commit 79bfc83

    Copy link
    Contributor

    Persistent review updated to latest commit 245feb3

    @sharadregoti
    Copy link
    Contributor Author

    PR Checklist Failed

    @sharadregoti This PR does not pass all the checklist mentioned in description, because of which we are closing the PR. Re-open it when all checklist items are passed

    Failed Items

    1: Make sure you have started your change off our latest master.
    2: I labeled the PR

    @sharadregoti
    Copy link
    Contributor Author

    PR Checklist Failed

    @sharadregoti This PR does not pass some of the required checklist items mentioned in description, because of which we are closing the PR. Re-open it when all checklist items are passed

    Failed Items

    1: Make sure you have started your change off our latest master.
    2: I labeled the PR

    @sharadregoti
    Copy link
    Contributor Author

    PR Checklist Failed

    @sharadregoti This PR does not pass some of the required checklist items mentioned in description, because of which we are closing the PR. Re-open it when all checklist items are passed

    Failed Items

    1: For internal users - Please add a Jira DX PR ticket to the subject!
    2: I have added the appropriate release label to this PR:

    • If it is for a future release, label it as

    @sharadregoti
    Copy link
    Contributor Author

    PR Checklist Failed

    @sharadregoti This PR does not pass some of the required checklist items mentioned in description, because of which we are closing the PR. Re-open it when all checklist items are passed

    Failed Items

    1: For internal users - Please add a Jira DX PR ticket to the subject!
    2: I have added the appropriate release label to this PR:

    • If it is for a future release, label it as future-release and specify the version (e.g., future-release, 6.0).
    • If it should be merged into an older version, use the specific version label (e.g., 4.1, 5.1).
    • If no label is added, it will be assumed that the PR should be merged into the latest current version (e.g., 5.5) and master.

    @sharadregoti sharadregoti reopened this Jan 31, 2025
    Copy link
    Contributor

    Persistent review updated to latest commit 245feb3

    Copy link
    Contributor

    Persistent review updated to latest commit a5e9a0a

    Copy link
    Contributor

    PR Checklist Failed

    @sharadregoti This PR does not pass some of the required checklist items mentioned in description, because of which we are closing the PR. Re-open it when all checklist items are passed

    Failed Items

    1: For internal users - Please add a Jira DX PR ticket to the subject!
    2: I have added the appropriate release label to this PR:

    • If it is for a future release, label it as future-release and specify the version (e.g., future-release, 6.0).
    • If it should be merged into an older version, use the specific version label (e.g., 4.1, 5.1).
    • If no label is added, it will be assumed that the PR should be merged into the latest current version (e.g., 5.5) and master.

    @github-actions github-actions bot closed this Jan 31, 2025
    Copy link
    Contributor

    Persistent review updated to latest commit 542373e

    @sharadregoti sharadregoti reopened this Jan 31, 2025
    Copy link
    Contributor

    PR Checklist Failed

    @sharadregoti This PR does not meet all the required checklist items mentioned in the description. As a result, we are closing the PR. Please re-open it once all checklist items are completed (ensure they are checked in the description).

    Failed Items

    1: For internal users - Please add a Jira DX PR ticket to the subject!
    2: I have added the appropriate release label to this PR:

    • If it is for a future release, label it as future-release and specify the version (e.g., future-release, 6.0).
    • If it should be merged into an older version, use the specific version label (e.g., 4.1, 5.1).
    • If no label is added, it will be assumed that the PR should be merged into the latest current version (e.g., 5.5) and master.

    @github-actions github-actions bot closed this Jan 31, 2025
    Copy link
    Contributor

    Persistent review updated to latest commit 8e6dd58

    @sharadregoti sharadregoti reopened this Feb 3, 2025
    Copy link
    Contributor

    github-actions bot commented Feb 3, 2025

    PR Checklist Failed

    @sharadregoti This PR does not meet all the required checklist items mentioned in the description. As a result, we are closing the PR. Please re-open it once all checklist items are completed (ensure they are checked in the description).

    Failed Items

    1: For internal users - Please add a Jira DX PR ticket to the subject!
    2: I have added the appropriate release label to this PR:

    • If it is for a future release, label it as future-release and specify the version (e.g., future-release, 6.0).
    • If it should be merged into an older version, use the specific version label (e.g., 4.1, 5.1).
    • If no label is added, it will be assumed that the PR should be merged into the latest current version (e.g., 5.5) and master.

    @github-actions github-actions bot closed this Feb 3, 2025
    Copy link
    Contributor

    github-actions bot commented Feb 3, 2025

    Persistent review updated to latest commit 8e6dd58

    Copy link
    Contributor

    github-actions bot commented Feb 3, 2025

    PR Checklist Failed

    @sharadregoti This PR does not meet all the required checklist items mentioned in the description. As a result, we are closing the PR. Please re-open it once all checklist items are completed (ensure they are checked in the description).

    Failed Items

    1: For internal users - Please add a Jira DX PR ticket to the subject!
    2: I have added the appropriate release label to this PR:

    • If it is for a future release, label it as future-release and specify the version (e.g., future-release, 6.0).
    • If it should be merged into an older version, use the specific version label (e.g., 4.1, 5.1).
    • If no label is added, it will be assumed that the PR should be merged into the latest current version (e.g., 5.5) and master.

    1 similar comment
    Copy link
    Contributor

    github-actions bot commented Feb 3, 2025

    PR Checklist Failed

    @sharadregoti This PR does not meet all the required checklist items mentioned in the description. As a result, we are closing the PR. Please re-open it once all checklist items are completed (ensure they are checked in the description).

    Failed Items

    1: For internal users - Please add a Jira DX PR ticket to the subject!
    2: I have added the appropriate release label to this PR:

    • If it is for a future release, label it as future-release and specify the version (e.g., future-release, 6.0).
    • If it should be merged into an older version, use the specific version label (e.g., 4.1, 5.1).
    • If no label is added, it will be assumed that the PR should be merged into the latest current version (e.g., 5.5) and master.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    1 participant