About

This is a GitHub bot written using Flask for the Astropy project which can be installed via GitHub integrations. We use the concept of flask "blueprints" to separate the different functionality. Each blueprint is defined in a separate file inside changebot/blueprints. The bot includes the following functionality:

• Check pull requests whenever they are changed. For now the checks only include making sure the changelog is consistent with the milestone and the labels but this could be expanded in future. This is defined in pull_request_checker.py.

• Check for issues labeled as "Close?" that have become stale, and close them (with a warning period). This is defined in stale_issues.py.

• Check for pull requests that have become stale, and close them (with a warning period). This is defined in stale_pull_requests.py.

More details about the bot and its components is provided in the sections below.

Overall bot set-up

Optional: Heroku settings (creating a separate instance of this bot)

For astropy-bot, the astrochangebot.herokuapp.com server is a GitHub app that is set up on Heroku. We use the auto-deploy functionality in Heroku to re-deploy the app anytime a change is made to this repository.

To create your own instance of the bot, first, if you don't already have a Heroku account, create a free one there. When you see the option to create a new app, select it (ignore the "add to pipeline" option). Give a name to your app; You need to select a name that is not already taken and it does not have to be the same as the bot's name here.

You should now be on the "Deploy" section. Again, ignore the pipeline option. Select Github as "Deployment Method". Enter the relevant GitHub organization or account that the bot resides in (this should be automatically populated if you have given Heroku access to your GitHub account) and type in the bot's repository name (either this bot or a forked version of it).

If you want to enable automatic deployment from a selected branch of the repository, click the "Enable Automatic Deploys" button. This will pick up changes to the given branch and re-deploy the bot as needed. For most cases, you don't need the "wait for CI to pass before deploy" option as the bot is already tested here.

For the first time, you also need to manually deploy the bot by clicking "Deploy Branch".

Once it is successfully deployed, go to "Settings" tab of the bot and you can customize its behavior using "Config Vars". This is the only custom configuration on Heroku and can be set through the Heroku admin interface, as mentioned. The main required environment variables (also see "Authentication" section below) are::

• GITHUB_APP_INTEGRATION_ID, which should be set to the integration ID provided by GitHub app (see "GitHub settings" section below) under "General Settings", specifically "About... ID". For astropy-bot, the integration ID is currently 3400 (this is not a secret).

• GITHUB_APP_PRIVATE_KEY, which is generated by the GitHub app (see "GitHub settings" section below). This private key should look like:

  -----BEGIN RSA PRIVATE KEY-----
  <some random characters>
  -----END RSA PRIVATE KEY-----
  

  The whole key, including the BEGIN and END header and footer should be pasted into the field. To obtain this key for astropy-bot or one that you did not set up, please contact bot administrator.

• CRON_TOKEN, which should be set to a random string known only to the person or people who need to run cron jobs manually. Some of the functionality in astropy-bot is not intended to be triggered by GitHub but instead by cron jobs. However, we don't want just anyone to be able to run these cron jobs, so this environment variable is the shared secret/password that allows certain actions to be triggered. If you don't need the cron jobs, this needs to be set anyway due to the way webapp.py is currently coded, so just set to some string like "N/A".

In addition to these environment variables, there are several others required for specific components of the bot, as described in "Stale issue and pull request checkers" section below. These must also be set whether you wish to use them or not.

Optional: GitHub settings (adding a separate instance of this bot)

NOTE: If you are creating a new instance of this bot from your fork using a different bot name, it is important that you replace astropy-bot with your new bot's name in the production branch of your fork. Otherwise, the bot will not able to edit its comments. However, you don't have to modify the https://github.com/astropy/astropy-bot/issues URLs because your fork will not have its own issues tracker.

To add the bot to your own organization or account, go to your GitHub organization or account URL (not the repository) and then its settings. Then, click on "Developer settings" at the very bottom of the left navigation bar and the "New GitHub App" button on top right.

Give your bot a "GitHub App name" as you want it to appear on GitHub activities. Under "Homepage URL", enter the GitHub repository URL where the bot code resides (either here or your fork, as appropriate).

For the User authorization callback URL, it should be in the format of http://<heroku-bot-name>.herokuapp.com/callback. For astropy-bot, this is http://astrochangebot.herokuapp.com/callback.

For the Webhook URL, it should be in the format of http://<heroku-bot-name>.herokuapp.com/hook. For astropy-bot, this is http://astrochangebot.herokuapp.com/hook

You can ignore "Setup URL" and "Webhook secret". It would be useful to provide a description of what your bot intends to do but not required.

The permissions of the app should be read/write access to Commit statuses, Issues, and Pull requests. Once you have checked these options, you will see extra "Subscribe to events" entries that you can check as well. For the events, it should be sufficient to only check Status, Issue comment, Issues, Pull request, Pull request review, and Pull request review comment.

It is up to you to choose whether you want to allow your GitHub app here to be installed only on your account or by any user or organization.

Once you have clicked "Create GitHub App" button, you can go back to the app's "General" settings and upload a logo, which is basically a profile picture of your bot.

For astropy-bot, don't be tempted to re-generate a key from the GitHub app settings here. There is a key that is already generated that we should use; Ask @astrofrog if you would like to know the key. However, for your own bot, you can generate a private key for the app to avoid GitHub anonymous API limits; Save this key somewhere private and secure.

For Astropy, the GitHub app for this bot is defined in the Astropy organization settings (not everyone will be able to see this).

Install the bot

Go to https://github.com/apps/<github-app-name>; For this bot, it is https://github.com/apps/astropy-bot . Then, click on the big green "Install" button. You can choose to install the bot on all or select repositories under your account or organization. It is recommended to only install it for select repositories by start typing a repository name and let auto-completion do the hard work for you (repeat this once per repository). Once you are done, click "Install".

After a successfull installation, you will be taken to a https://github.com/settings/installations/<installation-number> page. This page is also accessible from your account or organization settings in "Applications", specifically under "Installed GitHub Apps". You can change the installation settings by clicking the "Configure" button next to the listed app, if desired.

For Astropy, the app can be added to specific repositories under the Your installations tab, by clicking on the gearbox next to Astropy, which goes to this page (not everyone will be able to see this).

Pull request checker

The pull request checker is intended to be triggered by GitHub whenever there is a change in a pull request. Currently the main code that gets run when this happens is the check_changelog_consistency function, which makes sure that the changelog is consistent with the pull request number, labels, and milestone. The main logic is defined in pull_request_checker.py and changelog_helpers.py

If CHANGES.rst, CHANGES, or CHANGES.md file does not exist in the repository, bot will not comment at all.

For repositories that do not use a changelog, this checker can be disabled by defining changelog_check: false in the [tool.stsci-bot] section of pyproject.toml at the top level of the repository.

Stale issue and pull request checkers

The components to check for stale issues and pull requests are intended to be triggered by a cron job rather than from GitHub. The main reasons for this are because these tend to do more API calls than the single pull request checker (because e.g. all open pull requests need to be queried) and because by definition we are looking for inactivity rather than activity.

These components can be configured via the following environment variables on Heroku:

• STALE_ISSUE_CLOSE, which should be set to TRUE or FALSE. If FALSE, this doesn't check for whether issues labeled as Close? should be closed, only whether the warning about closing should be posted. The intent is that this should always be TRUE except the first time that this is run or if you never want your stale issues be closed. This requires that you have a Close? label (case sensitive; include the question mark) on each of your repositories that your bot watches.

• STALE_ISSUE_WARN_SECONDS, which is the time in seconds from when an issue was labeled as Close? in order to be warned that it will be closed. If you never want this warning, set it to a very high number or do not apply the Close? label. The recommended value is 12960000, which is approximately 5 months.

• STALE_ISSUE_CLOSE_SECONDS, which is the time in seconds from when an issue was labeled as Close? in order to be considered ready for closing. This should be larger than STALE_ISSUE_WARN_SECONDS. This setting is only relevant if STALE_ISSUE_CLOSE is set to TRUE. The recommended value is 15552000, which is approximately 6 months.

• STALE_PULL_REQUEST_CLOSE, which should be set to TRUE or FALSE. If FALSE, this doesn't check for whether pull requests should be closed, only whether the warning about closing should be posted. The intent is that this should always be TRUE except the first time that this is run or if you never want your stale pull requests be closed.

  For repositories that do not want to ever close stale pull requests, this feature can be permanently disabled by defining autoclose_stale_pull_request: false in the [tool.stsci-bot] section of pyproject.toml at the top level of the repository.

• STALE_PULL_REQUEST_WARN_SECONDS, which is the time in seconds from the last commit in order to be warned that it will be closed. If you never want this warning, set it to a very high number. The recommended value is 12960000, which is approximately 5 months.

• STALE_PULL_REQUEST_CLOSE_SECONDS, which is the time in seconds from the last commit in order to be considered ready for closing unless a keep-open is attached to the pull request. This should be larger than STALE_PULL_REQUEST_WARN_SECONDS. This setting is only relevant if STALE_PULL_REQUEST_CLOSE is set to TRUE. The recommended value is 15552000, which is approximately 6 months.

For issues, removing the Close? label and adding it back resets the clock. For pull requests, adding a new commit resets the clock, while adding the keep-open label means that this pull request will not be touched by the bot.

To run these checks, you can access http://astrochangebot.herokuapp.com/close_stale_issues and http://astrochangebot.herokuapp.com/close_stale_pull_requests using a POST request and with the following parameters encoded in JSON:

• 'repository': the name of the repository to run the checks for, including the owner (e.g. astropy/astropy)
• 'cron_token': this should be the same as the CRON_TOKEN environment variable
• 'installation': this should be the installation ID (see the Authentication section). For the Astropy organization repositories, this is 36238.

An example for how to do this with the requests package is:

import requests

data = {'repository': 'astrofrog/test-bot',
        'cron_token': 'theactualtoken',
        'installation': '36238'}

requests.post(URL, json=data)

GitHub API

The different components of the bot interact with GitHub via a set of helper classes that live in changebot/github/github_api.py. These classes are RepoHandler, IssueHandler, and PullRequestHandler. It is possible to try these out locally, at least for the parts of the GitHub API that do not require authentication. For example, the following should work:

>>> from changebot.github.github_api import RepoHandler, IssueHandler, PullRequestHandler
>>> repo = RepoHandler('astropy/astropy')
>>> repo.get_issues('open', 'Close?')
[6025, 5193, 4842, 4549, 4058, 3951, 3845, 2603, 2232, 1920, 1024, 435, 383, 282]
>>> issue = IssueHandler('astropy/astropy', 6597)
>>> issue.labels
['Bug', 'coordinates']
>>> pr = PullRequestHandler('astropy/astropy', 6606)
>>> pr.labels
['Enhancement', 'Refactoring', 'testing', 'Work in progress']
>>> pr.last_commit_date
1506374526.0

However since these are being run un-authenticated, you may quickly run into the GitHub public API limits. If you are interested in authenticating locally, see the Authentication section below.

Code-wise, the authentication of the app is handled in the changebot/github/github_auth.py file - the main function from there that is used in changebot/github/github_api.py is the github_request_headers function which returns headers for requests that contain the appropriate tokens.

Authentication

In some cases, you may want to test the bot locally as if it was running on Heroku. In order to do this you will need to make sure you have all the environment variables described above set correctly.

The main ones to get right as far as authentication is concerned are as follows (see "Heroku settings" section above for further descriptions):

• GITHUB_APP_INTEGRATION_ID
• GITHUB_APP_PRIVATE_KEY

The last thing you will need is an Installation ID - a GitHub app can be linked to different GitHub accounts, and for each account or organization, it has a unique ID. You can find out this ID by going to Your installations and then clicking on the settings box next to the account where you have a test repository you want to interact with. The URL of the page you go to will look like:

https://github.com/settings/installations/36238

In this case, 36238 is the installation ID. Provided you set the environment variables correctly, you should then be able to do e.g.:

>>> from changebot.github.github_api import IssueHandler
>>> issue = IssueHandler('astrofrog/test-bot', 5, installation=36238)
>>> issue.submit_comment('I am alive!')

Use this power wisely! (And avoid testing out things on the main Astropy repos...)

Note: Authentication will not work properly if you have a .netrc file in your home directory, so you will need to rename this file temporarily.

Requirements

This app requires Python 3.6 to run, and dependencies are listed in requirements.txt