We welcome your contributions and participation! If you aren't sure what to expect, here are some norms for our project so you feel more comfortable with how things will go.
If this is your first contribution to Porter, we have a tutorial that walks you through how to setup your developer environment, make a change and test it.
The Porter community is governed by our Code of Conduct. This includes but isn't limited to: the porter and related mixin repositories, slack, interactions on social media, project meetings, conferences and meetups.
Use the porter.sh/find-issue link to find good first issues for new contributors and help wanted issues for our other contributors.
When you have been contributing for a while, take a look at the "Backlog" column on our project board for high priority issues. The project board is at the organization level, so it contains issues from across all the Porter repositories.
good first issueshas extra information to help you make your first contribution.help wantedare issues suitable for someone who isn't a core maintainer.hmm 🛑🤔issues should be avoided. They are not ready to be worked on yet because they are not finished being designed or we aren't sure if we want the feature, etc.
Maintainers will do their best to regularly make new issues for you to solve and then help out as you work on them. 💖
We have a roadmap that will give you a good idea of the larger features that we are working on right now. That may help you decide what you would like to work on after you have tackled an issue or two to learn how to contribute to Porter. If you have a big idea for Porter, learn how to propose a change to Porter.
Another great way to contribute is to create a mixin! You can start using the Porter Skeletor repository as a template to start, along with the Mixin Developer Guide.
When you create your first pull request, add your name to the bottom of our Contributors list. Thank you for making Porter better! 🙇♀️
Unless the issue specifically mentions a branch, please create your feature branch from the main branch.
For example:
# Make sure you have the most recent changes to main
git checkout main
git pull
# Create a branch based on main named MY_FEATURE_BRANCH
git checkout -b MY_FEATURE_BRANCH mainIt's OK to submit a PR directly for problems such as misspellings or other things where the motivation/problem is unambiguous.
If there isn't an issue for your PR, please make an issue first and explain the problem or motivation for the change you are proposing. When the solution isn't straightforward, for example, "Implement missing command X", then also outline your proposed solution. Your PR will go smoother if the solution is agreed upon before you've spent a lot of time implementing it.
Since Porter is a CLI, the "solution" will usually look like this:
$ porter newcommand [OPTIONAL] [--someflag VALUE]
example outputWe recommend running the following every time:
mage Build TestUnit
If your test modified anything related to running a bundle, also run:
mage TestIntegration
If you want to know all the targets that the CI runs, look at <build/azure-pipelines.pr-automatic.yml>.
🚧 If you aren't done yet, create a draft pull request or put WIP in the title so that reviewers wait for you to finish before commenting.
1️⃣ Limit your pull request to a single task. Don't tackle multiple unrelated things, especially refactoring. If you need large refactoring for your change, chat with a maintainer first, then do it in a separate PR first without any functionality changes.
🎳 Group related changes into separate commits to make it easier to review.
😅 Make requested changes in new commits. Please don't amend or rebase commits that we have already reviewed. When your pull request is ready to merge, you can rebase your commits yourself, or we can squash when we merge. Just let us know what you are more comfortable with.
🚀 We encourage follow-on PRs and a reviewer may let you know in their comment if it is okay for their suggestion to be done in a follow-on PR. You can decide to make the change in the current PR immediately, or agree to tackle it in a reasonable amount of time in a subsequent pull request. If you can't get to it soon, please create an issue and link to it from the pull request comment so that we don't collectively forget.
You can automatically sign your commits to meet the DCO requirement for this
project by running the following command: mage SetupDCO.
Licensing is important to open source projects. It provides some assurances that the software will continue to be available based under the terms that the author(s) desired. We require that contributors sign off on commits submitted to our project's repositories. The Developer Certificate of Origin (DCO) is a way to certify that you wrote and have the right to contribute the code you are submitting to the project.
You sign-off by adding the following to your commit messages:
Author: Your Name <your.name@example.com>
Date: Thu Feb 2 11:41:15 2018 -0800
This is my commit message
Signed-off-by: Your Name <your.name@example.com>Notice the Author and Signed-off-by lines match. If they don't, the PR will
be rejected by the automated DCO check.
Git has a -s command line option to do this automatically:
git commit -s -m 'This is my commit message'
If you forgot to do this and have not yet pushed your changes to the remote repository, you can amend your commit with the sign-off by running
git commit --amend -s
-
You create a draft or WIP pull request. Reviewers will ignore it mostly unless you mention someone and ask for help. Feel free to open one and use the pull request to see if the CI passes. Once you are ready for a review, remove the WIP or click "Ready for Review" and leave a comment that it's ready for review.
If you create a regular pull request, a reviewer won't wait to review it.
-
A reviewer will assign themselves to the pull request. If you don't see anyone assigned after 3 business days, you can leave a comment asking for a review, or ping in slack. Sometimes we have busy days, sick days, weekends and vacations, so a little patience is appreciated! 🙇♀️
-
The reviewer will leave feedback.
nits: These are suggestions that you may decide to incorporate into your pull request or not without further comment.- It can help to put a 👍 on comments that you have implemented so that you can keep track.
- It is okay to clarify if you are being told to make a change or if it is a suggestion.
-
After you have made the changes (in new commits please!), leave a comment. If 3 business days go by with no review, it is okay to bump.
-
When a pull request has been approved, the reviewer will squash and merge your commits. If you prefer to rebase your own commits, at any time leave a comment on the pull request to let them know that.
At this point your changes are available in the canary release of Porter! After your first pull request is merged, you will be invited to the Contributors team which you may choose to accept (or not). Joining the team lets you have issues in GitHub assigned to you.
A follow-on PR is a pull request that finishes up suggestions from another pull request.
When the core of your changes are good, and it won't hurt to do more of the changes later, our preference is to merge early, and keep working on it in a subsequent. This allows us to start testing out the changes in our canary builds, and more importantly enables other developers to immediately start building their work on top of yours.
This helps us avoid pull requests to rely on other pull requests. It also avoids pull requests that last for months, and in general we try to not let "perfect be the enemy of the good". It's no fun to watch your work sit in purgatory, and it kills contributor momentum.
Our contribution ladder defines the roles and responsibilities for this project and how to participate with the goal of moving from a user to a maintainer.
We have a tutorial that walks you through how to set up your developer environment, make a change and test it.
Here are the key steps, if you run into trouble, the tutorial has more details:
- Install Go version 1.17 or higher.
- Clone this repository with
git clone https://github.com/getporter/porter.git ~/go/src/get.porter.sh/porter. - Run
go run mage.go EnsureMageto install mage. - Run
mage Build Installfrom within the newly cloned repository.
If you are planning on contributing back to the project, you'll need to fork and clone your fork. If you want to build porter from scratch, you can follow the process above and clone directly from the project.
You now have canary builds of porter and all the mixins installed.
Porter uses a cross-platform make alternative called mage, where the targets are written in Go.
Mage targets are not case-sensitive, but in our docs we use camel case to make
it easier to read. You can run either mage TestSmoke or mage testsmoke for
example.
- Build builds all binaries, porter and internal mixins.
- BuildClient just builds the porter client for your operating system.
It does not build the porter-runtime binary. Useful when you just want to do a
build and don't remember the proper way to call
go buildyourself. - BuildPorter builds both the porter client and runtime.
- BuildClient just builds the porter client for your operating system.
It does not build the porter-runtime binary. Useful when you just want to do a
build and don't remember the proper way to call
- Clean removes artifacts from previous builds and test runs.
- UpdateTestfiles updates the "golden" test files to match the latest test output. This is mostly useful for when you change the schema of porter.yaml which will break TestPorter_PrintManifestSchema. Run this target to fix it. Learn more about golden files.
- Test runs all the tests.
- TestUnit runs the unit tests
- TestSmoke runs a small suite of tests using the Porter CLI to validate that Porter is (mostly) working.
- TestIntegration runs our integration tests, which run the bundles against a test KIND cluster.
- Install installs porter and the mixins from source into $(HOME)/.porter/.
- DocsPreview hosts the docs site. See Preview Documentation.
- DocsGen generates the CLI documentation for the website. This is run automatically by build.
- SetupDCO installs a git commit hook that automatically signsoff your commit messages per the DCO requirement.
We have a few different kinds of tests in Porter. You can run all tests types
with mage test.
mage TestUnit
Should not rely on Docker, or try to really run bundles without key components
mocked. Most structs have test functions, e.g. porter.NewTestPorter that are
appropriate for unit tests.
Fast! 🏎💨 This takes about 15s - 3 minutes, depending on your computer hardware.
mage TestIntegration
These tests run parts of Porter, using the Porter structs instead of the cli.
They can use Docker, expect that a cluster is available, etc. These tests all
use functions like porter.SetupIntegrationTest() to update the underlying
components so that they hit the real filesystem, and don't mock out stuff like
Docker.
You must have Docker on your computer to run these tests. The test setup handles creating a Kubernetes cluster and Docker registry. Since they are slow, it is perfectly fine to not run these locally and rely on the CI build that's triggered when you push commits to your pull request instead.
When I am troubleshooting an integration test, I will run just the single test
locally by using go test -run TESTNAME ./.... If the test needs infrastructure,
we have scripts that you can use, like mage StartDockerRegistry or
mage EnsureTestCluster.
Slow! 🐢 This takes between 8-16 minutes, depending on your computer hardware.
🚧 We are in the process of updating our integration tests to reduce total test time by combining test cases into larger tests so that we aren't running expensive set up operations like building bundles as often.
The goal is to make the integration tests use the same test framework tests/tester which tests porter using the CLI instead of by calling porter's code directly.
Follow the example of the hello smoke test for a good test to copy and make new tests from.
We use the Testify library's require package (instead of assert) in these long tests because we want the test to stop on the first failure instead of having the rest of the test output a bunch of failures too.
The first failure is the relevant one, and stopping immediately makes it faster and easier to see the root problem.
mage testSmoke
Smoke tests test Porter using the CLI and quickly identify big problems with a build that would make it unusable. If you intend to change Porter code (not docs), we recommend running smoke tests locally before submitting a pull request with your changes. Doing so allows you to immediately ensure you haven't broken or altered any core functionality.
Short! We want this to always be something you can run in under 3 minutes.
When you run mage build, the canary* build of mixins are automatically
installed into your bin directory in the root of the repository. You can use
porter mixin install NAME to install the latest released version of a mixin.
* canary = most recent successful build of the "main" branch
If you are developing a plugin and you want to debug it follow these steps:
The plugin to be debugged should be compiled and placed in porters plugin path (e.g. in the Azure plugin case the plugin would be copied to $PORTER_HOME/plugins/azure/.
The following environment variables should be set:
PORTER_RUN_PLUGIN_IN_DEBUGGER should be set to the name of the plugin to be
debugged (e.g. secrets.azure.keyvault to debug the azure secrets plugin)
PORTER_DEBUGGER_PORT should be set to the port number where the delve API will
listen, if not set it defaults to 2345
PORTER_PLUGIN_WORKING_DIRECTORY should be the path to the directory containing
source code for the plugin being executed.
When porter is run it will start delve and attach it to the plugin process, this exposes the delve API so that any delve client can connect to the server and debug the plugin.
We use Hugo to build our documentation site, and it is hosted on Netlify. You don't have to install Hugo locally because the preview happens inside a docker container.
- Run
mage DocsPreviewto start serving the docs. It will watch the file system for changes. - Our mage target should open http://localhost:1313/docs to preview the site/docs.
or use only Hugo
- Download and install Hugo 0.117.0 extended version.
cd docshugo server --watch
We welcome your contribution to improve our documentation, and we hope it is an easy process! ❤️
Thank you for writing a post for our blog! 🙇♀️ Here's what you need to do to create a new blog post and then preview it:
-
Go to /docs/content/blog and create a new file. Whatever you name the file will be the last part of the URL. For example a file named "porter-collaboration.md" will be located at https://porter.sh/blog/porter-collaboration/.
-
At the top of the file copy and paste the frontmatter template below. The frontmatter is YAML that instucts the blogging software, Hugo, how to render the blog post.
--- title: "Title of Your Blog Post in Titlecase" description: "SEO description of your post, displayed in search engine results." date: "2020-07-28" authorname: "Your Name" author: "@yourhandle" #Not used to link to github/twitter, but informally that's what people put here authorlink: "https://link/to/your/website" # link to your personal website, github, social media... authorimage: "https://link/to/your/profile/picture" # Optional, https://github.com/yourhandle.png works great tags: [] # Optional, look at other pages and pick tags that are already in use, e.g. ["mixins"] ---
-
Preview the website and click "Blog" at the top right to find your blog post.
-
When you create a pull request, look at the checks run by the pull request, and click "Details" on the netlify/porter/deploy-preview one to see a live preview of your pull request.
Our pull request preview and the live site will not show posts with a date in the future. If you don't see your post, change the date to today's date.
Porter can send trace data about the commands run to an OpenTelemetry backend. It can be very helpful when figuring out why a command failed because you can see the values of variables and stack traces.
In development, you can use the otel-jaeger bundle to set up a development instance of Jaeger, which gives you a nice website to see each command run.
porter install OtelJaeger --reference ghcr.io/getporter/examples/otel-jaeger:v0.1.0 --allow-docker-host-access
Then to turn on tracing in Porter, set the following environment variables. This tells Porter to turn on tracing, and connect to OpenTelemetry server that you just installed.
Posix
export PORTER_TELEMETRY_ENABLED="true"
export OTEL_EXPORTER_OTLP_PROTOCOL="grpc"
export OTEL_EXPORTER_OTLP_INSECURE="true"Powershell
$env:PORTER_TELEMETRY_ENABLED="true"
$env:OTEL_EXPORTER_OTLP_PROTOCOL="grpc"
$env:OTEL_EXPORTER_OTLP_INSECURE="true"Next run a Porter command to generate some trace data, such as porter list.
Then go to the Jaeger website to see your data: http://localhost:16686.
On the Jaeger dashboard, select "porter" from the service drop down, and click "Find Traces".
The smoke and integration tests will run with telemetry enabled when the PORTER_TEST_TELEMETRY_ENABLED environment variable is true.
If you want to attach a debugger to Porter when it is running in a smoke test, first install delve:
go install github.com/go-delve/delve/cmd/dlv@latest
Next, determine the porter command that you want the smoke test to automatically run through delve so that you can attach to it and debug.
Set PORTER_RUN_IN_DEBUGGER to the command(s) to run in delve.
Use porter to target any porter command, or target a specific command for example porter installation apply.
When the smoke tests run a porter command that has the prefix contained in PORTER_RUN_IN_DEBUGGER, instead of running porter directly, porter is run in delve. Delve will wait for you to attach a debugger, and then proceed to run the porter command so that you can debug into it.
The default debugger port is 55942 which you can override with the PORTER_DEBUGGER_PORT environment variable.
Now run the smoke test with go test -run TESTNAME -tags smoke ./tests/smoke, then use your Go IDE or delve directly to attach to Porter.
If you are using GoLand, use the Go Remote debug configuration and make sure to specify the same port that you used when running the smoke test (default is 55942).
Our commands are documented at https://porter.sh/docs/references/ and that documentation is
generated by our CLI. You should regenerate that documentation when you change
any files in cmd/porter by running mage DocsGen which is run every time
you run mage build.
Instructions for building the Porter Operator from source are located in its repository: https://github.com/getporter/operator. Sometimes you may need to make changes to Porter and work on the Operator at the same time. Here's how to build porter so that you can use it locally:
- You must be on a feature branch. Not main. This matters because it affects the generated docker image tag.
- Deploy the operator to a KinD cluster by running
mage deployfrom inside the operator repository. That cluster has a local registry running that you can publish to, and it will pull images from it, running on localhost:5000. - Run the following command from the porter repository to build the Porter Agent image, and publish it
to the test cluster's registry.
mage XBuildAll LocalPorterAgentBuild. - Edit your AgentConfig in the Porter Operator and set it to use your local build of the porter-agent.
apiVersion: porter.sh/v1
kind: AgentConfig
metadata:
name: porter
namespace: test # You may need to change this depending on what you are testing
spec:
porterRepository: localhost:5000/porter-agent
porterVersion: canary-dev
serviceAccount: porter-agentCarolyn Van Slyck gave a talk about the design of Porter, Designing Command-Line Tools People Love that you may find helpful in understanding the why's behind its command grammar, package structure, use of dependency injection and testing strategies.
- cmd: go here to add a new command or flag to porter or one of the mixins in this repository
- docs: our website
- pkg
- build: implements building the bundle image.
- cache: handles the cache of bundles that have been pulled by commands
like
porter install --reference. - cnab: deals with the CNAB spec
- cnab-to-oci: talking to an OCI registry.
- config-adapter: converting porter.yaml to bundle.json.
- extensions: extensions to the CNAB spec, at this point that's just dependencies.
- provider: the CNAB runtime, i.e.
porter install.
- config: anything related to
porter.yamland~/.porter. - context: essentially dependency injection that's needed throughout Porter, such as stdout, stderr, stdin, filesystem and command execution.
- exec: the exec mixin
- mixin: enums, functions and interfaces for the mixin framework.
- feed: works with mixin atom feeds
- provider: handles communicating with mixins
- porter: the implementation of the porter commands. Every command in Porter
has a corresponding function in here.
- version: reusable library used by all the mixins for implementing a mixin
- secrets: used to access porter's secret store through plugins.
- storage: used to access porter's data store through plugins.
- templates: files that need to be compiled into the porter binary with version command.
- scripts:
- install: Porter installation scripts
- tests have Go-based integration tests.
Print to the Out property for informational messages and send debug messages to the Err property.
Example:
fmt.Fprintln(p.Out, "Initiating battlestar protocol")
fmt.Fprintln(p.Err, "DEBUG: loading plans from r2d2...")Most of the structs in Porter have an embedded
get.porter.sh/porter/pkg/portercontext.Context struct. This has both Out and
Err which represent stdout and stderr respectively. You should log to those
instead of directly to stdout/stderr because that is how we capture output in
our unit tests. That means use fmt.Fprint* instead of fmt.Print* so that you
can pass in Out or Err.
Some of our commands are designed to be consumed by another tool and intermixing
debug lines and the command output would make the resulting output unusable. For
example, porter schema outputs a json schema and if log lines were sent to
stdout as well, then the resulting json schema would be unparsable. This is why
we send regular command output to Out and debug information to Err. It
allows us to then run the command and see the debug output separately, like so
porter schema --debug 2> err.log.
Sometimes when debugging your code you may need to print out variables that can contain sensitive data, for example printing out the resolved values for parameters and credentials. In this case, do not use fmt.Println and instead use the open telemetry trace logger to include the data in attributes.
func myFunc(ctx context.Context) {
ctx, span := tracing.StartSpan(ctx)
defer span.EndSpan()
// This contains resolved sensitive values, so only trace it in special dev builds (nothing is traced for release builds)
span.SetSensitiveAttributes(attribute.String("sensitive-stuff", mysensitiveVar))
}In normal builds of Porter, created with go build, mage Build or mage XBuildAll, no sensitive data is ever traced because SetSensitiveAttributes is compiled to do nothing by default.
Calls to SetSensitiveAttributes are only implemented when Porter is built with the traceSensitiveAttributes build tag.
Each time you call SetSensitiveAttributes include in your comment why the data is sensitive, and that it is only traced in special dev builds since people may not think to read the function documentation.
In order to debug your code and see the sensitive data, you need to:
- Compile a build of porter with sensitive tracing turned on,
go build -tags traceSensitiveAttributes -o bin/porter ./cmd/porter. - Enable tracing with Jaeger
- Run a porter command and then view the sensitive attributes in the trace data sent to Jaeger.
Some changes in Porter break our compatibility with previous versions of Porter. When that happens, we need to release that change with a new major version number to indicate to users that it contains breaking changes. When you realize that you may need to make a breaking change, discuss it with a maintainer on the issue or pull request and we'll come up with a plan for how it should be released. Here are some examples of breaking changes:
- The schema of porter.yaml changed.
- The schema of Porter's file formats changed.
- The schema of Porter's config file changed.
- Flags or behavior of a CLI command changed, such as removing a flag or adding a validation that can result in a hard error, preventing the command from running.
All of Porter's documents have a schemaVersion field and when the schema of the document is changed, the version number should be incremented as well in the default set on new documents, the supported schema version constant in the code, and in the documentation for that document.
This section includes overviews of infrastructure Porter relies on, mostly intended for maintainers.
See the CDN Setup Doc for details on the services Porter uses to host and distribute its release binaries.
Some of our tests need to run on Windows, like the Smoke Tests - Windows stage of our build pipeline. We use a custom Windows agent registered with Azure Pipelines that we build and maintain ourselves. See the Custom Windows CI Agent documentation for details on how the agent is created and configured.
Our version strategy explains how we version the project, when you should expect breaking changes in a release, and the process for the v1 release.