Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert the project from setuptools -> poetry #52

Merged
merged 22 commits into from
May 20, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
6a431d8
Set up poetry as the project package manager
jwilhelm-godaddy Apr 28, 2020
480fb37
Store the lock file for Poetry
jwilhelm-godaddy Apr 28, 2020
ce98c01
Remove setuptools-centric files
jwilhelm-godaddy Apr 28, 2020
a1e3bc6
Read project version from pyproject.toml
jwilhelm-godaddy Apr 28, 2020
7a9c166
Install the tartufo executable
jwilhelm-godaddy Apr 28, 2020
41b103c
Use a multi-stage build installing via Poetry
jwilhelm-godaddy Apr 28, 2020
bb0aa8b
Remove no longer relevant scripts
jwilhelm-godaddy Apr 28, 2020
fb49d10
Remove old requirements files
tarkatronic May 6, 2020
8872ecd
Fix installed dependencies to be more specific
tarkatronic May 6, 2020
dfd2856
Update the locked dependencies
tarkatronic May 6, 2020
4343813
Update tox to work with the new Poetry setup
tarkatronic May 6, 2020
3fd7634
Wheel config is no longer necessary in setup.cfg
tarkatronic May 6, 2020
ae6d7c6
Fully deprecate the setup.cfg
tarkatronic May 12, 2020
2e3c72f
Move pylint config into pyproject.toml
tarkatronic May 12, 2020
7e4c24c
Move coverage config into pyproject.toml
tarkatronic May 12, 2020
e10e923
Install poetry in Travis to support the new tox config
tarkatronic May 12, 2020
b12a23c
Attempt to switch over to poetry publish for deploys
tarkatronic May 12, 2020
f0c8f30
Add pylint
tarkatronic May 13, 2020
ddc62fc
Update contributing docs for poetry
tarkatronic May 13, 2020
e987f57
Point contributing doc to the full docs site
tarkatronic May 13, 2020
1f9a2ad
Small updates for clarity on the contributing doc
tarkatronic May 13, 2020
d2bf668
Fix a couple of grammar nits
tarkatronic May 20, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 0 additions & 24 deletions .coveragerc

This file was deleted.

39 changes: 0 additions & 39 deletions .pylintrc

This file was deleted.

1 change: 1 addition & 0 deletions .tartufo-excludes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
poetry.lock
32 changes: 18 additions & 14 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ python:
- "3.7"
- "3.8"
- "pypy3"
install: pip install tox-travis codecov
install: pip install poetry tox-travis codecov
script: tox
after_success: codecov

Expand Down Expand Up @@ -35,16 +35,20 @@ jobs:
install: true
script: true
after_success: true
deploy:
- provider: pypi
user: "__token__"
skip_existing: true
distributions: "sdist bdist_wheel"
on:
branch: master
tags: true
- provider: script
script: ./scripts/build_and_publish_docker.sh
on:
branch: master
tags: true

before_deploy:
- pip install --upgrade pip
- pip install poetry
- poetry config pypi-token.pypi $PYPI_PASSWORD

deploy:
- provider: script
script: poetry publish
on:
branch: master
tags: true
- provider: script
script: ./scripts/build_and_publish_docker.sh
on:
branch: master
tags: true
167 changes: 6 additions & 161 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,165 +1,10 @@
# Contributing

Everyone is welcome to contribute to GoDaddy's Open Source Software.
Contributing doesn’t just mean submitting pull requests. To get involved you can
report or triage bugs and participate in discussions on the evolution of each
project.
First of all, thank you for considering contributing to our project! We welcome
contributions of all shapes and sizes from all contributors!

No matter how you want to get involved, we ask that you first learn what’s
expected of anyone who participates in the project by reading the Contribution
Guidelines.
For more information and how to get started, please see the [contributing doc]
on our [main doc site].

**Please Note:** GitHub is for bug reports and contributions primarily - if you
have a support question head over to [GoDaddy's Open Source Software Slack][slack].

## Answering Questions

One of the most important and immediate ways you can support this project is to
answer questions on [Slack][slack] or [Github][issues]. Whether you’re helping a
newcomer understand a feature or troubleshooting an edge case with a seasoned
developer, your knowledge and experience with Python or security can go a long
way to help others.

## Reporting Bugs

**Do not report potential security vulnerabilities here. Refer to
[SECURITY.md](./SECURITY.md) for more details about the process of reporting
security vulnerabilities.**

Before submitting a ticket, please be sure to have a simple replication of the
behavior. If the issue is isolated to one of the dependencies of this project,
please create a Github issue in that project. All dependencies are open source
software and can be easily found through [PyPI].

Submit a ticket for your issue, assuming one does not already exist:

- Create it on our [Issue Tracker][issues]
- Clearly describe the issue by following the template layout
- Make sure to include steps to reproduce the bug.
- A reproducible (unit) test could be helpful in solving the bug.
- Describe the environment that (re)produced the problem.

> For a bug to be actionable, it needs to be reproducible. If you or
> contributors can’t reproduce the bug, try to figure out why. Please take care
> to stay involved in discussions around solving the problem.

## Triaging bugs or contributing code

If you're triaging a bug, try to reduce it. Once a bug can be reproduced, reduce
it to the smallest amount of code possible. Reasoning about a sample or unit
test that reproduces a bug in just a few lines of code is easier than reasoning
about a longer sample.

From a practical perspective, contributions are as simple as:

- Forking the repository on GitHub.
- Making changes to your forked repository.
- When committing, reference your issue (if present) and include a note about
the fix.
- If possible, and if applicable, please also add/update unit tests for your
changes.
- Push the changes to your fork and submit a pull request to the 'master' branch
of the projects' repository.

If you are interested in making a large change and feel unsure about its overall
effect, please make sure to first discuss the change and reach a consensus with
core contributors through [slack]. Then ask about the best way to go about
making the change.

## Code Review

Any open source project relies heavily on code review to improve software
quality:

> All significant changes, by all developers, must be reviewed before they are
> committed to the repository. Code reviews are conducted on GitHub through
> comments on pull requests or commits. The developer responsible for a code
> change is also responsible for making all necessary review-related changes.

Sometimes code reviews will take longer than you would hope for, especially for
larger features. Here are some accepted ways to speed up review times for your
patches:

- Review other people’s changes. If you help out, others will be more willing to
do the same for you. Good will is our currency.
- Split your change into multiple smaller changes. The smaller your change, the
higher the probability that somebody will take a quick look at it.
- Ping the change on [slack]. If it is urgent, provide reasons why it is
important to get this change landed. Remember that you’re asking for valuable
time from other professional developers.

**Note that anyone is welcome to review and give feedback on a change, but only
people with commit access to the repository can approve it.**

## Attribution of Changes

When contributors submit a change to this project, after that change is approved,
other developers with commit access may commit it for the author. When doing so,
it is important to retain correct attribution of the contribution. Generally
speaking, Git handles attribution automatically.

## Writing Code

### Setting Up A Development Environment

As a general best practice, you should work on this project in an isolated
Python virtual environment. The following set of commands will set that up for
you:

```sh
> python -m venv .venv
> source .venv/bin/activate
> pip install -e .[tests]
```

### Code Style

To make code formatting easy on developers, and to simplify the conversation
around pull request reviews, this project has adopted the
[black] code formatter. This formatter must be run against any new code written
for this project. The advantage is, you no longer have to think about how your
code is styled; it's all handled for you!

To make this easier on you, you can [set up most editors][black-editors] to
auto-run `black` for you. We have also set up a [pre-commit] hook to run
automatically on every commit, which is detailed below!

## Running tests

This project support multiple Python versions in this project. Thus, we ask
that you use the `tox` tool to test against them. If you do not have all of the
supported Python versions, that's perfectly okay. They will all be tested
against by our CI process. But keep in mind that this may delay the adoption
of your contribution, if those tests don't all pass.

Finally, this project uses multiple [pre-commit] hooks to help ensure our code
quality. If you have followed the instructions above for setting up your virtual
environment, `pre-commit` will already be installed, and you only need to run
the following:

```sh
> pre-commit install --install-hooks
```

Now, any time you make a new commit to the repository, you will see something
like the following:

```sh
Tartufo..................................................................Passed
mypy.....................................................................Passed
black....................................................................Passed
pylint...................................................................Passed
```

## Additional Resources

- [General GitHub Documentation](https://help.github.com/)
- [GitHub Pull Request documentation](https://help.github.com/send-pull-requests/)

[black]: https://github.com/psf/black
[black-editors]: https://github.com/psf/black#editor-integration
[issues]: https://github.com/godaddy/tartufo/issues
[PEP 8]: https://www.python.org/dev/peps/pep-0008/
[pre-commit]: https://pre-commit.com/
[PyPI]: http://pypi.org/
[slack]: https://godaddy-oss.slack.com/
[contributing doc]: https://tartufo.readthedocs.io/en/stable/CONTRIBUTING.html
[main doc site]: https://tartufo.readthedocs.io/en/stable/
27 changes: 22 additions & 5 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,14 +1,31 @@
FROM python:3-alpine
RUN apk update && apk add git openssh-client
FROM python:3-alpine as base

COPY . /app
WORKDIR /app

RUN pip install -e .
FROM base as builder

ENV PIP_DEFAULT_TIMEOUT=100 \
PIP_DISABLE_PIP_VERSION_CHECK=1 \
PIP_NO_CACHE_DIR=1 \
POETRY_VERSION=1.0.5

RUN apk add --no-cache gcc libffi-dev musl-dev openssl-dev
RUN pip --no-cache-dir install "poetry==$POETRY_VERSION"
RUN python -m venv /venv

COPY pyproject.toml poetry.lock ./
RUN poetry export -f requirements.txt | /venv/bin/pip install -r /dev/stdin

COPY . .
RUN poetry build && /venv/bin/pip install dist/*.whl
Comment on lines +14 to +20
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious why we are going this route, rather than just using poetry install and poetry build?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea here is that we ship a final image with none of the build artifacts; only the finally installed product. If you notice, poetry is not installed into the venv, and so doesn't actually get copied over to the final image. Same with the -dev packages installed on line 12. The main goal here is to provide as compact and clean an image as we can.

So with all of that in mind, the easiest way to get Poetry to use the Python interpreter in the venv, is to do this export and pip install.

Of course, all this is adapted from a number of examples I've found online. So I'd be happy if somebody is able to help build a better way!


FROM base as final

RUN apk add --no-cache git openssh-client
COPY --from=builder /venv /venv
COPY scripts/docker/gitconfig /root/.gitconfig

WORKDIR /git

ENTRYPOINT [ "tartufo" ]
ENTRYPOINT [ "/venv/bin/tartufo" ]
CMD [ "-h" ]
10 changes: 0 additions & 10 deletions MANIFEST.in

This file was deleted.

1 change: 0 additions & 1 deletion VERSION

This file was deleted.

Loading