Skip to content

Commit

Permalink
Add types-boto3 package everywhere
Browse files Browse the repository at this point in the history
  • Loading branch information
vemel committed Nov 29, 2024
1 parent 2d35d29 commit 68c5ea2
Show file tree
Hide file tree
Showing 31 changed files with 100 additions and 99 deletions.
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:

1. Install `boto3-stubs[...]`
1. Install `types-boto3[...]`
2. Run `mypy`/`pyright` on the following code sample

```python
Expand All @@ -34,4 +34,4 @@ import boto3
```

**Additional context**
Your OS, `boto3-stubs` installation method, `boto3` version, etc.
Your OS, `types-boto3` installation method, `boto3` version, etc.
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@ A clear and concise description of what is needed.
Provide a code sample to test the feature implementation.

**Additional context**
Your OS, `boto3-stubs` installation method, `boto3` version, etc.
Your OS, `types-boto3` installation method, `boto3` version, etc.
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/vulnerability_report.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: Vulnerability report
about: Report a vulnerability in boto3-stubs, types-aiobotocore or types-aioboto3
about: Report a vulnerability in types-boto3, boto3-stubs, types-aiobotocore or types-aioboto3
title: "Vulnerability report:"
labels: "🐞 bug"
assignees: ""
Expand All @@ -11,5 +11,5 @@ A clear and concise description of what the vulnerability is.

**Additional context**

Product: `boto3-stubs`/`types-aiobotocore`/`types-aioboto3`
Product: `types-boto3`/`boto3-stubs`/`types-aiobotocore`/`types-aioboto3`
Version: `x.x.x`
5 changes: 5 additions & 0 deletions .github/workflows/helpers.js
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,10 @@ async function getStubsVersions(boto3Version) {
return await getPackageVersions('boto3-stubs', boto3Version)
}

async function getTypesBoto3Versions(boto3Version) {
return await getPackageVersions('types-boto3', boto3Version)
}

async function getPackageVersions(packageName, version) {
const allVersions = await getReleaseVersions(packageName)
const versions = allVersions.filter(v => v === version || v.startsWith(`${version}.`))
Expand Down Expand Up @@ -300,6 +304,7 @@ module.exports = {
getBoto3Version,
getPackageVersions,
getStubsVersions,
getTypesBoto3Versions,
getBotocoreVersion,
extractVersions,
extractDownloadLinks,
Expand Down
55 changes: 20 additions & 35 deletions .github/workflows/publish_types_boto3.yml
Original file line number Diff line number Diff line change
Expand Up @@ -201,40 +201,24 @@ jobs:
run: |
rm -rf mypy_boto3_builder
python -m pip install mypy_boto3_builder
# - name: Build updated packages only
# if: needs.versions.outputs.build-all == 'false'
# env:
# VERSION: ${{ needs.versions.outputs.version }}
# EXTRA_FLAGS: ${{ needs.versions.outputs.extra-flags }}
# run: |
# rm -rf mypy_boto3_output/*

# echo "Building updated packages"
# python -m mypy_boto3_builder mypy_boto3_output \
# -b ${VERSION} \
# --product types-boto3 types-boto3-lite types-boto3-services \
# ${EXTRA_FLAGS} \
# --output-type wheel sdist \
# --download-static-stubs \
# -s updated -d
# - name: Build all packages
# if: needs.versions.outputs.build-all == 'true'
# env:
# VERSION: ${{ needs.versions.outputs.version }}
# EXTRA_FLAGS: ${{ needs.versions.outputs.extra-flags }}
# run: |
# rm -rf mypy_boto3_output/*
- name: Build updated packages only
if: needs.versions.outputs.build-all == 'false'
env:
VERSION: ${{ needs.versions.outputs.version }}
EXTRA_FLAGS: ${{ needs.versions.outputs.extra-flags }}
run: |
rm -rf mypy_boto3_output/*
# echo "Building all packages"
# python -m mypy_boto3_builder mypy_boto3_output \
# -b ${VERSION} \
# --product types-boto3 types-boto3-lite types-boto3-services \
# ${EXTRA_FLAGS} \
# --output-type wheel sdist \
# --download-static-stubs \
# -s all -d
# FIXME: Delete tomorrow
- name: Temp initial release all packages
echo "Building updated packages"
python -m mypy_boto3_builder mypy_boto3_output \
-b ${VERSION} \
--product types-boto3 types-boto3-lite types-boto3-services \
${EXTRA_FLAGS} \
--output-type wheel sdist \
--download-static-stubs \
-s updated -d
- name: Build all packages
if: needs.versions.outputs.build-all == 'true'
env:
VERSION: ${{ needs.versions.outputs.version }}
EXTRA_FLAGS: ${{ needs.versions.outputs.extra-flags }}
Expand All @@ -245,9 +229,10 @@ jobs:
python -m mypy_boto3_builder mypy_boto3_output \
-b ${VERSION} \
--product types-boto3 types-boto3-lite types-boto3-services \
${EXTRA_FLAGS} \
--output-type wheel sdist \
--skip-published \
-s all
--download-static-stubs \
-s all -d
- name: Install dependencies for publishing
run: |
python -m pip install setuptools wheel twine
Expand Down
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,11 @@

![boto3.typed](https://github.com/youtype/mypy_boto3_builder/raw/main/logo.png)

Type annotations builder for [boto3-stubs](https://pypi.org/project/boto3-stubs/), [types-aiobotocore](https://pypi.org/project/types-aiobotocore/), and [types-aioboto3](https://pypi.org/project/types-aioboto3/) projects. Compatible with
Type annotations generator for [types-boto3](https://pypi.org/project/types-boto3/),
[boto3-stubs](https://pypi.org/project/boto3-stubs/),
[types-aiobotocore](https://pypi.org/project/types-aiobotocore/),
and [types-aioboto3](https://pypi.org/project/types-aioboto3/) projects.
Compatible with
[VSCode](https://code.visualstudio.com/),
[PyCharm](https://www.jetbrains.com/pycharm/),
[Emacs](https://www.gnu.org/software/emacs/),
Expand Down
7 changes: 7 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Security Policy

## Supported `types-boto3` versions

| Version | Supported |
| ------- | ------------------ |
| 1.24.x | :white_check_mark: |
| <= 1.23.x | :x: |

## Supported `boto3-stubs` versions

| Version | Supported |
Expand Down
17 changes: 7 additions & 10 deletions docsmd/how_to_build.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,19 @@

```bash
# Install preferred version of `boto3`
python -m pip install boto3==1.16.25 botocore==1.19.25
python -m pip install boto3==1.35.71 botocore==1.35.71

# Install `mypy-boto3-builder`
python -m pip install mypy-boto3-builder

# Build all packages in mypy_boto3_output directory
python -m mypy_boto3_builder mypy_boto3_output
# Build all packages in typings directory
python -m mypy_boto3_builder ./typings --product types-boto3 types-boto3-services --output-type wheel

# Or specify required services explicitly
python -m mypy_boto3_builder mypy_boto3_output -s ec2 s3
python -m mypy_boto3_builder ./typings --product types-boto3 types-boto3-services --output-type wheel -s ec2 s3

# Install custom `boto3-stubs` packages
cd mypy_boto3_output
python -m pip install -e ./mypy_boto3_ec2_package
python -m pip install -e ./mypy_boto3_s3_package
python -m pip install -e ./boto3_stubs_package
# Install custom `types-boto3` packages
python -m pip install ./typings/*.whl
```

## With Docker image
Expand All @@ -44,7 +41,7 @@ docker run -v `pwd`/output:/output -ti mypy_boto3_builder_stable
docker run -v `pwd`/output:/output -ti mypy_boto3_builder_stable -s s3

# generate stubs for a specific boto3 version
docker run -e BOTO3_VERSION=1.16.25 BOTOCORE_VERSION=1.19.25 -v `pwd`/output:/output -ti mypy_boto3_builder_stable
docker run -e BOTO3_VERSION=1.35.71 BOTOCORE_VERSION=1.35.71 -v `pwd`/output:/output -ti mypy_boto3_builder_stable
```

- Install packages from `output` directory as described above
6 changes: 5 additions & 1 deletion docsmd/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
# Mypy Boto3 Builder

Type annotations builder for [boto3-stubs](https://pypi.org/project/boto3-stubs/), [types-aiobotocore](https://pypi.org/project/types-aiobotocore/), and [types-aioboto3](https://pypi.org/project/types-aioboto3/) projects. Compatible with
Type annotations generator for [types-boto3](https://pypi.org/project/types-boto3/),
[boto3-stubs](https://pypi.org/project/boto3-stubs/),
[types-aiobotocore](https://pypi.org/project/types-aiobotocore/),
and [types-aioboto3](https://pypi.org/project/types-aioboto3/) projects.
Compatible with
[VSCode](https://code.visualstudio.com/),
[PyCharm](https://www.jetbrains.com/pycharm/),
[Emacs](https://www.gnu.org/software/emacs/),
Expand Down
4 changes: 2 additions & 2 deletions mypy_boto3_builder/cli_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ def parse_args(args: Sequence[str]) -> CLINamespace:

parser = argparse.ArgumentParser(
PROG_NAME,
description="Builder for boto3-stubs and types-aiobotocore.",
description="Type annotations generator for boto3, aiobotocore and aioboto3.",
)
parser.add_argument("-d", "--debug", action="store_true", help="Show debug messages")
parser.add_argument(
Expand Down Expand Up @@ -188,7 +188,7 @@ def parse_args(args: Sequence[str]) -> CLINamespace:
parser.add_argument(
"--partial-overload",
action="store_true",
help="Build boto3-stubs client/service overload only for selected services",
help="Build client/service overload only for selected services",
)
parser.add_argument(
"--installed",
Expand Down
2 changes: 1 addition & 1 deletion mypy_boto3_builder/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
# Root path for the project
ROOT_PATH: Final = Path(__file__).parent

# Jinja2 templates for boto3-stubs
# Jinja2 templates
TEMPLATES_PATH: Final = ROOT_PATH / "templates"

# Max line length for formatting
Expand Down
2 changes: 1 addition & 1 deletion mypy_boto3_builder/parsers/parse_wrapper_package.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ def parse_types_boto3_package(
version: str,
) -> TypesBoto3Package:
"""
Parse data for boto3-stubs package.
Parse data for types-boto3 package.
Arguments:
service_names -- All available service names
Expand Down
2 changes: 1 addition & 1 deletion mypy_boto3_builder/parsers/wrapper_package_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@

class WrapperPackageParser:
"""
Parser for boto3-stubs/types-aiobotocore/types-aioboto3 package.
Parser for types-boto3/types-aiobotocore/types-aioboto3 package.
Arguments:
package -- Prepared wrapper package with data
Expand Down
2 changes: 1 addition & 1 deletion mypy_boto3_builder/service_name.py
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ def extras_name(self) -> str:

def is_essential(self) -> bool:
"""
Whether service is included to `boto3-stubs[essential]`.
Whether service is included to `types-library[essential]`.
"""
return self.name in self.ESSENTIAL_NAMES

Expand Down
2 changes: 1 addition & 1 deletion mypy_boto3_builder/structures/mypy_boto3_package.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""
Structure for boto3-stubs module.
Structure for mypy-boto3 package.
Copyright 2024 Vlad Emelianov
"""
Expand Down
2 changes: 1 addition & 1 deletion mypy_boto3_builder/structures/wrapper_package.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@

class WrapperPackage(Package, ABC):
"""
Structure for boto3-stubs/types-aiobotocore/types-aioboto3 module.
Main package module data.
"""

def __init__(
Expand Down
16 changes: 8 additions & 8 deletions mypy_boto3_builder/templates/boto3-stubs-full/README.md.jinja2
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ To speed up type checking and code completion, you can set types explicitly.
import boto3
from boto3.session import Session

from mypy_boto3_ec2.client import EC2Client
from mypy_boto3_ec2.service_resource import EC2ServiceResource
from mypy_boto3_ec2.waiter import BundleTaskCompleteWaiter
from mypy_boto3_ec2.paginator import DescribeVolumesPaginator
from {{ package.data.SERVICE_PREFIX }}_ec2.client import EC2Client
from {{ package.data.SERVICE_PREFIX }}_ec2.service_resource import EC2ServiceResource
from {{ package.data.SERVICE_PREFIX }}_ec2.waiter import BundleTaskCompleteWaiter
from {{ package.data.SERVICE_PREFIX }}_ec2.paginator import DescribeVolumesPaginator

session = Session(region_name="us-west-1")

Expand All @@ -28,7 +28,7 @@ describe_volumes_paginator: DescribeVolumesPaginator = ec2_client.get_paginator(

### Pylint compatibility

It is totally safe to use `TYPE_CHECKING` flag in order to avoid `boto3-stubs`
It is totally safe to use `TYPE_CHECKING` flag in order to avoid `{{ package.pypi_name }}`
dependency in production.
However, there is an issue in `pylint` that it complains about undefined
variables. To fix it, set all types to `object` in non-`TYPE_CHECKING` mode.
Expand All @@ -38,9 +38,9 @@ import boto3
from typing import TYPE_CHECKING

if TYPE_CHECKING:
from mypy_boto3_ec2 import EC2Client, EC2ServiceResource
from mypy_boto3_ec2.waiters import BundleTaskCompleteWaiter
from mypy_boto3_ec2.paginators import DescribeVolumesPaginator
from {{ package.data.SERVICE_PREFIX }}_ec2 import EC2Client, EC2ServiceResource
from {{ package.data.SERVICE_PREFIX }}_ec2.waiters import BundleTaskCompleteWaiter
from {{ package.data.SERVICE_PREFIX }}_ec2.paginators import DescribeVolumesPaginator
else:
EC2Client = object
EC2ServiceResource = object
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ setup(
{{ "\n " -}}"{{ classifier -}}",
{%- endfor %}
],
keywords='boto3 type-annotations boto3-stubs mypy typeshed autocomplete',
keywords='{{ package.library_name }} {{ package.library_name }}-stubs type-annotations typeshed autocomplete',
long_description=LONG_DESCRIPTION,
long_description_content_type="text/markdown",
package_data={
Expand Down
16 changes: 8 additions & 8 deletions mypy_boto3_builder/templates/boto3-stubs/README.md.jinja2
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@ To speed up type checking and code completion, you can set types explicitly.
import boto3
from boto3.session import Session

from mypy_boto3_ec2.client import EC2Client
from mypy_boto3_ec2.service_resource import EC2ServiceResource
from mypy_boto3_ec2.waiter import BundleTaskCompleteWaiter
from mypy_boto3_ec2.paginator import DescribeVolumesPaginator
from {{ package.data.SERVICE_PREFIX }}_ec2.client import EC2Client
from {{ package.data.SERVICE_PREFIX }}_ec2.service_resource import EC2ServiceResource
from {{ package.data.SERVICE_PREFIX }}_ec2.waiter import BundleTaskCompleteWaiter
from {{ package.data.SERVICE_PREFIX }}_ec2.paginator import DescribeVolumesPaginator

session = Session(region_name="us-west-1")

Expand All @@ -30,7 +30,7 @@ describe_volumes_paginator: DescribeVolumesPaginator = ec2_client.get_paginator(

### Pylint compatibility

It is totally safe to use `TYPE_CHECKING` flag in order to avoid `boto3-stubs`
It is totally safe to use `TYPE_CHECKING` flag in order to avoid `{{ package.pypi_name }}`
dependency in production.
However, there is an issue in `pylint` that it complains about undefined
variables. To fix it, set all types to `object` in non-`TYPE_CHECKING` mode.
Expand All @@ -40,9 +40,9 @@ import boto3
from typing import TYPE_CHECKING

if TYPE_CHECKING:
from mypy_boto3_ec2 import EC2Client, EC2ServiceResource
from mypy_boto3_ec2.waiters import BundleTaskCompleteWaiter
from mypy_boto3_ec2.paginators import DescribeVolumesPaginator
from {{ package.data.SERVICE_PREFIX }}_ec2 import EC2Client, EC2ServiceResource
from {{ package.data.SERVICE_PREFIX }}_ec2.waiters import BundleTaskCompleteWaiter
from {{ package.data.SERVICE_PREFIX }}_ec2.paginators import DescribeVolumesPaginator
else:
EC2Client = object
EC2ServiceResource = object
Expand Down
2 changes: 1 addition & 1 deletion mypy_boto3_builder/templates/boto3-stubs/setup.py.jinja2
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ setup(
{{ "\n " -}}"{{ classifier -}}",
{%- endfor %}
],
keywords='boto3 type-annotations boto3-stubs mypy typeshed autocomplete',
keywords='{{ package.library_name }} {{ package.library_name }}-stubs type-annotations typeshed autocomplete',
long_description=LONG_DESCRIPTION,
long_description_content_type="text/markdown",
package_data={"{{ package.name }}": ["py.typed", "*.pyi", "*/*.pyi"]},
Expand Down
2 changes: 1 addition & 1 deletion mypy_boto3_builder/templates/common/header.md.jinja2
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,4 @@ More information can be found in [{{ package.data.PYPI_STUBS_NAME }} docs]({{ pa

See how it helps you find and fix potential bugs:

![boto3-stubs demo]({{ builder_repo_url }}/raw/main/demo.gif)
![types-boto3 demo]({{ builder_repo_url }}/raw/main/demo.gif)
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ setup(
{{ "\n " -}}"{{ classifier -}}",
{%- endfor %}
],
keywords='{{ package.library_name }} {{ service_name.boto3_name }} type-annotations botocore mypy typeshed autocomplete',
keywords='{{ package.library_name }} {{ service_name.boto3_name }} {{ package.library_name }}-stubs type-annotations mypy typeshed autocomplete',
long_description=LONG_DESCRIPTION,
long_description_content_type="text/markdown",
package_data={"{{ package.name }}": ["py.typed", "*.pyi"]},
Expand Down
Loading

0 comments on commit 68c5ea2

Please sign in to comment.