Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python: Generate/write (basic) type stubs #184

Closed
dangotbanned opened this issue Aug 27, 2024 · 5 comments · Fixed by #185
Closed

python: Generate/write (basic) type stubs #184

dangotbanned opened this issue Aug 27, 2024 · 5 comments · Fixed by #185

Comments

@dangotbanned
Copy link
Member

Using vl_convert in an ipython environment a full listing of the API available - including full signatures and docs.

Sadly this isn't the case in a static environment, which leads to false positives from a type checker like the below.

https://github.com/vega/altair/blob/aeb0c721cce0c8099e0688d7f83beae2e0fa103d/tools/generate_schema_wrapper.py#L376

Screenshot

image

PyO3 seems to have some degree of support, not sure how complex it would be to implement

FYI, I'm less interested in the types and moreso the names of functions/parameters

@dangotbanned
Copy link
Member Author

dangotbanned commented Aug 27, 2024

Managed to put this together pretty quickly, as a python alternative:

from __future__ import annotations

import inspect
import operator
from functools import partial
from types import ModuleType 
from typing import Any, Iterator

import vl_convert as vlc


def predicate_module_member(member: Any, /, module: ModuleType) -> bool:
    if name := getattr(member, "__name__", None):
        return name in module.__all__
    else:
        return False


def extract_members(module: ModuleType, /) -> Iterator[tuple[str, inspect.Signature]]:
    for name, member in sorted(
        inspect.getmembers(module, partial(predicate_module_member, module=module)),
        key=operator.itemgetter(0),
    ):
        yield name, inspect.signature(member)


def format_member(name: str, signature: inspect.Signature, /) -> str:
    if not signature.parameters:
        return f"def {name}() -> Any: ..."
    params = ", ".join(f"{p}: Any" for p in signature.parameters)
    return f"def {name}({params}) -> Any: ..."


def generate_stub(module: ModuleType, /) -> str:
    _imports = ("from typing import Any",)
    imports = "\n".join(_imports)
    contents = "\n".join(starmap(format_member, extract_members(module)))
    return f"{imports}\n\n" f"__all__ = {module.__all__!r}\n\n" f"{contents}"


print(generate_stub(vlc))

Which after running the output through ruff produces:

vl_convert.__init__.pyi

from typing import Any

__all__ = [
    "__version__",
    "get_format_locale",
    "get_local_tz",
    "get_themes",
    "get_time_format_locale",
    "javascript_bundle",
    "register_font_directory",
    "svg_to_jpeg",
    "svg_to_pdf",
    "svg_to_png",
    "vega_to_html",
    "vega_to_jpeg",
    "vega_to_pdf",
    "vega_to_png",
    "vega_to_scenegraph",
    "vega_to_svg",
    "vega_to_url",
    "vegalite_to_html",
    "vegalite_to_jpeg",
    "vegalite_to_pdf",
    "vegalite_to_png",
    "vegalite_to_scenegraph",
    "vegalite_to_svg",
    "vegalite_to_url",
    "vegalite_to_vega",
]


def get_format_locale(name: Any) -> Any: ...
def get_local_tz() -> Any: ...
def get_themes() -> Any: ...
def get_time_format_locale(name: Any) -> Any: ...
def javascript_bundle(snippet: Any, vl_version: Any) -> Any: ...
def register_font_directory(font_dir: Any) -> Any: ...
def svg_to_jpeg(svg: Any, scale: Any, quality: Any) -> Any: ...
def svg_to_pdf(svg: Any, scale: Any) -> Any: ...
def svg_to_png(svg: Any, scale: Any, ppi: Any) -> Any: ...
def vega_to_html(
    vg_spec: Any,
    bundle: Any,
    format_locale: Any,
    time_format_locale: Any,
    renderer: Any,
) -> Any: ...
def vega_to_jpeg(
    vg_spec: Any,
    scale: Any,
    quality: Any,
    allowed_base_urls: Any,
    format_locale: Any,
    time_format_locale: Any,
) -> Any: ...
def vega_to_pdf(
    vg_spec: Any,
    scale: Any,
    allowed_base_urls: Any,
    format_locale: Any,
    time_format_locale: Any,
) -> Any: ...
def vega_to_png(
    vg_spec: Any,
    scale: Any,
    ppi: Any,
    allowed_base_urls: Any,
    format_locale: Any,
    time_format_locale: Any,
) -> Any: ...
def vega_to_scenegraph(
    vg_spec: Any, allowed_base_urls: Any, format_locale: Any, time_format_locale: Any
) -> Any: ...
def vega_to_svg(
    vg_spec: Any, allowed_base_urls: Any, format_locale: Any, time_format_locale: Any
) -> Any: ...
def vega_to_url(vg_spec: Any, fullscreen: Any) -> Any: ...
def vegalite_to_html(
    vl_spec: Any,
    vl_version: Any,
    bundle: Any,
    config: Any,
    theme: Any,
    format_locale: Any,
    time_format_locale: Any,
    renderer: Any,
) -> Any: ...
def vegalite_to_jpeg(
    vl_spec: Any,
    vl_version: Any,
    scale: Any,
    quality: Any,
    config: Any,
    theme: Any,
    show_warnings: Any,
    allowed_base_urls: Any,
    format_locale: Any,
    time_format_locale: Any,
) -> Any: ...
def vegalite_to_pdf(
    vl_spec: Any,
    vl_version: Any,
    scale: Any,
    config: Any,
    theme: Any,
    allowed_base_urls: Any,
    format_locale: Any,
    time_format_locale: Any,
) -> Any: ...
def vegalite_to_png(
    vl_spec: Any,
    vl_version: Any,
    scale: Any,
    ppi: Any,
    config: Any,
    theme: Any,
    show_warnings: Any,
    allowed_base_urls: Any,
    format_locale: Any,
    time_format_locale: Any,
) -> Any: ...
def vegalite_to_scenegraph(
    vl_spec: Any,
    vl_version: Any,
    config: Any,
    theme: Any,
    show_warnings: Any,
    allowed_base_urls: Any,
    format_locale: Any,
    time_format_locale: Any,
) -> Any: ...
def vegalite_to_svg(
    vl_spec: Any,
    vl_version: Any,
    config: Any,
    theme: Any,
    show_warnings: Any,
    allowed_base_urls: Any,
    format_locale: Any,
    time_format_locale: Any,
) -> Any: ...
def vegalite_to_url(vl_spec: Any, fullscreen: Any) -> Any: ...
def vegalite_to_vega(
    vl_spec: Any, vl_version: Any, config: Any, theme: Any, show_warnings: Any
) -> Any: ...

@jonmmease
Copy link
Collaborator

Thanks for looking into this,

I think I'd prefer to follow the If you do not have other Python files paradigm, and maintain a hand-written stub file at vl-convert-python/vl_convert.pyi.

But what you generated here is a great starting point, and then we can refine the types of the function args over time. How does that sound?

@dangotbanned
Copy link
Member Author

Thanks for looking into this,

I think I'd prefer to follow the If you do not have other Python files paradigm, and maintain a hand-written stub file at vl-convert-python/vl_convert.pyi.

But what you generated here is a great starting point, and then we can refine the types of the function args over time. How does that sound?

Sounds good to me!

After working through #184 (comment) it seems like there is a lot of overlap between the signatures, so a hand-written stub should be easy to maintain.

Another benefit there would be that including docstrings and accurate typing is only one extra (small) manual job to do in addition.

I'm more than happy to do that today, if you'd like?

@jonmmease
Copy link
Collaborator

I'm more than happy to do that today, if you'd like?

Sure!

@dangotbanned
Copy link
Member Author

I'm more than happy to do that today, if you'd like?

Sure!

Phew

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants