Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: Add math example #156

Merged
merged 7 commits into from
Jan 4, 2024
Merged

Python: Add math example #156

merged 7 commits into from
Jan 4, 2024

Conversation

hillary-mutisya
Copy link
Contributor

@hillary-mutisya hillary-mutisya commented Jan 3, 2024

This change adds a simple calculator example which uses JsonProgram for its schema. This is similar to the math example in the typescript implementation. Some changes:

  1. The JsonProgram validator and translator are included in the example folder. We'll evaluate whether to include them in the main typechat python implementation at a later date.
  2. The translator takes an API that exposes the functions a model can call. The example shows two ways of providing this- either as a TypedDict with Callable values (schema.py) or as a Protocol with instance methods (schemaV2.py).

Copy link
Contributor

@pcdeadeasy pcdeadeasy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great to have parity between the py-typechat and typechat 👍

JsonProgram = TypedDict("Program", {"@steps": list[FunctionCall]})


async def evaluate_json_program(program: Mapping[str, Any], onCall: Callable[[str, list[Any]], Awaitable[Any]]) -> Any:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use Dict[str, Any], I haven't seen Mapping being used as frequently?
Instead of list[Any] can we Change it to List[any]. Also for clarity, can we use type hints from the typing module like List, Dict instead of list, dict etc.

Copy link
Contributor Author

@hillary-mutisya hillary-mutisya Jan 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it turns out, the typing module's Dict, List, Set, Tuple, Type and other types are deprecated aliases for their lowercase siblings (https://docs.python.org/3/library/typing.html#deprecated-aliases). The guidance is to use the lowercase instances of these constructs for folks writing new code.

We're currently building on python 3.12. If we want to support conventions from previous python versions, we may need to pull in typing_extensions as a dependency. @DanielRosenwasser suggested this in another PR and I will be looking into this.

async def evaluate_array(array: list[Any]) -> list[Any]:
return await asyncio.gather(*[evaluate_call(e) for e in array])

async def evaluate_object(expr: FunctionCall):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be good to define the return types expected for this function like maybe Union[List[str], ... , None]:

Copy link
Contributor Author

@hillary-mutisya hillary-mutisya Jan 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call, I updated the methods to return Expression or Sequence[Expression]. This enables us to handle simple str, int, float etc. types, user-defined objects, callable types and more. JsonPrograms are very versatile.



class TypeChatProgramValidator(TypeChatValidator[T]):
def __init__(self, py_type: type[T]):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type instead of type

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The uppercase Type has been deprecated in favor of the lowercase type: https://docs.python.org/3/library/typing.html#typing.Type

Copy link
Member

@DanielRosenwasser DanielRosenwasser Jan 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be mistaken, but I do think it should be type[T]

Copy link
Contributor

@pcdeadeasy pcdeadeasy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good.

@hillary-mutisya hillary-mutisya merged commit 7fa71bc into main Jan 4, 2024
6 checks passed
return s


program_schema_text = '''
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just generate this with python_type_to_typescript_schema so you have a single source of truth?

async def evaluate_object(expr: FunctionCall):
if "@ref" in expr:
index = expr["@ref"]
if index < len(results):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably need to make sure expr is an int, and report otherwise, and report if it is out of bounds.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd get rid of schemaV2, it's not used.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's sync up on this. As I see it, the goal of the examples is to show devs how to use the library. In this case, the program APIs can be presented using the typeddict-of-callables approach (schema.py) or the protocol approach (schemav2.py). We support both approaches and the devs are free to choose either one. I'm going to add some Readme files that will explain how to switch between the different schema definitions

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rename this to program_translator

@DanielRosenwasser DanielRosenwasser deleted the users/hillarym/math branch March 29, 2024 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants