Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Type annotations throughout the engine for PyCharm #151

Merged
merged 26 commits into from
May 8, 2017

Conversation

pmer
Copy link
Member

@pmer pmer commented May 8, 2017

Related to #142 and #148.

While adding type annotations, PyCharm showed some new highlights in the code that contained a few bugs so I fixed them. \o/

Some modifications were made to allow the type checker to work:

  • The type checker doesn't follow builtin variables... it would have to solve the halting problem to do that reliably. So all the source files that use CHECK or CheckFailure now import it from __main__.
  • Standardize on using a list rather than a tuple for srcrect and dstrect.
  • Re-arrange the imports in __main__.py (moving the import lines down lower) to avoid circular imports in Python 3. This is because __main__.py imports pretty much every file, and those files all now import __main__ themselves due to the first bullet point.

All the errors that PyCharm was throwing around from the CHECK() calls being undefined disappeared once CHECK was imported rather than being passed through the builtins.

Why is this useful by itself?

It causes PyCharm to highlight more types of logic errors while working on the engine. It is useful to engine developers.

What do static type check errors look like?

screen shot 2017-05-07 at 7 39 26 pm

screen shot 2017-05-07 at 7 42 41 pm

screen shot 2017-05-07 at 7 43 43 pm

But it's not perfect:

screen shot 2017-05-07 at 7 48 05 pm

Ideally that should have shown an error. PEP 526 solves this but it requires Python 3.6. We can poke around with that if you want whenever we next upgrade the minimum language version.

How do I write my own type annotations?

This is explained in detail by Guido in PEP 484 but I'll try to condense the high level ideas below.

  • Try to add annotations to the parameters and return values of all functions & methods. You can skip the self parameter on methods, though. Annotations are completely optional (like documentation!), so if you leave them out nothing bad will happen. But if you fill them in we'll get static type checks within that function and on all calls to that function.
  • An annotation can either be a regular Python type like str, float, int, list (a list with no further restrictions), dict, etc. Or it can be the value None which stands for the type of the None value. Or it can be a type from the new typing module introduced with Python 3.5. The types from this module allow more expression so you can write List[int] (a list that is restricted to holding ints), Union[str, int] (something that can either be a string or an int) and so on. The typing module provides Optional[T] as shorthand for Union[T, None] since that's so common. This means something with the Optional[str] type can either be None or a str. Also, you can just write float in places where you'd ordinarily want to write Union[int, float] since PEP 484 says otherwise there'd be pretty much no reason to just use float by itself! Finally, an annotation can be a class that is in the current scope, either because it was declared earlier in the file above or a Python class that was imported from another file.
  • If you need to refer to a class that doesn't exist yet because (1) it's lower down in the file, (2) it's currently being defined, or (3) Python is still importing the file it is supposed to come from (this happens as the result of a circular dependency) then put your annotation in a string. This is officially sanctioned by PEP 484 as proper and type checkers will understand it. You'll know you need to do this if adding a type annotation breaks the program & you get an error at runtime.

Can I have a tool write the type annotations for me?

Yes, PyCharm and Pytype do this. The former is the easier of the two to use.

Next steps

My plans for after this PR.

  • There are some situations where PyCharm should show an error but it doesn't. I'm working on figuring out what causes this & hope to fix it.
  • Get Mypy to work with these annotations.
  • Add type checking to world code.

@pmer pmer requested a review from seisatsu May 8, 2017 02:49
@pmer pmer force-pushed the pmer/type-annotations branch from 8fd5b1a to 7e24772 Compare May 8, 2017 03:51
@pmer pmer force-pushed the pmer/type-annotations branch from 7e24772 to 98bb029 Compare May 8, 2017 03:56
@pmer pmer changed the title Add type annotations to engine (but not world... yet) Type annotations throughout the engine for PyCharm May 8, 2017
@seisatsu
Copy link
Member

seisatsu commented May 8, 2017

I like the way this looks!

@pmer
Copy link
Member Author

pmer commented May 8, 2017

Oh yeah if a parameter is annotated by the programmer with some type T but it has a default value of None then its type is automatically upgraded to Optional[T] during type checking.

@seisatsu seisatsu added the style label Oct 5, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants