Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use a new jinja2 templating process to update library to latest unicode (14.0) #61

Merged
merged 59 commits into from
Jan 15, 2023

Conversation

jquast
Copy link
Owner

@jquast jquast commented Mar 20, 2022

  • update tox.ini, requirements* files
    • use pip-compile with "requirements.in" files to generate "requirements.txt" files
    • use older testing tools for older python versions (requirements-tests38.in and requirements-tests39.in)
    • use python3.10 for most build steps
  • bin/update-tables.py changes,
    • use https
    • use jinja2 (.j2) templating to generate code and documentation,
      • easier to support code generation of other languages
    • unreliable connection support using tenacity library to retry on connection timeout, etc.
    • optional --check-last-modified will compare last-modified for redownload
      • a unicode.org data file was only once updated for a botched release
  • for wcwidth/table*.py files
    • uses new template system to generate new table data files for unicode version 14
    • when a description of unicode is single codepoint, describe only as 'A' instead of 'A..A'

@jquast jquast requested a review from GalaxySnail March 20, 2022 22:59
@jquast jquast self-assigned this Mar 20, 2022
@jquast jquast changed the title Use jinja2 to update library to latest unicode (14.0) Use a new jinja2 templating process to update library to latest unicode (14.0) Mar 20, 2022
@lgtm-com

This comment was marked as outdated.

@lgtm-com
Copy link

lgtm-com bot commented Mar 20, 2022

This pull request fixes 2 alerts when merging bbe041b into 4d45480 - view on LGTM.com

fixed alerts:

  • 2 for Variable defined multiple times

@jquast jquast requested a review from GalaxySnail January 14, 2023 00:54
This was referenced Jan 14, 2023
Comment on lines +51 to +53
JINJA_ENV = jinja2.Environment(
loader=jinja2.FileSystemLoader(os.path.join(PATH_UP, 'code_templates')),
keep_trailing_newline=True)

Check warning

Code scanning / CodeQL

Jinja2 templating with autoescape=False

Using jinja2 templates with autoescape=False can potentially allow XSS attacks.
name_start, name_end = '(nil)', '(nil)'
try:
name_start = string.capwords(unicodedata.name(ucs_start))
except ValueError:

Check notice

Code scanning / CodeQL

Empty except

'except' clause does nothing but pass and there is no explanatory comment.
pass
try:
name_end = string.capwords(unicodedata.name(ucs_end))
except ValueError:

Check notice

Code scanning / CodeQL

Empty except

'except' clause does nothing but pass and there is no explanatory comment.
bin/update-tables.py Outdated Show resolved Hide resolved
tox.ini Outdated Show resolved Hide resolved
Copy link
Owner Author

@jquast jquast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the help, I'll merge & release in ~8 hours or so, I just want to be sure I am home to address any issues in release, and maybe address a few small open issues as well, #22, #24, #55, #56.

With any luck I will be able to work on Cythonification tomorrow, I found this stack overflow solution that addresses my concern, that we can include a cython or c extension as optional and fallback to pure python if it is not able to compile, https://stackoverflow.com/questions/41778153/compiling-an-optional-cython-extension-only-when-possible-in-setup-py

@jquast jquast requested a review from GalaxySnail January 15, 2023 03:46
@GalaxySnail
Copy link
Collaborator

GalaxySnail commented Jan 15, 2023

thanks for the help, I'll merge & release in ~8 hours or so, I just want to be sure I am home to address any issues in release, and maybe address a few small open issues as well, #22, #24, #55, #56.

Thanks for your efforts! Everything LGTM. Updating to python 3.11 in tox.ini can be done in another PR.

With any luck I will be able to work on Cythonification tomorrow, I found this stack overflow solution that addresses my concern, that we can include a cython or c extension as optional and fallback to pure python if it is not able to compile, https://stackoverflow.com/questions/41778153/compiling-an-optional-cython-extension-only-when-possible-in-setup-py

This anwser looks really old, we have pyproject.toml and PEP 517 nowadays. The get_requires_for_build_wheel hook and an in-tree build backend will be helpful. Please see: psycopg/psycopg#471 (comment). Also, I think it can be done in another PR.

@jquast jquast merged commit f0232e5 into master Jan 15, 2023
@jquast jquast deleted the use-jinja branch January 15, 2023 03:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants