Add new Package Management system based on hatch #90

luclaurent · 2024-11-29T10:41:51Z

The main idea is to use more moderne approach based on hatch PDM. This approach allow to use for instance the following commands:

… to pyproject.toml

… rename bin's scripts)

…ions

… (remove update of TEXINPUTS variable)

…t for calculated parser

…oml + decalre all test in tests' __init__.py

…le2amc)

luclaurent · 2024-12-02T14:46:23Z

pyproject.toml

+[tool.hatch.build]
+packages = ["amc2moodle"]
+# only-packages = true
+ignore-vcs = false


to be checked

With this all files in the source tree not filtered with the .gitignore are included in the sdist and in the whl.
The approach with the include seems perhaps better finally.

include =[ "LICENSE", "README.md", "amc2moodle/moodle2amc/README.md", "amc2moodle/amc2moodle/README.md", "amc2moodle/amc2moodle/bin/amc2moodle", "amc2moodle/amc2moodle/*.xslt", "amc2moodle/amc2moodle/*sty.ltxml", "tests/payload_test_amc2moodle/Figures/*", "tests/payload_test_amc2moodle/Figures/other/*", "tests/payload_test_amc2moodle/4.png", "tests/payload_test_amc2moodle/QCM.pdf", "tests/payload_test_amc2moodle/QCM.tex", "tests/payload_test_amc2moodle/QCM.xml", "tests/payload_test_amc2moodle/QCM_wo-tikz.tex", "tests/payload_test_amc2moodle/QCM_wo-tikz.xml", "tests/payload_test_amc2moodle/tikz.tex", "tests/payload_test_amc2moodle/common-bank.tex", "tests/payload_test_amc2moodle/numerical.tex", "tests/payload_test_amc2moodle/numerical.xml", "tests/payload_test_amc2moodle/cleaning.tex", "tests/payload_test_amc2moodle/cleaning.xml", "tests/payload_test_amc2moodle/element.tex", "tests/payload_test_amc2moodle/includestyles.tex", "amc2moodle/moodle2amc/bin/moodle2amc", "amc2moodle/moodle2amc/html2tex.xslt", "amc2moodle/moodle2amc/struc2tex.xslt", "tests/payload_test_moodle2amc/automultiplechoice.sty", "tests/payload_test_moodle2amc/moodle-bank-exemple.xml", "tests/payload_test_moodle2amc/moodle-bank-exemple.tex", "tests/payload_test_moodle2amc/moodle-bank-exemple.pdf"]

doesn't seems to prevent other files to be added.

I was expecting that hatch can use .gitignore filter + use only tracked file but it doesn't seem possible. To build, a clean working directory is still needed :-(

So finally gitignore seems perhaps the simplest way...

Run hatch build and unzip -l dist/amc2moodle-2.11.0-py3-none-any.whl

with ignore-vcs=false:

Archive: dist/amc2moodle-2.11.0-py3-none-any.whl Length Date Time Name --------- ---------- ----- ---- 1214 02-02-2020 00:00 amc2moodle/__init__.py 23 02-02-2020 00:00 amc2moodle/_version.py 14865 02-02-2020 00:00 amc2moodle/amc2moodle/README.md 0 02-02-2020 00:00 amc2moodle/amc2moodle/__init__.py 14237 02-02-2020 00:00 amc2moodle/amc2moodle/amc2moodle_class.py 9030 02-02-2020 00:00 amc2moodle/amc2moodle/automultiplechoice.sty.ltxml 43296 02-02-2020 00:00 amc2moodle/amc2moodle/convert.py 2350 02-02-2020 00:00 amc2moodle/amc2moodle/fp.sty.ltxml 370 02-02-2020 00:00 amc2moodle/amc2moodle/mhchem.sty.ltxml 4864 02-02-2020 00:00 amc2moodle/amc2moodle/transform.xslt 15439 02-02-2020 00:00 amc2moodle/amc2moodle/transform2html.xslt 741 02-02-2020 00:00 amc2moodle/amc2moodle/transform_ns.xslt 4581 02-02-2020 00:00 amc2moodle/amc2moodle/transform_qtype.xslt 6893 02-02-2020 00:00 amc2moodle/amc2moodle/bin/amc2moodle.py 3606 02-02-2020 00:00 amc2moodle/moodle2amc/README.md 169 02-02-2020 00:00 amc2moodle/moodle2amc/__init__.py 24322 02-02-2020 00:00 amc2moodle/moodle2amc/_questions.py 13056 02-02-2020 00:00 amc2moodle/moodle2amc/_quiz.py 5691 02-02-2020 00:00 amc2moodle/moodle2amc/html2tex.xslt 2708 02-02-2020 00:00 amc2moodle/moodle2amc/struc2tex.xslt 5905 02-02-2020 00:00 amc2moodle/moodle2amc/bin/moodle2amc.py 1512 02-02-2020 00:00 amc2moodle/tests/__init__.py 18807 02-02-2020 00:00 amc2moodle/tests/test_amc2moodle.py 4200 02-02-2020 00:00 amc2moodle/tests/test_moodle2amc.py 7532 02-02-2020 00:00 amc2moodle/tests/test_utils_calculatedParser.py 2616 02-02-2020 00:00 amc2moodle/tests/test_utils_text.py 1312 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/4.png 187722 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/QCM.pdf 2981 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/QCM.tex 63905 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/QCM.xml 2983 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/QCM_wo-tikz.tex 59980 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/QCM_wo-tikz.xml 3143 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/cleaning.tex 2171 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/cleaning.xml 10906 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/common-bank.tex 2573 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/element.tex 440 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/includestyles.tex 4397 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/numerical.tex 4392 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/numerical.xml 484 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/tikz.tex 3386 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/Figures/tinymonk.pdf 3896 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/Figures/other/4r.png 14813 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/Figures/other/schema_interpL.png 48501 02-02-2020 00:00 amc2moodle/tests/payload_test_moodle2amc/automultiplechoice.sty 141172 02-02-2020 00:00 amc2moodle/tests/payload_test_moodle2amc/moodle-bank-exemple.pdf 10728 02-02-2020 00:00 amc2moodle/tests/payload_test_moodle2amc/moodle-bank-exemple.tex 29495 02-02-2020 00:00 amc2moodle/tests/payload_test_moodle2amc/moodle-bank-exemple.xml 0 02-02-2020 00:00 amc2moodle/utils/__init__.py 18196 02-02-2020 00:00 amc2moodle/utils/calculatedParser.py 6151 02-02-2020 00:00 amc2moodle/utils/customLogging.py 5658 02-02-2020 00:00 amc2moodle/utils/flatex.py 2891 02-02-2020 00:00 amc2moodle/utils/misc.py 2656 02-02-2020 00:00 amc2moodle/utils/text.py 50136 02-02-2020 00:00 amc2moodle-2.11.0.dist-info/METADATA 87 02-02-2020 00:00 amc2moodle-2.11.0.dist-info/WHEEL 126 02-02-2020 00:00 amc2moodle-2.11.0.dist-info/entry_points.txt 35139 02-02-2020 00:00 amc2moodle-2.11.0.dist-info/licenses/LICENSE 5719 02-02-2020 00:00 amc2moodle-2.11.0.dist-info/RECORD --------- ------- 934166 58 files

with ignore-vcs=true

Archive: dist/amc2moodle-2.11.0-py3-none-any.whl Length Date Time Name --------- ---------- ----- ---- 1214 02-02-2020 00:00 amc2moodle/__init__.py 23 02-02-2020 00:00 amc2moodle/_version.py 14865 02-02-2020 00:00 amc2moodle/amc2moodle/README.md 0 02-02-2020 00:00 amc2moodle/amc2moodle/__init__.py 16347 02-02-2020 00:00 amc2moodle/amc2moodle/amc2moodle_class.latexml.log 14237 02-02-2020 00:00 amc2moodle/amc2moodle/amc2moodle_class.py 9030 02-02-2020 00:00 amc2moodle/amc2moodle/automultiplechoice.sty.ltxml 43296 02-02-2020 00:00 amc2moodle/amc2moodle/convert.py 2350 02-02-2020 00:00 amc2moodle/amc2moodle/fp.sty.ltxml 370 02-02-2020 00:00 amc2moodle/amc2moodle/mhchem.sty.ltxml 4864 02-02-2020 00:00 amc2moodle/amc2moodle/transform.xslt 15439 02-02-2020 00:00 amc2moodle/amc2moodle/transform2html.xslt 741 02-02-2020 00:00 amc2moodle/amc2moodle/transform_ns.xslt 4581 02-02-2020 00:00 amc2moodle/amc2moodle/transform_qtype.xslt 6893 02-02-2020 00:00 amc2moodle/amc2moodle/bin/amc2moodle.py 3606 02-02-2020 00:00 amc2moodle/moodle2amc/README.md 169 02-02-2020 00:00 amc2moodle/moodle2amc/__init__.py 24322 02-02-2020 00:00 amc2moodle/moodle2amc/_questions.py 13056 02-02-2020 00:00 amc2moodle/moodle2amc/_quiz.py 5691 02-02-2020 00:00 amc2moodle/moodle2amc/html2tex.xslt 2708 02-02-2020 00:00 amc2moodle/moodle2amc/struc2tex.xslt 5905 02-02-2020 00:00 amc2moodle/moodle2amc/bin/moodle2amc.py 1512 02-02-2020 00:00 amc2moodle/tests/__init__.py 18807 02-02-2020 00:00 amc2moodle/tests/test_amc2moodle.py 4200 02-02-2020 00:00 amc2moodle/tests/test_moodle2amc.py 7532 02-02-2020 00:00 amc2moodle/tests/test_utils_calculatedParser.py 2616 02-02-2020 00:00 amc2moodle/tests/test_utils_text.py 1312 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/4.png 187722 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/QCM.pdf 2981 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/QCM.tex 63905 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/QCM.xml 14198 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/QCM_iptjftm5_magic.tex 2983 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/QCM_wo-tikz.tex 59980 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/QCM_wo-tikz.xml 13737 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/QCM_wo-tikz_9tvop5bq_magic.tex 3143 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/cleaning.tex 2171 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/cleaning.xml 3143 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/cleaning_8e6huv9h_magic.tex 10906 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/common-bank.tex 2573 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/element.tex 440 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/includestyles.tex 4397 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/numerical.tex 4392 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/numerical.xml 484 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/tikz.tex 2370 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/xmlclean67042zf5 3386 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/Figures/tinymonk.pdf 3896 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/Figures/other/4r.png 14813 02-02-2020 00:00 amc2moodle/tests/payload_test_amc2moodle/Figures/other/schema_interpL.png 48501 02-02-2020 00:00 amc2moodle/tests/payload_test_moodle2amc/automultiplechoice.sty 141172 02-02-2020 00:00 amc2moodle/tests/payload_test_moodle2amc/moodle-bank-exemple.pdf 10728 02-02-2020 00:00 amc2moodle/tests/payload_test_moodle2amc/moodle-bank-exemple.tex 29495 02-02-2020 00:00 amc2moodle/tests/payload_test_moodle2amc/moodle-bank-exemple.xml 0 02-02-2020 00:00 amc2moodle/utils/__init__.py 18196 02-02-2020 00:00 amc2moodle/utils/calculatedParser.py 6151 02-02-2020 00:00 amc2moodle/utils/customLogging.py 5658 02-02-2020 00:00 amc2moodle/utils/flatex.py 2891 02-02-2020 00:00 amc2moodle/utils/misc.py 2656 02-02-2020 00:00 amc2moodle/utils/text.py 50136 02-02-2020 00:00 amc2moodle-2.11.0.dist-info/METADATA 87 02-02-2020 00:00 amc2moodle-2.11.0.dist-info/WHEEL 126 02-02-2020 00:00 amc2moodle-2.11.0.dist-info/entry_points.txt 35139 02-02-2020 00:00 amc2moodle-2.11.0.dist-info/licenses/LICENSE 6316 02-02-2020 00:00 amc2moodle-2.11.0.dist-info/RECORD --------- ------- 984558 63 files

Note in the second case, 3 addtional files that are not managed by VCS (*_magic.tex and xmlcleanXXX). So we can use ignore-vcs=false.

yep, with a clean working environment :-)

In my case, when I ran thes tests, my cwd was not clean at all but I obtain the previous results.

…it) using ignore-vcs = false

nennigb

Good job !
I listed few small points in the review that could be clean-up.

.github/workflows/ci-mac-os.yml

MANIFEST.in

nennigb · 2024-12-02T15:47:43Z

amc2moodle/tests/__init__.py

+from .test_utils_calculatedParser import *
+from .test_utils_text import *
+# import glob
+# import unittest


Not sure it is required.

hatch test

works without these four lines

I clean the __init__.py . Note that pytest --pyargs amc2moodle will continue to work properly.

nennigb · 2024-12-02T15:51:51Z

pyproject.toml

+[tool.hatch.build]
+packages = ["amc2moodle"]
+# only-packages = true
+ignore-vcs = false


With this all files in the source tree not filtered with the .gitignore are included in the sdist and in the whl.
The approach with the include seems perhaps better finally.

include =[ "LICENSE", "README.md", "amc2moodle/moodle2amc/README.md", "amc2moodle/amc2moodle/README.md", "amc2moodle/amc2moodle/bin/amc2moodle", "amc2moodle/amc2moodle/*.xslt", "amc2moodle/amc2moodle/*sty.ltxml", "tests/payload_test_amc2moodle/Figures/*", "tests/payload_test_amc2moodle/Figures/other/*", "tests/payload_test_amc2moodle/4.png", "tests/payload_test_amc2moodle/QCM.pdf", "tests/payload_test_amc2moodle/QCM.tex", "tests/payload_test_amc2moodle/QCM.xml", "tests/payload_test_amc2moodle/QCM_wo-tikz.tex", "tests/payload_test_amc2moodle/QCM_wo-tikz.xml", "tests/payload_test_amc2moodle/tikz.tex", "tests/payload_test_amc2moodle/common-bank.tex", "tests/payload_test_amc2moodle/numerical.tex", "tests/payload_test_amc2moodle/numerical.xml", "tests/payload_test_amc2moodle/cleaning.tex", "tests/payload_test_amc2moodle/cleaning.xml", "tests/payload_test_amc2moodle/element.tex", "tests/payload_test_amc2moodle/includestyles.tex", "amc2moodle/moodle2amc/bin/moodle2amc", "amc2moodle/moodle2amc/html2tex.xslt", "amc2moodle/moodle2amc/struc2tex.xslt", "tests/payload_test_moodle2amc/automultiplechoice.sty", "tests/payload_test_moodle2amc/moodle-bank-exemple.xml", "tests/payload_test_moodle2amc/moodle-bank-exemple.tex", "tests/payload_test_moodle2amc/moodle-bank-exemple.pdf"]

doesn't seems to prevent other files to be added.

nennigb · 2024-12-02T16:27:46Z

amc2moodle/tests/test_amc2moodle.py

@@ -34,6 +34,15 @@
 # Catch Logger
 Logger = logObj.getLogger()



In all test files, I suggest to remove __NAME__ and to use _NAME.

nennigb · 2024-12-02T16:40:14Z

pyproject.toml

+[tool.hatch.build]
+packages = ["amc2moodle"]
+# only-packages = true
+ignore-vcs = false


I was expecting that hatch can use .gitignore filter + use only tracked file but it doesn't seem possible. To build, a clean working directory is still needed :-(

So finally gitignore seems perhaps the simplest way...

nennigb · 2024-12-02T16:43:11Z

amc2moodle/tests/test_moodle2amc.py

@@ -47,17 +58,25 @@ class TestSuite(unittest.TestCase):
    teacher / sandbox

    """
-
+    @decorator_set_cwd(__OUTPUT_TEST_DIR__)


We need to check if it is really required.
If it is required, could you add a comment mentioning the reason (the latex compilation ?)

Currently required by the LaTeX compilation (in order to find package automultiplechoice during testing). Could be remove with resolving #86 (proposing of fix are in branch branch fix-artifacts-behavior. Comment added

nennigb · 2024-12-02T16:48:43Z

amc2moodle/tests/test_moodle2amc.py

@@ -19,10 +19,11 @@
    along with this program.  If not, see <http://www.gnu.org/licenses/>.
 """
 from amc2moodle.utils.customLogging import customLogger
-from amc2moodle.amc2moodle.test import check_hash
+from amc2moodle.utils.misc import check_hash, for_all_methods, decorator_set_cwd


for_all_methods imported but not used.
If you modified the file, there is few whitespace on blank lines

nennigb · 2024-12-02T16:57:07Z

.gitignore


 # Distribution / packaging
 *.egg-info/
 build/
+dist/

 # pdoc


# tests output_tests/*

nennigb · 2024-12-02T17:03:31Z

pyproject.toml

+command_line = "-m unittest discover -s amc2moodle/tests/"
+
+[tool.coverage.html]
+directory = "coverage_html_report"


I didn't see how to see the html report with hatch.

you can run hatch run coverage html. Improvement of coverage support are proposed in #92 other extension could be to redefine hatch test based on pypa/hatch#1477

or maybe hatch test -c

…s and formatting issues

nennigb · 2024-12-03T08:02:02Z

For me it is fine. Thanks!
Concerning the path where tests are running I suggest to add in the CI a true install from the build wheel + running the tests. It could be done probably with hatch or with venv.
With this, we will test the two main scenarios i) running tests from working tree, ii) running tests after install from the wheel.
I don't know if you prefer doing here or elsewhere. If you want I can do it (base are in deploy.sh)

I let you merge the PR if you have few things to polish.

luclaurent linked an issue Nov 29, 2024 that may be closed by this pull request

Add new setup process based on more modern Python #87

Closed

luclaurent added 16 commits November 29, 2024 14:34

refactor folder for testing without installation of packages + switch…

79a2c51

… to pyproject.toml

update project with tests as submodule

7936abc

add test folder as submodule + decalre scripts in pyproject.toml (and…

8aa67d2

… rename bin's scripts)

move again to initial folder syntax to work well with new tests posit…

e0cf3bf

…ions

moves tests from test to test folder

895a178

add new utils file with check_hash and update tests + update workflow…

fb1db41

… (remove update of TEXINPUTS variable)

udpate workflows with correct syntax for tests

17d3f97

update test scripts with new output directory and rename test's scrip…

0ff10e8

…t for calculated parser

update workflows with new syntax and directory for tests

a1a7f13

try to use hatchling (issue with tests)

728c888

move tests' files to packages' folder + finish version of pyproject.t…

63bee9c

…oml + decalre all test in tests' __init__.py

small fix in workflows

7b534ea

fix location of file for latexml's test

73a350f

add decorator for forcing current direc workingory during tests (mood…

2b28a30

…le2amc)

update version

da159a8

add non verification of TexLive repo when run texlive install

33ba3ac

luclaurent force-pushed the new-setup-hatchling branch from 283b038 to 33ba3ac Compare November 29, 2024 13:38

luclaurent assigned luclaurent and nennigb Nov 29, 2024

remove .vscode/settings.json

8b5c0ef

luclaurent mentioned this pull request Nov 29, 2024

Fix issue for building docker container #91

Merged

luclaurent commented Dec 2, 2024

View reviewed changes

remove manual adding of files (non python), use declaration of VCS (g…

b1d4f76

…it) using ignore-vcs = false

nennigb requested changes Dec 2, 2024

View reviewed changes

remove MANIFEST.in, clean amc2moodle/tests/__init__.py, fix many typo…

a99b9d0

…s and formatting issues

nennigb approved these changes Dec 3, 2024

View reviewed changes

luclaurent mentioned this pull request Dec 3, 2024

Improvement of testings in CI #93

Open

slightly improve .gitignore

1b12be3

luclaurent merged commit a5b0b86 into master Dec 3, 2024
24 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add new Package Management system based on hatch #90

Add new Package Management system based on hatch #90

luclaurent commented Nov 29, 2024

luclaurent Dec 2, 2024

luclaurent Dec 2, 2024

nennigb Dec 2, 2024

nennigb Dec 2, 2024

luclaurent Dec 2, 2024

luclaurent Dec 2, 2024

nennigb Dec 3, 2024

luclaurent Dec 3, 2024

nennigb left a comment

nennigb Dec 2, 2024

luclaurent Dec 2, 2024

nennigb Dec 2, 2024

nennigb Dec 2, 2024

luclaurent Dec 2, 2024

nennigb Dec 2, 2024

nennigb Dec 2, 2024

luclaurent Dec 2, 2024

nennigb Dec 2, 2024

luclaurent Dec 2, 2024

nennigb Dec 2, 2024

luclaurent Dec 2, 2024

nennigb Dec 2, 2024

luclaurent Dec 2, 2024

luclaurent Dec 2, 2024

nennigb commented Dec 3, 2024 •

edited

Loading

Add new Package Management system based on hatch #90

Add new Package Management system based on hatch #90

Conversation

luclaurent commented Nov 29, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nennigb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nennigb commented Dec 3, 2024 • edited Loading

nennigb commented Dec 3, 2024 •

edited

Loading