Footnotes translation

[ValentinConstantin]

Describe the problem

The expected behaviour of footnotes with text before and after when they are translated is wrong (without text, everything seems to be ok).

The reference to footnotes like [^2] doesn't work. The only way that I can reference to footnotes is [^2]: , but this way works only if the footnotes doesn't have text before and after.

It fails with:
footnotes_i18n/source/introduction.md:8: WARNING: inconsistent footnote references in translated message. original: ['[^2]'], translated: []

Link to your repository or website

https://github.com/ValentinConstantin/footnotes_i18n

Steps to reproduce

python3 -m venv .venv
source .venv/bin/activate
pip install pip wheel setuptools
pip install -r requirements.txt
make -e SPHINXOPTS="-v -D language='fr'" html
xdg-open build/html/index.html
make -e SPHINXOPTS="-v -D language='en'" html

The version of Python you're using

Python 3.9.8

Your operating system

No response

Versions of your packages

No response

Additional context

No response

[welcome]

Thanks for opening your first issue here! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.

If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).

Welcome to the EBP community! 🎉

[chrisjsewell]

ping you again for this @jpmckinney 😬

[jpmckinney]

I’ve never used footnotes (I sometimes forget they exist) so this scenario wasn’t covered in the tests. I’ll try to have a look.

[jpmckinney]

Hmm, in the past I looked at how I fixed i18n in recommonmark, but recommonmark never supported footnotes: readthedocs/recommonmark#62

I can reproduce the error, but I have no idea. The footnote code is not clear as crystal. Sphinx has some special logic for footnotes, but not much, and I can't see a problem on that side.

Someone more familiar with the footnote code will have to read it and identify if there are any cases where the nodes/tokens are being broken up in unexpected ways, such that the (translated) node that Sphinx sees lacks a reference. For example, if the text and footnote-reference are split apart by MyST-Parser, but Sphinx expects them to be together in one node, then that's one way you'll end up seeing this error.

The code makes a lot more docutils calls that Sphinx does not make (like note_footnote_ref).

[jpmckinney]

Some of the changes to the tests fixtures for later (I haven't updated the XML and HTML fixtures, as the warning interferes). This patch is based off #459:

diff --git a/tests/test_sphinx/sourcedirs/gettext/fr/LC_MESSAGES/index.po b/tests/test_sphinx/sourcedirs/gettext/fr/LC_MESSAGES/index.po
index 43e5320..c1a9bc7 100644
--- a/tests/test_sphinx/sourcedirs/gettext/fr/LC_MESSAGES/index.po
+++ b/tests/test_sphinx/sourcedirs/gettext/fr/LC_MESSAGES/index.po
@@ -125,3 +125,15 @@ msgid ".. image:: fun-fish.png\n"
 "   :alt: Fun Fish 3"
 msgstr ".. image:: poisson-amusant.png\n"
 "   :alt: Poisson amusant 3"
+
+msgid "[^1]"
+msgstr "[^1]"
+
+msgid "Text with footnote [^2]"
+msgstr "Text avec note [^2]"
+
+msgid "Définition de la note 1"
+msgstr ""
+
+msgid "Définition de la note 2"
+msgstr ""
diff --git a/tests/test_sphinx/sourcedirs/gettext/index.md b/tests/test_sphinx/sourcedirs/gettext/index.md
index a721944..d716e80 100644
--- a/tests/test_sphinx/sourcedirs/gettext/index.md
+++ b/tests/test_sphinx/sourcedirs/gettext/index.md
@@ -61,3 +61,11 @@ doctest block
 ```{figure} fun-fish.png
 :alt: Fun Fish 3
 ```
+
+- [^1]
+
+[^1]: Footnote definition 1
+
+- Text with footnote [^2]
+
+[^2]: Footnote definition 2
diff --git a/tests/test_sphinx/test_sphinx_builds/test_gettext.sphinx3.pot b/tests/test_sphinx/test_sphinx_builds/test_gettext.sphinx3.pot
index 5f76760..1e296ef 100644
--- a/tests/test_sphinx/test_sphinx_builds/test_gettext.sphinx3.pot
+++ b/tests/test_sphinx/test_sphinx_builds/test_gettext.sphinx3.pot
@@ -67,3 +67,19 @@ msgstr ""
 #: ../../index.md:55
 msgid "![Fun Fish 1](fun-fish.png)"
 msgstr ""
+
+#: ../../index.md:65
+msgid "[^1]"
+msgstr ""
+
+#: ../../index.md:69
+msgid "Text with footnote [^2]"
+msgstr ""
+
+#: ../../index.md:67
+msgid "Footnote definition 1"
+msgstr ""
+
+#: ../../index.md:71
+msgid "Footnote definition 2"
+msgstr ""
diff --git a/tests/test_sphinx/test_sphinx_builds/test_gettext.sphinx4.pot b/tests/test_sphinx/test_sphinx_builds/test_gettext.sphinx4.pot
index 933b8b6..b880e9c 100644
--- a/tests/test_sphinx/test_sphinx_builds/test_gettext.sphinx4.pot
+++ b/tests/test_sphinx/test_sphinx_builds/test_gettext.sphinx4.pot
@@ -79,3 +79,19 @@ msgstr ""
 #: ../../index.md:61
 msgid "Fun Fish 3"
 msgstr ""
+
+#: ../../index.md:65
+msgid "[^1]"
+msgstr ""
+
+#: ../../index.md:69
+msgid "Text with footnote [^2]"
+msgstr ""
+
+#: ../../index.md:67
+msgid "Footnote definition 1"
+msgstr ""
+
+#: ../../index.md:71
+msgid "Footnote definition 2"
+msgstr ""
diff --git a/tests/test_sphinx/test_sphinx_builds/test_gettext_additional_targets.sphinx3.pot b/tests/test_sphinx/test_sphinx_builds/test_gettext_additional_targets.sphinx3.pot
index 839d877..400edc2 100644
--- a/tests/test_sphinx/test_sphinx_builds/test_gettext_additional_targets.sphinx3.pot
+++ b/tests/test_sphinx/test_sphinx_builds/test_gettext_additional_targets.sphinx3.pot
@@ -124,3 +124,19 @@ msgstr ""
 msgid ".. image:: fun-fish.png\n"
 "   :alt: Fun Fish 3"
 msgstr ""
+
+#: ../../index.md:65
+msgid "[^1]"
+msgstr ""
+
+#: ../../index.md:69
+msgid "Text with footnote [^2]"
+msgstr ""
+
+#: ../../index.md:67
+msgid "Footnote definition 1"
+msgstr ""
+
+#: ../../index.md:71
+msgid "Footnote definition 2"
+msgstr ""
diff --git a/tests/test_sphinx/test_sphinx_builds/test_gettext_additional_targets.sphinx4.pot b/tests/test_sphinx/test_sphinx_builds/test_gettext_additional_targets.sphinx4.pot
index 11c51d4..0fca955 100644
--- a/tests/test_sphinx/test_sphinx_builds/test_gettext_additional_targets.sphinx4.pot
+++ b/tests/test_sphinx/test_sphinx_builds/test_gettext_additional_targets.sphinx4.pot
@@ -127,3 +127,19 @@ msgstr ""
 #: ../../index.md:61
 msgid "Fun Fish 3"
 msgstr ""
+
+#: ../../index.md:65
+msgid "[^1]"
+msgstr ""
+
+#: ../../index.md:69
+msgid "Text with footnote [^2]"
+msgstr ""
+
+#: ../../index.md:67
+msgid "Footnote definition 1"
+msgstr ""
+
+#: ../../index.md:71
+msgid "Footnote definition 2"
+msgstr ""