Numbers in templates are still getting localized

[Pomax]

This is caused by L10N=True: even though we removed USE_THOUSAND_SEPARATOR, which turns this behaviour off for English/default, the L10N setting "overrules" that and turns on silent number reformatting with even less control. There is the {{ value | unlocalize }} template filter from the l10n tags that we can use, but it would be fantastic if we can find out how to actually turns this off for every locale, except where intcomma is explicitly used.

[TheoChevalier]

@Pomax I’m curious where you see this happening?

[TheoChevalier]

Ah, the data-creepiness attribute, just caught up with the PNI PR.

[Pomax]

yup. Apparently we missed this part in https://docs.djangoproject.com/en/3.2/ref/settings/#use-thousand-separator:

These settings may also be dictated by the locale, which takes precedence.

Without any clues as to how it does that, and how you stop it (that I can find) beyond "you don't, you have to use that template tag".

[benhohner]

Need to do audit to determine where it's being used across locales

[richbrennan]

Hey @Pomax

I've audited the templates by searching for instances of data-* or <script> in order to identify variables that may need the unlocalize filter:

mozfest_footer.html

• data-signup-id="{{ mozfest_footer.id }}"

signup.html

• data-signup-id="{{ page.signup.id }}"

buyersguide_item.html - ALREADY ADDED FOR PNI

• data-creepiness="{{ product.creepiness }}">

bannered_campaign_page.html

• data-signup-id="{{ page.signup.id }}"

index_page.html

• data-page-size="{{ page.specific.page_size }}"

profile_blocks.html

• data-program-year="{{ value.program_year }}" - APPLY DATE FILTER?

pulse_project_list.html

• data-max="{{ value.size }}"

petition.html

• data-petition-id="{{ cta.id }}"

load-more-regrets.html

• {{ page.specific.batch_size }}
• {{ page.specific.max_batch_size }}

Let me know if you'd like me to add the unlocalize filter to any of the above fields.

I also reviewed the Django docs but couldn't see a documented way to switch the template system so it doesn't assume that numerical values in templates are intended for human consumption and therefore have L10n applied.

[Pomax]

this would be good to revisit

[mtdenton]

@cdanfon Let's move this up the backlog too and see if it's still relevant, might be a fairly easy win.

[tbrlpld]

yup. Apparently we missed this part in https://docs.djangoproject.com/en/3.2/ref/settings/#use-thousand-separator:

These settings may also be dictated by the locale, which takes precedence.

Without any clues as to how it does that, and how you stop it (that I can find) beyond "you don't, you have to use that template tag".

This is the full documentation of the USE_THOUSAND_SEPARATOR setting:

USE_THOUSAND_SEPARATOR¶

Default: False

A boolean that specifies whether to display numbers using a thousand separator. When set to True and USE_L10N is also True, Django will format numbers using the NUMBER_GROUPING and THOUSAND_SEPARATOR settings. These settings may also be dictated by the locale, which takes precedence.

So it's a bit ambiguous, but it seems to me like the locale override would refer to NUMBER_GROUPING and THOUSAND_SEPARATOR.

Is this a bug in Django? From the above it sounds like USE_THOUSAND_SEPARATOR = False should prevent the localization of the numbers... unless USE_THOUSAND_SEPARATOR is included in [t]hese settings may also be dictated by the locale, which takes precedence... which feels counter intuitive.

[tbrlpld]

I have filed https://code.djangoproject.com/ticket/34064 to clarify the docs at least.

[tbrlpld]

I guess the issue is that number localization is not only the thousand separator, but also the decimal separator. This means turning off the thousand separator is not sufficient to get unlocalized numbers. Actually, thousand separators are more of an accounting way of writing numbers, while the decimal separator is more general.

[tbrlpld]

Isn't USE_L10N the setting we want to deactivate? Unless other logic is referring to this setting, from the docs it sounds like this is what controls the rendering of data:

USE_L10N¶

Default: False

A boolean that specifies if localized formatting of data will be enabled by default or not. If this is set to True, e.g. Django will display numbers and dates using the format of the current locale.

-- https://docs.djangoproject.com/en/3.2/ref/settings/#use-l10n

So turning this off should prevent the automatic conversion in the templates.

Unfortunately though, this is deprecated starting with Django 4.0: https://docs.djangoproject.com/en/4.0/ref/settings/#use-l10n

So I guess we will need to go the manual route.

[tbrlpld]

See also: https://docs.djangoproject.com/en/4.1/topics/i18n/formatting/#controlling-localization-in-templates

[tbrlpld]

I have updated the locations identified above and also searched for id, count, size, num, width and height in the template.

For the future we need to pay attention to this in general, but it would also be good to include the original locations where the issue was identified in the ticket.

[tbrlpld]

@cdanfon Similarly tricky testing as #6255. I guess here it's even harder because a bunch of the changes affect potentially many pages. So a general functionality check of the site should do it. The only things I can really think of for testing is the newsletter signup. 😅 Sorry about this.

[cdanfon]

Hey @tbrlpld I've tested the newsletter signup in https://foundation.mofostaging.net/en both in the header and footer with different emails and I don't think it's working as I haven't received any email confirmation, unless the SMTP is not configured to work on staging?

[tbrlpld]

@cdanfon I actually don't really know how this works, but I don't think we are sending the emails directly. I think the external newsletter service is handling the sending of the emails. But it is a good question if everything is configured correctly on staging.

[tbrlpld]

Ok, so it looks like we are using this client to use the Mozilla newsletter service "Basket".

From the config on staging, I would guess this is not configured to be working as it is not using the production settings for BASKET_... settings.

[tbrlpld]

@cdanfon Nevermind my comment then about testing the newsletter signup. Seems like this should not be working and I don't know if it worked before. Guess we will just have to wait and see in prod 🤷‍♂️

[cdanfon]

Makes sense @tbrlpld Can you give me a nudge when this gets to production to check please? Thanks

[tbrlpld]

@cdanfon This has been on prod since monday. I was not aware there was a prod deployment on monday.

[cdanfon]

Hey @tbrlpld tested signing up to the newsletter on prod and all worked fine