tzdata bad parse for Mexico April 1931, bad dow name

[spannella]

# Rule NAME FROM TO - IN ON AT SAVE LETTER/S Rule Mexico 1931 only - April 30 0:00 1:00 D

Seems to fail to parse April in northamerica which is causing tz.cpp:825 to throw:
std::runtime_error("oops: bad dow name: il")

[jmjatlanta]

Based on the other lines, I believe it was expecting Apr, but it received April, so the next parse3() call got il\t (the leftovers)

[HowardHinnant]

I've requested a fix to the IANA database: https://lists.iana.org/hyperkitty/list/tz@iana.org/thread/LBVCBNK5NPWBNQ5UDUY3YEZPWA2QV262/

[grownalchemist]

Any simple/quick fixes in the meantime?

[spannella]

Any simple/quick fixes in the meantime?

I just edited the northamerica file and changed April to Apr and it worked fine

[HowardHinnant]

That works.

Or you could grab 2024a from https://github.com/eggert/tz/tree/380c07cef01c71c1f93e9709d9f8c79b91cff063 and compile with AUTO_DOWNLOAD=0.

More details on this option here: https://howardhinnant.github.io/date/tz.html#Installation

[HowardHinnant]

The fix is on its way: eggert/tz@926b507

Not sure how long before release.

[HowardHinnant]

Java also took a hit: https://lists.iana.org/hyperkitty/list/tz@iana.org/thread/LBVCBNK5NPWBNQ5UDUY3YEZPWA2QV262/

[nielscastle]

Just to expand on @spannella earlier workaround post.

If you are automatically downloading updated versions of the tz data - and you probably are if you are seeing this error - the tz data files are downloaded to your users dowloads folder. E.g. if you are running on a macOS machine this would be the users Downloads folder ~/Downloads which is also visible from the desktop.

If you are running jobs via agents or daemons as root on macOS your downloads folder is located at /var/root/Downloads.

Locate the tzdata folder in the Downloads folder and edit the northamerica file. Around line 2634 you can change April to Apr.

Thank you for a great library ❤️

[grumpyquant]

Hi @HowardHinnant - the IANA thread you link to also indicates they think full month names are valid. To avoid this in the future should the parser be modified to permit these?

[HowardHinnant]

Possibly, but that's low priority for me. This library is obsoleted by C++20. I realize that implementations are still coming online and that this library will need to continue to exist for a couple more years.

IANA has no real motivation to use full month names. And doing so breaks more tools than just this one.

[spannella]

Just a proposed fix (didn't do up a pull request for it yet) but I am using this currently for myself:

static
std::string
parse_until_whitespace(std::istream& in)
{
    std::string result;
    in >> result;
    return result;
}

static
unsigned
parse_month(std::istream& in)
{
    CONSTDATA char* const month_names[] = {
        "Jan", "Feb", "Mar", "Apr", "May", "Jun",
        "Jul", "Aug", "Sep", "Oct", "Nov", "Dec",
        "January", "February", "March", "April", "May", "June",
        "July", "August", "September", "October", "November", "December"
    };
    std::string s = parse_until_whitespace(in);
    auto it = std::find(std::begin(month_names), std::end(month_names), s);
    auto index = it - std::begin(month_names);
    if (it == std::end(month_names))
        throw std::runtime_error("oops: bad month name: " + s);
    return static_cast<unsigned>((index % 12) + 1);
}

[HowardHinnant]

Thanks. I sincerely appreciate the suggestion.

A few constructive comments:

• parse_until_whitespace can be implemented with just in >> result;. I.e. the string extraction operator does this.
• We need to do the same for parse_dow.
• We need to do the same for keywords such as maximum, only, Rule and Zone.
• The parse should be case-insensitive.
• The matching prefix can be any non-zero length which creates an unambiguous match among the set of names that could exist within the parse context. E.g. "F" and "f" match February and so does any other prefix length for this month. But June and July require at least 3 letters to disambiguate.

In summary, to strictly follow their spec is a fair amount of work, and will likely add to the run-time expense of doing the parse (at least a little). Certainly doable. But I don't see the point of taking advantage of all this flexibility within the IANA database. It makes more sense for the single IANA db to follow a more strict syntax, and pass the savings in parsing complexity onto the several parsers.

I don't know this, but I would not be surprised that if they took full advantage of all this flexibility in their spec that they would break their own parser.

They just received another report of a broken parser: Azul ZUUpdater Tool.

[WinstonPrivacy]

spannella

Thank you, this works like a charm.

[eggert]

I would not be surprised that if they took full advantage of all this flexibility in their spec that they would break their own parser.

No, that parser won't break. For example, TZDB ships, and various distros use, a file tzdata.zi that contains lines like this:

Z America/Los_Angeles -7:52:58 - LMT 1883 N 18 20u

where Z is short for Zone and N is short for November. I'm copying this line from my Ubuntu 24.04.1 desktop's /usr/share/zoneinfo/tzdata.zi file, and zic works just fine with this file.

Every zic-compatible parser should process abbreviations correctly, so that it can handle tzdata.zi among other things.

[eggert]

A TZDB user reports, "The library https://github.com/HowardHinnant/date will check for latest release and it auto downloads the latest version. With latest version and scheduler is broken." He write that there is no option to stick with TZDB release 2024a.

I don't know whether the breakage is a problem with the TZDB users's configuration or with date itself. Although I suspect the former thought I should mention the issue here. Obviously an entire app shouldn't stop working merely because date can't parse "April".

[ironman198]

A TZDB user reports, "The library https://github.com/HowardHinnant/date will check for latest release and it auto downloads the latest version. With latest version and scheduler is broken." He write that there is no option to stick with TZDB release 2024a.

I don't know whether the breakage is a problem with the TZDB users's configuration or with date itself. Although I suspect the former thought I should mention the issue here. Obviously an entire app shouldn't stop working merely because date can't parse "April".

To be clear, we went with default setting of auto download and released into our agents. After this issue, we realized there is a option to turn off auto download tz db files , we are releasing a patch with it. But to recompile it and release it to all the agents is not a quick solution and it requires an app upgrade in our case for all the customers. We rely on date lib for timezone conversions, as date lib is unable to load tzdb files the scheduler functionality is broken.

[HowardHinnant]

Thank you both @eggert and @ironman198 for the further information.

I would also like to make a further clarification. This won't directly help either of you as I know you both already know this. Rather this clarification is for the wider audience reading this thread:

• A fix to this parser would not get a fix to @ironman198's clients any faster than his changing the auto-download configuration flag and redistributing. Both of these options require an app upgrade for him.
• A released fix to the IANA tzdb will not only immediately fix the problems of clients of this parser, but also immediately fix a growing number of reported problems with other parsers across a variety of platforms and languages.

I also note that this library is obsoleted by the MSVC implementation of C++20 chrono, and the gcc implementation of C++20 chrono. An LLVM implementation of C++20 chrono is still under development. I also know that the gcc implementation is not impacted by the 2024b release from IANA. I'm unsure if the MSVC implementation is impacted.

Also tangentially on this subject, here is an article showing how an application can use this library to control if and when a new IANA tzdb is downloaded, and even contend with multiple databases in a concurrent environment. The article was motivated by long running applications which can't reboot but want to take advantage of newly released IANA tzdbs. An airline reservation system might be an example of an application needing this functionality.

[WinstonPrivacy]

Speaking from an end-user's perspective, it appears that there is a lot of complex thought going into this but I'm not sure it's justified when a very simple patch resolves the current problem and greatly ameliorates the issue moving forward (especially given that IANA says you should support full month names).

I'm just one user but this issue forced me to refactor this library out of all production apps. It remains only in a single non-production application at this time.