expose a function in Lua to generate identifiers/sort keys from UTF8 text

[massifrg]

I need a function similar to the one that computes auto identifiers.

I need it for Lua filters/writers.

It should:

• convert text to lowercase (this is already available in Lua thanks to pandoc.text

• remove accents from letters: "àéü" => "aeu"

• convert multiple non-alphanumeric chars to one space or "-", or "_": "foo (11/12), bar" => "foo-11-12-bar"

In particular, I need it to automatically generate sort keys for indices.

[tarleb]

I think that's a bit too specific and hence out of scope, especially the "remove accents" part.

See this StackOverflow Q&A for a hack to generate auto-identifiers from text.

[tarleb]

Addendum: you might like sluaggo, it was even written specifically with pandoc Lua filters in mind.

[massifrg]

Thanks @tarleb for the hints.

I also found issue #6415 that's related to what I'm working on.

There @jgm cites his unicode-collation library, which is used in pandoc for citeproc.

Though not strictly related to generating sort keys, a pandoc.text.collate(lang, text1, text2) in Lua may be useful in conjunction with pandoc.list:sort.

[jgm]

I'll note that we do have textToIdentifier in Text.Pandoc.Shared. In principle it could be wrapped and made available in the Lua API. Though I share @tarleb's sense that this might be a bit special-purpose for us to include in the API.

[tarleb]

@massifrg Having access to collate in Lua would certainly be helpful. Would you raise an issue for the collate feature request over at hslua? I'll look into it when I can, and it would find the way into pandoc from there.

[massifrg]

Would you raise an issue for the collate feature request over at hslua?

Thanks @tarleb.
I did it.