Handle weird characters

[boogheta]

Apparently there are in parsed texts a lot of crazy characters :)

cat */procedure/*/texte/texte.json | awk '{for(i=1;i<=NF;i++)if("'!'"a[\$i]++)print \$i}' FS=\"\" | sort | grep -vi '[a-z0-9]' | sed 's/.*/\`&\`/'

`´`
```
`×`
`<`
`≤`
`=`
`>`
`°`
` `
`_`
`­`
`-`
`,`
`;`
`:`
`?`
`/`
`.`
`·`
`¸`
`'`
`‘`
`"`
`(`
`)`
`[`
`]`
`{`
`}`
`§`
`€`
`*`
`\`
`&`
`#`
`%`
`+`
`̀`
`́`
`ʺ`
`ˮ`
`​`
`□`
`…`
`‰`
`•`
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
``
`⅓`
`�`
`¼`
`²`
`³`
`º`
` `

[mdamien]

For all the , it's because of some Senate texts like this one:

https://www.lafabriquedelaloi.fr/articles.html?loi=pjl17-567&article=1er&etape=4

The text: https://www.senat.fr/leg/pjl17-631.html

[boogheta]

Comme on ne les voit pas bien, voilà une capture :)

[mdamien]

Got it, the AN is using a font named "Numero" to display the pastilles.

@font-face {
 font-family: "Numero";
 src: url('/monalisa/resources/fonts/pastilles.woff') format('woff');
}

This font, of course 😔, rely on crazy unicode characters.

Fix incoming !

[davidbgk]

😱 we're working hard on getting that info in a computable format but it's harder than expected, keep you posted…