Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error when loading View LUTs depending on locale setting #379

Closed
Animtim opened this issue Sep 18, 2014 · 2 comments · Fixed by #1496
Closed

Error when loading View LUTs depending on locale setting #379

Animtim opened this issue Sep 18, 2014 · 2 comments · Fixed by #1496

Comments

@Animtim
Copy link

Animtim commented Sep 18, 2014

I found this bug in Krita when trying to load any profile for "View" return an error:
https://bugs.kde.org/show_bug.cgi?id=338357

After a long investigation with dmitryK, it appears to happen just because my system locale is French (and so probably because we use comma as decimal separator..).
If I run "LANG=en krita" (or change main system locale to english), the bug doesn't happen anymore.

Exactly, it appears that in FileFormatSpi1D.cpp (and most probably others) 'if(sscanf(lineBuffer, "From %f %f", &from_min, &from_max)!=2)' takes system locale into consideration and cannot load the files written in C locale (e.g. srgb lut table).
Looks like sscanf is to blame.

Please fix it when you can...
@dbr
Copy link
Contributor

dbr commented Sep 18, 2014

Yep, the parsing(/writing?) functions should be explicitly set the locale - there's an old ticket for this, #297

@sobotka
Copy link
Contributor

sobotka commented Mar 9, 2015

Possibly related with a solution?
https://groups.google.com/d/msg/academyaces/0_y9KxH-8Mw/Jma0wNgjryAJ

@johnhaddon johnhaddon mentioned this issue Mar 19, 2015
Closed
@doug-walker doug-walker mentioned this issue Apr 21, 2020
Merged
amyspark added a commit to amyspark/OpenColorIO that referenced this issue Sep 20, 2021
@amyspark
Use the fast_float library to parse floating-point numbers
f9b26eb 
This commit brings in the amalgamated header of the fast_float library,
and applies it to our uses of sscanf with %f.

This library is locale-free, and does not incur the performance penalty of
using a thread locale RAII class.

Fixes AcademySoftwareFoundation#297
Fixes AcademySoftwareFoundation#379
Fixes AcademySoftwareFoundation#1022
amyspark added a commit to amyspark/OpenColorIO that referenced this issue Sep 20, 2021
@amyspark
Use the fast_float library to parse floating-point numbers
40822e8 
This commit brings in the amalgamated header of the fast_float library,
and applies it to our uses of sscanf with %f.

This library is locale-free, and does not incur the performance penalty of
using a thread locale RAII class.

Fixes AcademySoftwareFoundation#297
Fixes AcademySoftwareFoundation#379
Fixes AcademySoftwareFoundation#1022

Signed-off-by: L. E. Segovia <13498015+amyspark@users.noreply.github.com>
@amyspark amyspark mentioned this issue Sep 20, 2021
Merged
amyspark added a commit to amyspark/OpenColorIO that referenced this issue Sep 20, 2021
@amyspark
Use the fast_float library to parse floating-point numbers
ee60b48 
This commit brings in the amalgamated header of the fast_float library,
and applies it to our uses of sscanf with %f.

This library is locale-free, and does not incur the performance penalty of
using a thread locale RAII class.

Fixes AcademySoftwareFoundation#297
Fixes AcademySoftwareFoundation#379
Fixes AcademySoftwareFoundation#1022

Signed-off-by: L. E. Segovia <13498015+amyspark@users.noreply.github.com>
amyspark added a commit to amyspark/OpenColorIO that referenced this issue Sep 20, 2021
@amyspark
Use the fast_float library to parse floating-point numbers
ae53620 
This commit brings in the amalgamated header of the fast_float library,
and applies it to our uses of sscanf with %f.

This library is locale-free, and does not incur the performance penalty of
using a thread locale RAII class.

Fixes AcademySoftwareFoundation#297
Fixes AcademySoftwareFoundation#379
Fixes AcademySoftwareFoundation#1322

Signed-off-by: L. E. Segovia <13498015+amyspark@users.noreply.github.com>
amyspark added a commit to amyspark/OpenColorIO that referenced this issue Oct 9, 2021
@amyspark
Use the fast_float library to parse floating-point numbers
4937d08 
This commit adds Daniel Lemire's fast_float library as an external dependency,
and applies it to our uses of sscanf with %f as well as istringstream.

This library is locale-free, and does not incur the performance penalty of
using a thread locale RAII class or imbuing a locale into a
stringstream.

See:
    Daniel Lemire, Number Parsing at a Gigabyte per Second, Software:
    Pratice and Experience 51 (8), 2021.
    <https://arxiv.org/abs/2101.11408>

Fixes AcademySoftwareFoundation#297
Fixes AcademySoftwareFoundation#379
Fixes AcademySoftwareFoundation#1322

Signed-off-by: L. E. Segovia <13498015+amyspark@users.noreply.github.com>
amyspark added a commit to amyspark/OpenColorIO that referenced this issue Oct 9, 2021
@amyspark
Use the fast_float library to parse floating-point numbers
6b6ce60 
This commit adds Daniel Lemire's fast_float library as an external dependency,
and applies it to our uses of sscanf with %f as well as istringstream.

This library is locale-free, and does not incur the performance penalty of
using a thread locale RAII class or imbuing a locale into a
stringstream.

See:
    Daniel Lemire, Number Parsing at a Gigabyte per Second, Software:
    Pratice and Experience 51 (8), 2021.
    <https://arxiv.org/abs/2101.11408>

Fixes AcademySoftwareFoundation#297
Fixes AcademySoftwareFoundation#379
Fixes AcademySoftwareFoundation#1322

Signed-off-by: L. E. Segovia <13498015+amyspark@users.noreply.github.com>
amyspark added a commit to amyspark/OpenColorIO that referenced this issue Nov 10, 2021
@amyspark
Implement locale-agnostic number parsing
76a23ac 
This commit adds Daniel Lemire's fast_float library as an external
dependency, and applies it to our uses of sscanf with %f, strtod,
and istringstream.

Additionally, for parsing integer numbers, we implement a from_chars
shim that forwards the call to strtol_l along with a statically
initialized locale constant.

The usage of fast_float is warded by a new OCIO_USE_FAST_FLOAT
configuration variable; if disabled, an identical approach to integers
is followed.

See:
    Daniel Lemire, Number Parsing at a Gigabyte per Second, Software:
    Pratice and Experience 51 (8), 2021.
    <https://arxiv.org/abs/2101.11408>

Fixes AcademySoftwareFoundation#297
Fixes AcademySoftwareFoundation#379
Fixes AcademySoftwareFoundation#1322

Co-Authored-By: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

Signed-off-by: L. E. Segovia <13498015+amyspark@users.noreply.github.com>
amyspark added a commit to amyspark/OpenColorIO that referenced this issue Nov 10, 2021
@amyspark
Implement locale-agnostic number parsing
92ad96e 
This commit adds Daniel Lemire's fast_float library as an external
dependency, and applies it to our uses of sscanf with %f, strtod,
and istringstream.

Additionally, for parsing integer numbers, we implement a from_chars
shim that forwards the call to strtol_l along with a statically
initialized locale constant.

The usage of fast_float is warded by a new OCIO_USE_FAST_FLOAT
configuration variable; if disabled, an identical approach to integers
is followed.

See:
    Daniel Lemire, Number Parsing at a Gigabyte per Second, Software:
    Pratice and Experience 51 (8), 2021.
    <https://arxiv.org/abs/2101.11408>

Fixes AcademySoftwareFoundation#297
Fixes AcademySoftwareFoundation#379
Fixes AcademySoftwareFoundation#1322

Co-Authored-By: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

Signed-off-by: L. E. Segovia <13498015+amyspark@users.noreply.github.com>
amyspark added a commit to amyspark/OpenColorIO that referenced this issue Nov 10, 2021
@amyspark
Implement locale-agnostic number parsing
c0eef4e 
This commit adds Daniel Lemire's fast_float library as an external
dependency, and applies it to our uses of sscanf with %f, strtod,
and istringstream.

Additionally, for parsing integer numbers, we implement a from_chars
shim that forwards the call to strtol_l along with a statically
initialized locale constant.

The usage of fast_float is warded by a new OCIO_USE_FAST_FLOAT
configuration variable; if disabled, an identical approach to integers
is followed.

See:
    Daniel Lemire, Number Parsing at a Gigabyte per Second, Software:
    Pratice and Experience 51 (8), 2021.
    <https://arxiv.org/abs/2101.11408>

Fixes AcademySoftwareFoundation#297
Fixes AcademySoftwareFoundation#379
Fixes AcademySoftwareFoundation#1322

Co-Authored-By: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

Signed-off-by: L. E. Segovia <13498015+amyspark@users.noreply.github.com>
amyspark added a commit to amyspark/OpenColorIO that referenced this issue Nov 10, 2021
@amyspark
Implement locale-agnostic number parsing
cb05eb7 
This commit adds Daniel Lemire's fast_float library as an external
dependency, and applies it to our uses of sscanf with %f, strtod,
and istringstream.

Additionally, for parsing integer numbers, we implement a from_chars
shim that forwards the call to strtol_l along with a statically
initialized locale constant.

The usage of fast_float is warded by a new OCIO_USE_FAST_FLOAT
configuration variable; if disabled, an identical approach to integers
is followed.

See:
    Daniel Lemire, Number Parsing at a Gigabyte per Second, Software:
    Pratice and Experience 51 (8), 2021.
    <https://arxiv.org/abs/2101.11408>

Fixes AcademySoftwareFoundation#297
Fixes AcademySoftwareFoundation#379
Fixes AcademySoftwareFoundation#1322

Co-Authored-By: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

Signed-off-by: L. E. Segovia <13498015+amyspark@users.noreply.github.com>
amyspark added a commit to amyspark/OpenColorIO that referenced this issue Nov 10, 2021
@amyspark
Use the fast_float library to parse floating-point numbers
a831b2c 
This commit adds Daniel Lemire's fast_float library as an external dependency,
and applies it to our uses of sscanf with %f as well as istringstream.

This library is locale-free, and does not incur the performance penalty of
using a thread locale RAII class or imbuing a locale into a
stringstream.

See:
    Daniel Lemire, Number Parsing at a Gigabyte per Second, Software:
    Pratice and Experience 51 (8), 2021.
    <https://arxiv.org/abs/2101.11408>

Fixes AcademySoftwareFoundation#297
Fixes AcademySoftwareFoundation#379
Fixes AcademySoftwareFoundation#1322

Signed-off-by: L. E. Segovia <13498015+amyspark@users.noreply.github.com>
amyspark added a commit to amyspark/OpenColorIO that referenced this issue Nov 11, 2021
@amyspark
Implement locale-agnostic number parsing
1e34906 
This commit adds Daniel Lemire's fast_float library as an external
dependency, and applies it to our uses of sscanf with %f, strtod,
and istringstream.

Additionally, for parsing integer numbers, we implement a from_chars
shim that forwards the call to strtol_l along with a statically
initialized locale constant. Unfortunately, to preserve consistency with
from_chars the character range must be copied to a temporary
std::string. The test suite has been adapted to account for this new
behaviour.

The usage of fast_float is warded by a new OCIO_USE_FAST_FLOAT
configuration variable; if disabled, an identical approach to integers
is followed.

See:
    Daniel Lemire, Number Parsing at a Gigabyte per Second, Software:
    Pratice and Experience 51 (8), 2021.
    <https://arxiv.org/abs/2101.11408>

Fixes AcademySoftwareFoundation#297
Fixes AcademySoftwareFoundation#379
Fixes AcademySoftwareFoundation#1322

Co-Authored-By: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

Signed-off-by: L. E. Segovia <13498015+amyspark@users.noreply.github.com>
amyspark added a commit to amyspark/OpenColorIO that referenced this issue Nov 24, 2021
@amyspark
Implement locale-agnostic number parsing
546e6b5 
This commit adds Daniel Lemire's fast_float library as an external
dependency, and applies it to our uses of sscanf with %f, strtod,
and istringstream.

Additionally, for parsing integer numbers, we implement a from_chars
shim that forwards the call to strtol_l along with a statically
initialized locale constant. Unfortunately, to preserve consistency with
from_chars the character range must be copied to a temporary
std::string. The test suite has been adapted to account for this new
behaviour.

The usage of fast_float is warded by a new OCIO_USE_FAST_FLOAT
configuration variable; if disabled, an identical approach to integers
is followed.

See:
    Daniel Lemire, Number Parsing at a Gigabyte per Second, Software:
    Pratice and Experience 51 (8), 2021.
    <https://arxiv.org/abs/2101.11408>

Fixes AcademySoftwareFoundation#297
Fixes AcademySoftwareFoundation#379
Fixes AcademySoftwareFoundation#1322

Co-Authored-By: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

Signed-off-by: L. E. Segovia <13498015+amyspark@users.noreply.github.com>
amyspark added a commit to amyspark/OpenColorIO that referenced this issue Dec 7, 2021
@amyspark
Implement locale-agnostic number parsing
5cc9446 
This commit adds support for parsing numbers without being influenced by
the current system locale. We implement a from_chars
shim that forwards the call to strto*_l along with a statically
initialized locale constant.

Fixes AcademySoftwareFoundation#297
Fixes AcademySoftwareFoundation#379
Fixes AcademySoftwareFoundation#1322

Co-Authored-By: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

Signed-off-by: L. E. Segovia <13498015+amyspark@users.noreply.github.com>
amyspark added a commit to amyspark/OpenColorIO that referenced this issue Dec 8, 2021
@amyspark
Implement locale-agnostic number parsing
2785410 
This commit adds support for parsing numbers without being influenced by
the current system locale. We implement a from_chars
shim that forwards the call to strto*_l along with a statically
initialized locale constant.

Fixes AcademySoftwareFoundation#297
Fixes AcademySoftwareFoundation#379
Fixes AcademySoftwareFoundation#1322

Co-Authored-By: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

Signed-off-by: L. E. Segovia <13498015+amyspark@users.noreply.github.com>
hodoulp pushed a commit that referenced this issue Dec 8, 2021
@amyspark
Implement locale-agnostic number parsing (#1496)
b2f927b 
This commit adds support for parsing numbers without being influenced by
the current system locale. We implement a from_chars
shim that forwards the call to strto*_l along with a statically
initialized locale constant.

Fixes #297
Fixes #379
Fixes #1322

Co-Authored-By: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

Signed-off-by: L. E. Segovia <13498015+amyspark@users.noreply.github.com>
hodoulp pushed a commit that referenced this issue Dec 8, 2021
@amyspark @hodoulp
Implement locale-agnostic number parsing (#1496)
769bf45 
This commit adds support for parsing numbers without being influenced by
the current system locale. We implement a from_chars
shim that forwards the call to strto*_l along with a statically
initialized locale constant.

Fixes #297
Fixes #379
Fixes #1322

Co-Authored-By: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

Signed-off-by: L. E. Segovia <13498015+amyspark@users.noreply.github.com>
@hodoulp hodoulp mentioned this issue Dec 8, 2021
Closed
hodoulp pushed a commit that referenced this issue Dec 9, 2021
@amyspark @hodoulp
Implement locale-agnostic number parsing (#1496)
084c3b4 
This commit adds support for parsing numbers without being influenced by
the current system locale. We implement a from_chars
shim that forwards the call to strto*_l along with a statically
initialized locale constant.

Fixes #297
Fixes #379
Fixes #1322

Co-Authored-By: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

Signed-off-by: amyspark <13498015+amyspark@users.noreply.github.com>
@hodoulp hodoulp mentioned this issue Dec 9, 2021
Merged
hodoulp added a commit that referenced this issue Dec 9, 2021
@hodoulp @amyspark
Implement locale-agnostic number parsing (#1496) (#1548)
6c8cf83 
This commit adds support for parsing numbers without being influenced by
the current system locale. We implement a from_chars
shim that forwards the call to strto*_l along with a statically
initialized locale constant.

Fixes #297
Fixes #379
Fixes #1322

Co-Authored-By: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

Signed-off-by: amyspark <13498015+amyspark@users.noreply.github.com>

Co-authored-by: amyspark <13498015+amyspark@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Improve the speed of parsing floating-point numbers amyspark/OpenColorIO
4 participants
@dbr @sobotka @Animtim @scoopxyz