Skip to content

Commit

Permalink
[YAML] Fix glitch in formatDouble serialization
Browse files Browse the repository at this point in the history
Previous implementation assumed number format with four letter exponent
blocks; this fix ensures that formatting is correct for very small/large
numbers that use five letter exponent blocks (e.g. 'e+100')
  • Loading branch information
ischoegl committed Dec 2, 2021
1 parent 1b46a24 commit 1905f96
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 1 deletion.
10 changes: 9 additions & 1 deletion src/base/AnyMap.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,15 @@ string formatDouble(double x, long int precision)
string s0;
if (useExp) {
s0 = fmt::format(fmt::format("{:.{}e}", x, precision));
last = s0.size() - 5; // last digit of significand
// last digit of significand
last = s0.size() - 5;
if (s0[last + 1] == 'e') {
// pass - most values use four letter exponent (examples: e+05, e-03)
} else if (s0[last + 2] == 'e') {
last++; // exponents larger than e+99 or smaller than e-99
} else {
last = s0.find('e') - 1; // backstop; slower, but will always work
}
} else {
log10x = static_cast<int>(std::floor(std::log10(std::abs(x))));
s0 = fmt::format("{:.{}f}", x, precision - log10x);
Expand Down
3 changes: 3 additions & 0 deletions test/general/test_serialization.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@ TEST(YamlWriter, formatDouble)
// check edge cases
m["a"] = -1061.793215682400;
EXPECT_EQ(m.toYamlString(), "a: -1061.7932156824\n");

m["a"] = 7.820059054328200e-111;
EXPECT_EQ(m.toYamlString(), "a: 7.8200590543282e-111\n");
}

TEST(YamlWriter, formatDoubleExp)
Expand Down

0 comments on commit 1905f96

Please sign in to comment.