wrong set of tests registered with TEST_CASE_TEMPLATE get executed

[martinus]

Description

I have two separate cpp files. In one I have a test case like this:

TYPE_TO_STRING(robin_hood::unordered_flat_map<int, int>);
TYPE_TO_STRING(robin_hood::unordered_node_map<int, int>);

TEST_CASE_TEMPLATE("bench distinctness" * doctest::test_suite("bench"), Map,
                   robin_hood::unordered_flat_map<int, int>,
                   robin_hood::unordered_node_map<int, int>) {
...
}

In the other .cpp file I have this:

TYPE_TO_STRING(robin_hood::unordered_flat_map<int, int>);
TYPE_TO_STRING(robin_hood::unordered_node_map<int, int>);

TEST_CASE_TEMPLATE("copy and assign maps" * doctest::test_suite("unit"), Map,
                   robin_hood::unordered_flat_map<int, int>,
                   robin_hood::unordered_node_map<int, int>) {
...
}

It seems that somehow doctest does not like that. When I execute my binary with -ltc I get this output:

[doctest] listing all test case names
===============================================================================
copy and assign maps<robin_hood::unordered_flat_map<int, int>>
copy and assign maps<robin_hood::unordered_node_map<int, int>>
copy and assign maps<robin_hood::unordered_flat_map<int, int>>
copy and assign maps<robin_hood::unordered_node_map<int, int>>
===============================================================================
[doctest] unskipped test cases passing the current filters: 4

It seems I cannot execute the "bench distinctness" tests, they somehow are erroneously mapped to the "copy and assign maps" tests.

• doctest version: v2.3.1
• Operating System: Linux Mint 19.2
• Compiler+version: g++ 8.2.0

[martinus]

I have a minimal example:

In main.cpp:

#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN
#include <doctest.h>

in a.cpp:

#include <doctest.h>
#include <string>
TYPE_TO_STRING(std::string);
TEST_CASE_TEMPLATE("a", T, std::string) {}

in b.cpp:

#include <doctest.h>
#include <string>
TYPE_TO_STRING(std::string);
TEST_CASE_TEMPLATE("b", T, std::string) {}

Compiled everything, running ./app -ltc shows:

[doctest] listing all test case names
===============================================================================
a<std::string>
===============================================================================
[doctest] unskipped test cases passing the current filters: 1

So no mention of b<std::string>. Running ./app runs just one test:

[doctest] doctest version is "2.3.1"
[doctest] run with "--help" for options
===============================================================================
[doctest] test cases:      1 |      1 passed |      0 failed |      0 skipped
[doctest] assertions:      0 |      0 passed |      0 failed |
[doctest] Status: SUCCESS!

[onqtam]

I reproduced it... Thanks for reporting! I'll look into this ASAP

[onqtam]

This is due to an ODR violation... horrible and nasty bug - thanks for catching it!

The problem is that this constructor is implicitly inline and is thus a weak symbol so if there are multiple copies of it (with the same types for the template) in different source files with the same names then only 1 copy would be left (by random - or object file include order - AKA random) because of the linker removing all other duplicates.

Currently "unique" names are generated using __COUNTER__ (so they are unique within a given TU (translation unit)). If there is 1 normal test case above a templated one in one of the 2 files then all would be fine.

Alternatively I could add __LINE__ in addition to __COUNTER__ for unique names but even then your example wouldn't work because the 2 test cases are on the same lines in the 2 different TUs.

So I'm currently trying to figure out how to rewrite it so the symbols aren't weak.

[onqtam]

try it out now - should be fine - just pushed into the dev branch (will release an official version ... "soon"-ish)

[martinus]

Works great, thanks for fixing this so fast!