Movegen speedup via magic bitboards

[ddobbelaere]

This PR adds magic bitboards (introduced in #628) as a layer on top of #638.

Unconditional speedup for random backend up to 10% (tested with LTO builds on Linux Fedora 29 64-bit, gcc 8.2.1).

The measurements details (command line and considered positions) are given in #638.

For 1 search thread:

position	master 6a639b6	#638 + this PR	speedup
start	207600 ± 447	216966 ± 414	4.5%
opening	161055 ± 188	171825 ± 221	6.7%
endgame	175728 ± 808	193532 ± 354	10%

For 2 search threads:

position	master 6a639b6	#638 + this PR	speedup
start	268477 ± 590	272720 ± 717	1.6%
opening	221515 ± 461	228676 ± 491	3.2%
endgame	235705 ± 632	245195 ± 655	4.0%

[ddobbelaere]

On my machine chessboard_test (built with b_lto=true) with same uncommented tests as #638 (comment):

current master (includes #638):

[ RUN      ] ChessBoard.MoveGenStartingPos
[       OK ] ChessBoard.MoveGenStartingPos (8084 ms)
[ RUN      ] ChessBoard.MoveGenKiwipete
[       OK ] ChessBoard.MoveGenKiwipete (14440 ms)
[ RUN      ] ChessBoard.MoveGenPosition3
[       OK ] ChessBoard.MoveGenPosition3 (1077 ms)
[ RUN      ] ChessBoard.MoveGenPosition4
[       OK ] ChessBoard.MoveGenPosition4 (1300 ms)
[ RUN      ] ChessBoard.MoveGenPosition5
[       OK ] ChessBoard.MoveGenPosition5 (6128 ms)
[ RUN      ] ChessBoard.MoveGenPosition6
[       OK ] ChessBoard.MoveGenPosition6 (262 ms)
...
[----------] 15 tests from ChessBoard (31292 ms total)

this PR (without PEXT):

[ RUN      ] ChessBoard.MoveGenStartingPos
[       OK ] ChessBoard.MoveGenStartingPos (7512 ms)
[ RUN      ] ChessBoard.MoveGenKiwipete
[       OK ] ChessBoard.MoveGenKiwipete (10888 ms)
[ RUN      ] ChessBoard.MoveGenPosition3
[       OK ] ChessBoard.MoveGenPosition3 (838 ms)
[ RUN      ] ChessBoard.MoveGenPosition4
[       OK ] ChessBoard.MoveGenPosition4 (972 ms)
[ RUN      ] ChessBoard.MoveGenPosition5
[       OK ] ChessBoard.MoveGenPosition5 (5320 ms)
[ RUN      ] ChessBoard.MoveGenPosition6
[       OK ] ChessBoard.MoveGenPosition6 (218 ms)
...
[----------] 15 tests from ChessBoard (25749 ms total)

This PR (with PEXT, built with pext=true):

[ RUN      ] ChessBoard.MoveGenStartingPos
[       OK ] ChessBoard.MoveGenStartingPos (7371 ms)
[ RUN      ] ChessBoard.MoveGenKiwipete
[       OK ] ChessBoard.MoveGenKiwipete (10748 ms)
[ RUN      ] ChessBoard.MoveGenPosition3
[       OK ] ChessBoard.MoveGenPosition3 (821 ms)
[ RUN      ] ChessBoard.MoveGenPosition4
[       OK ] ChessBoard.MoveGenPosition4 (960 ms)
[ RUN      ] ChessBoard.MoveGenPosition5
[       OK ] ChessBoard.MoveGenPosition5 (5225 ms)
[ RUN      ] ChessBoard.MoveGenPosition6
[       OK ] ChessBoard.MoveGenPosition6 (215 ms)
...
[----------] 15 tests from ChessBoard (25340 ms total)

[ddobbelaere]

This PR is ready for review.

[mooskagh]

Does using a singleton instead make it much slower?
E.g.

class MagicBitBoards {
 public:
   static const MagicBitBoards& Get() {
       static MagicBitBoards boards_;
       return boards;
   }
 private:
  MagicBitBoards() { ... }
};

(and removing static from everywhere)

It would be neater (I guess), but I can see that it could be slower indeed..

On other hand, unused static variable can be optimized out, so the constructor will never run.
(we had this problem with backend registration btw, which are also unused static variables,
after some massaging linker now doesn't optimize them, but it didn't before so it's fragile)

[mooskagh]

If it's not slower, then doing MagicBitBoards::Get() in main() so that time is not spent during actual game is fine.

[mooskagh]

If it is slower, then still calling some InitializeMagicBitboards() from main() seems to be more robust approach.

In that case, maybe it's better to move everything into anonymous namespace inside board.cc (not bitboard.cc, for me magicbitboards are more part of movegen than bitboard itself, it feels similar only because of name), and only expose one function InitializeMagicBitboards() in .h file. Also in this case the class wrapper (which only serves purpose of namespace) is not necessary I think.

[mooskagh]

Actually what I wrote in the last paragraph I like more than singleton even..
But I wonder what's your opinion.

[ddobbelaere]

I thought the current way of using a constructor was quite neat (I was a bit in Java mindset where static initialization is very easy and common, even without this constructor trick), but this fails miserably if the dummy instance gets optimized away.

I like your last suggestion to move everything into anonymous namespace of board.cc (dropping the class on top) and initialize explicitly. This has the added benefit that we have full control where to do the initialization (it will probably take only few milliseconds, but still...). I'll implement the changes for this last approach in the coming days.

[ddobbelaere]

I implemented the requested changes.

One additional benefit of the current approach is the fact that it's probably less sensitive to whether LTO is enabled, as all magic bitboard structures and code are now in the same compilation unit where they are used/called.