In da56c93 we introduced an optimization to avoid going through all the
layers every time we need the index of a specific one.

However, when we set the layer index, we weren't setting the reversed
index, like we were doing before the optimization.

Fix it by setting the correct index.