PyTorch_NCF fix, normalize running loss

[hieuddo]

Description

Minor fix for PyTorch_NCF, nn.BCELoss(reduction="sum"), default is reduction="mean"

[tqtg]

@hieuddo any reason why we want to change to sum?

[hieuddo]

The current loss being printed is loss = running_loss / num_users. So the reduction of loss should be sum instead of mean.

cornac/cornac/models/ncf/recom_ncf_base.py

Line 244 in d300be2

count += len(batch_users)

[tqtg]

I see your point. Two perspectives:

1. We want to show what's the current loss value on the training dataset which is what the formula says if we're about to do batch learning, although it's not quite what the model is optimizing for when doing mini-batch.
2. Normalizing the loss by the size of mini-batch will help learning be more stable, less sensitive to the size of mini-batch. Think of them as doing SGD using the centroid of mini-batch.

I would suggest to keep the reduction="mean". If you're not comfortable with the current way of showing the loss value, we can change count to counting the number of mini-batches, and sum_loss to summing all the normalized mini-batch losses.

[hieuddo]

Understood. Just wanted to print the correct average loss over users.

One minor is current printed loss is relatively small (i.e., e-[3/4/5]), which could be hard to follow for hyperparams searching. This could be fixed by changing count += 1 I guess.

[tqtg]

Let's also synchronize this change to the tensorflow backend?