Skip to content

Commit

Permalink
Switch loss to mean (#67)
Browse files Browse the repository at this point in the history
Closes Should we use mean rather than sum for batchwise loss reduction? #63
  • Loading branch information
alan-cooney authored Nov 13, 2023
1 parent 292b615 commit 6a69e67
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion sparse_autoencoder/train/train_autoencoder.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ def train_autoencoder(
learned_activations_fired_count.add_(fired.sum(dim=0))

# Backwards pass
total_loss.sum().backward()
total_loss.mean().backward()

optimizer.step()

Expand Down

0 comments on commit 6a69e67

Please sign in to comment.