Skip to content

Commit

Permalink
Deepspeed example should use gather_for_metrics (#821)
Browse files Browse the repository at this point in the history
* Deepspeed example should use gather_for_metrics

I believe this example should be using gather_for_metrics here instead of gather.

* Update deepspeed_with_config_support.py
  • Loading branch information
HammadB authored Nov 10, 2022
1 parent f9317f2 commit ccdc225
Showing 1 changed file with 1 addition and 2 deletions.
3 changes: 1 addition & 2 deletions examples/by_feature/deepspeed_with_config_support.py
Original file line number Diff line number Diff line change
Expand Up @@ -285,10 +285,9 @@ def evaluate(args, model, eval_dataloader, accelerator, eval_dataset):
outputs = model(**batch)

loss = outputs.loss
losses.append(accelerator.gather(loss.repeat(args.per_device_eval_batch_size)))
losses.append(accelerator.gather_for_metrics(loss.repeat(args.per_device_eval_batch_size)))

losses = torch.cat(losses)
losses = losses[: len(eval_dataset)]
try:
eval_loss = torch.mean(losses)
perplexity = math.exp(eval_loss)
Expand Down

0 comments on commit ccdc225

Please sign in to comment.