Fix gradients with ignore_idx in softmax_with_cross_entropy

[guoshengCS]

PR types

Bug fixes

PR changes

OPs

Describe

Fix gradient calculation with ignore_idx in softmax_with_cross_entropy.

Example code for reproducing error gradients on elements with ignore_idx:

import paddle
import numpy as np

paddle.seed(123)
np.random.seed(123)

class Net(paddle.nn.Layer):
    def __init__(self):
        super(Net, self).__init__()
        self.embedder = paddle.nn.Embedding(100, 64)
        self.linear = paddle.nn.Linear(64, 5)

    def forward(self, x, y):
        x = self.embedder(x)
        self.logits = logits = self.linear(x)
        print(y)
        loss = paddle.nn.functional.softmax_with_cross_entropy(logits, y, ignore_index=1)
        loss = paddle.mean(loss)
        return loss

x_data = np.random.randint(0, 5, (4,)).astype("int64")
x = paddle.to_tensor(x_data)
y_data = np.random.randint(0, 5, (4, 1))
y_data[0, 0] = 1  # ignore_idx
y = paddle.to_tensor(y_data)

net = Net()
loss = net(x, y)
loss.backward()
print(net.logits.grad)

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[paddle-bot-old]

✅ This PR's description meets the template requirements!
Please wait for other CI results.

[Xreki]

LGTM for the change of max_relative_error