HTTP status code is wrong when deleting non-exist scheduler

[AstroProfundis]

Please answer these questions before submitting your issue. Thanks!

1. What did you do?

Say there is a store leader evicting scheduler for store 6, and I want to delete that scheduler to allow leaders to be transferred back to it.

The 1st attempt:

$ curl -v -XDELETE "http://172.16.5.218:2379/pd/api/v1/schedulers/evict-leader-scheduler-6"
* About to connect() to 172.16.5.218 port 2379 (#0)
*   Trying 172.16.5.218...
* Connected to 172.16.5.218 (172.16.5.218) port 2379 (#0)
> DELETE /pd/api/v1/schedulers/evict-leader-scheduler-6 HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 172.16.5.218:2379
> Accept: */*
> 
< HTTP/1.1 200 OK
< Access-Control-Allow-Headers: accept, content-type, authorization
< Access-Control-Allow-Methods: POST, GET, OPTIONS, PUT, DELETE
< Access-Control-Allow-Origin: *
< Content-Type: application/json; charset=UTF-8
< Date: Thu, 02 Apr 2020 09:49:48 GMT
< Content-Length: 5
< 
null
* Connection #0 to host 172.16.5.218 left intact

All looks good.

But if I repeat that request again:

$ curl -v -XDELETE "http://172.16.5.218:2379/pd/api/v1/schedulers/evict-leader-scheduler-6"
* About to connect() to 172.16.5.218 port 2379 (#0)
*   Trying 172.16.5.218...
* Connected to 172.16.5.218 (172.16.5.218) port 2379 (#0)
> DELETE /pd/api/v1/schedulers/evict-leader-scheduler-6 HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 172.16.5.218:2379
> Accept: */*
> 
< HTTP/1.1 500 Internal Server Error
< Access-Control-Allow-Headers: accept, content-type, authorization
< Access-Control-Allow-Methods: POST, GET, OPTIONS, PUT, DELETE
< Access-Control-Allow-Origin: *
< Content-Type: application/json; charset=UTF-8
< Date: Thu, 02 Apr 2020 09:50:25 GMT
< Content-Length: 22
< 
"scheduler not found"
* Connection #0 to host 172.16.5.218 left intact

The status code is 500 Internal Server Error, which is not correct for this circumstance.

2. What did you expect to see?

Return 404 Not Found or 410 Gone would be the proper response, as we can see the body says scheduler not found, this is clearly supposed to be 404 Not Found.

3. What did you see instead?

It responses a 500 Internal Server Error status code with scheduler not found in the body.

4. What version of PD are you using (pd-server -V)?

Release Version: v4.0.0-beta.2
Git Commit Hash: 7ebad5232e7df681d8d425900f52984574539568
Git Branch: HEAD
UTC Build Time:  2020-03-18 01:25:40

[july2993]

https://github.com/pingcap/pd/blob/878ac7e830af195912603501d5d4de8ccffff232/server/api/scheduler.go#L263

PD will request another API internally
and seems it will always return "scheduler not found"

so, when returning "scheduler not found", the user can't make sure it's truly "scheduler not found"