Deleting unit test is forbidden (English Version)

Deleting unit test is forbidden (English Version)

---

不允许禁用单测（中文版本）

---

Summary:

• Section 1: Instructions
• Section 2: The inspection method of this specification
• Section 3: Methods if inspection is failed

supplementary instruction:

The unit-test in Temporarily disabled UnitTest are needed to confirm the reasons for the failure, futherly. For those that can be solved by modifying the OP code, corresponding modifications should be made.

Instructions

• Some op have bugs, which are easy to fail in unit test. For various reasons, the bug has not been fixed but just removed the unit test, which is not allowed。

• Requirements: the number of unit tests is not allowed to decrease, only to increase.

• Therefore, in order to protect the existing unit tests and gradually improve the coverage of unit tests, this Specification is proposed, and the corresponding detection mechanism is set on CI.

The inspection method of this specification

1. PR_CI_Coverage MAC_PR_CI_python35 wilee checks approval firstly, and if there is an approval from the specified person (the corresponding approver will be prompted in CI), the check will be automatically exempted;

2. If there is no approval, CI begins to check the current PR: cmake, build, and run unit-tests in order;

3. Gets names of all unit tests for the current PR;

4. Gets names of all unit tests for develop branch in PaddlePaddle repository;

5. Compare unit test of the two versions. Intercept if the number of unit tests decreases;

6. Only unit test that fail to run (see step 2) can be added to Temporarily disabled UnitTest, and you should provide sufficient reason;

7. If one unit test is added to Temporarily disabled UnitTest and get approval from specified person, please do not resubmit the code again, just rerun CI directly to trigger the exemption check。

Methods if inspection is failed

This specification is currently enabled in PR_CI_Coverage MAC_PR_CI_python35. Existing diabled unit-tests have been added to Temporarily disabled UnitTest. Disabling the unit test will fail to run CI, and these message will appear in build_log:

************************************
It is forbidden to disable or delete the unit-test.

If you must delete it temporarily, please add it to [https://github.com/PaddlePaddle/Paddle/wiki/Temporarily-disabled-Unit-Test]
Then you must have one RD (kolinwei(recommended) or zhouwei25 or luotao1) approval for the deletion of unit-test. 

If you have any problems about deleting unit-test, please read the specification [https://github.com/PaddlePaddle/Paddle/wiki/Deleting-unit-test-is-forbidden]. 

Following unit-tests are deleted in this PR:
test_1
test_2
test_3
...
test_n

************************************

• The corresponding solutions are as follows:

1. In principle, unit-test is not allowed to be disabled. Please modify the OP code by yourself until the unit test passes;

2. For those can't be repaired in short term and must be disabled, please be sure to add it to Temporarily disabled UnitTest firstly.

3. Ask for the approval of the specified person (the corresponding approver will be prompted in CI);

4. After getting approval, please do not resubmit the code again, just re-run CI, CI will be exempted to check this unit-test;

5. Please as soon as possible to repair this unit-test, and remove it from Temporarily disabled UnitTest.

===================================

current pr ${GIT_PR_ID} has got approvals. So, Pass CI directly!

===================================

• Passed！

If you have any other questions, please contact us @zhouwei25。