Display which tests are running during force:source:deploy

[jhgihub0]

With @salesforce/cli/2.73.9 or earlier, there does not seem to be a way to see which tests are running during a deployment.

For example, If I run:

sfdx force:source:deploy --target-org OrgAlias --wait 120 --testlevel 'RunLocalTests' --sourcepath ./force-app

It will deploy the tests output like this:

...

12:15:01     Successful: 468/497 (94%)
12:15:02     Successful: 471/497 (95%)
12:15:09     Successful: 472/497 (95%)
12:15:14     Successful: 473/497 (95%)
12:15:20     Successful: 475/497 (96%)
12:15:27     Successful: 476/497 (96%)

...

I would like to see which test is running, not just which number. A log when each test starts and stops would be great as well, if possible.

We have long running tests and we are having a hard time finding which ones are the problems.

Thank you.

[cristiand391]

force:source:deploy has been gone since sf v2.65.8:
https://github.com/forcedotcom/cli/tree/main/releasenotes/#2658-november-6-2024

have you tried sf project deploy start --verbose? it shows test failures/successes.

[VivekMChawla]

@jhgihub0 - Thanks for your feature request!

As mentioned by @cristiand391, adding the --verbose flag will result in a closing summary that includes details about the test failed/passed test methods that were run.

Result of deploy with --verbose

Getting test execution time from a --verbose --json run

The core of your request is knowing the runtime of each test method so you can identify long-running ones. This is currently possible by inspecting the time value of the success and failure objects in the runTestResult object from the --json output of a deploy --verbose command run. For example:

"successes": [
          {
            "id": "01pVW0000007i1bYAA",
            "methodName": "getExperienceSessionsForDate_works",
            "name": "ExperienceControllerTest",
            "namespace": null,
            "time": 129
          },
          {
            "id": "01pVW0000007i1gYAA",
            "methodName": "sampleBookingBatch_works",
            "name": "SampleDataTest",
            "namespace": null,
            "time": 695
          },
          {
            "id": "01pVW0000007i1jYAA",
            "methodName": "getResortWeather_failsWithError",
            "name": "WeatherServiceTest",
            "namespace": null,
            "time": 84
          },
          {
            "id": "01pVW0000007i1jYAA",
            "methodName": "getResortWeather_works",
            "name": "WeatherServiceTest",
            "namespace": null,
            "time": 83
          }
        ],

Sharing test execution time in HRO

Adding the runtime for each test method seems like a great addition to the HRO (human readable output) for a deploy --verbose command. Here's what that would look like.

Test Success [4]
✓ ExperienceControllerTest.getExperienceSessionsForDate_works (129ms)
✓ WeatherServiceTest.getResortWeather_failsWithError (84ms)
✓ WeatherServiceTest.getResortWeather_works (83ms)
✓ SampleDataTest.sampleBookingBatch_works (695ms)

@jhgihub0 - Let me know if this change would address your core needs.

@soridalac - I created W-17674281 for this feature request.

[jhgihub0]

Sorry, yes we do use "sf project deploy start" with --verbose

We have a large jenkins framework that runs over 30+ Salesforce projects, and the project teams use Jenkins and the regular logs to view and monitor the jobs during deployments. If I change it over to --json, then Jenkins wont show anything until the runs are done, and will cause mass confusion.

Having an option to display the information in the regular logs will be great if all possible in future releases.

Thank you

[jhgihub0]

Great. Does show by default or do I have to pass a flag?

[cristiand391]

pass --verbose.
Note that we only added the total exec time in ms like this:

Test Success [4]
✓ ExperienceControllerTest.getExperienceSessionsForDate_works (129ms)
✓ WeatherServiceTest.getResortWeather_failsWithError (84ms)
✓ WeatherServiceTest.getResortWeather_works (83ms)
✓ SampleDataTest.sampleBookingBatch_works (695ms)

the API doesn't gives us a full timestamp so we can show the total exec time per test.

[cristiand391]

nvm, I forgot you were talking about CI usage.
We made the reporter print test failures as they are retrieved (CI only), see the after video here:
salesforcecli/plugin-deploy-retrieve#1215

[jhgihub0]

@cristiand391 was this implemented with 2.78.0?

I just updated to 2.78.3 and I dont see any difference when running

sf project deploy start --target-org OrgAlias --wait 120 --test-level "RunLocalTests" --source-dir ./force-ap --dry-run --verbose

[jhgihub0]

@cristiand391 @VivekMChawla please confirm which version of sf this was merged into