Remove addEndJob logic/customization in cmsDriver

[dmitrijus]

This removes the obsolete "addEndJob = True" logic in the cmsDriverOptions.py

It should not be necessary - most central wolkflows had this flag disabled.
This also fixes "extensive" logging in a few of runTheMatrix tests, where a legacy endJob module was added, even though it was not needed to be added.

Jobs which still need the endJob, should explicitly pass "ENDJOB" step to the cmsDriver.

[dmitrijus]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/18247/console Started: 2017/03/08 17:39
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/18249/console Started: 2017/03/08 17:50

[cmsbuild]

A new Pull Request was created by @dmitrijus (Dmitrijus) for master.

It involves the following packages:

Configuration/Applications

@cmsbuild, @franzoni, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @Martin-Grunewald this is something you requested to watch as well.
@Muzaffar, @davidlange6, @smuzaffar you are the release manager for this.

cms-bot commands are listed here

[cmsbuild]

-1

Tested at: f4056af

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
9a3af71
a668435
1cdb34b
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17827/18249/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17827/18249/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17827/18249/summary.html

I found follow errors while testing this PR

Failed tests: RelVals

• RelVals:

When I ran the RelVals I found an error in the following worklfows:
101.0 step1

runTheMatrix-results/101.0_SingleElectronE120EHCAL+SingleElectronE120EHCAL/step1_SingleElectronE120EHCAL+SingleElectronE120EHCAL.log

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
9a3af71
a668435
1cdb34b
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17827/18249/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17827/18249/git-merge-result

[cmsbuild]

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

[dmitrijus]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/18276/console Started: 2017/03/09 12:24

[cmsbuild]

Pull request #17827 was updated. @perrotta, @civanch, @mdhildreth, @dmitrijus, @cmsbuild, @franzoni, @slava77, @vanbesien, @davidlange6 can you please check and sign again.

[slava77]

It's getting close to a year now since the original submission.
Is this PR still needed?
It sounds like "not really".

[dmitrijus]

If anyone needs it, I will reopen it.

[dmitrijus]

please test workflow 1001.0,1004.0

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/26220/console Started: 2018/02/21 20:30

[cmsbuild]

+1
Tested at: a4b7c4e
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17827/26220/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17827/26220/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-17827/1004.0_RunHI2011+RunHI2011+TIER0EXPHI+ALCAEXPHI+ALCAHARVD1HI+ALCAHARVD2HI+ALCAHARVD3HI+ALCAHARVD5HI

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 2 differences found in the comparisons
• DQMHistoTests: Total files compared: 28
• DQMHistoTests: Total histograms compared: 2498056
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2497879
• DQMHistoTests: Total skipped: 176
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 1.06000000002 KiB( 23 files compared)
• Checked 115 log files, 9 edm output root files, 28 DQM output files