fix tracker.xml #2253

boudoul · 2014-01-30T23:56:48Z

tested runTheMatrix.py --what upgrade -l 4500
with 100 events, before (red) and after (black) on the #hit/eta

Fix was discussed in the PR 2232

hope I didn't screw up this PR as the status is " Error checking merge status" but don't worry it said ;-)

cmsbuild · 2014-01-30T23:57:46Z

A new Pull Request was created by @boudoul (boudoul) for CMSSW_6_2_X_SLHC.

fix tracker.xml

It involves the following packages:

Geometry/TrackerCommonData

@civanch, @Dr15Jones, @ianna, @mdhildreth, @cmsbuild, @nclopezo, @Degano, @ktf can you please review it and eventually sign? Thanks.
@ghellwig this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
@andersonjacob, @mark-grimes you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

ianna · 2014-01-31T06:53:53Z

+1

cmsbuild · 2014-01-31T06:57:45Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_6_2_X_SLHC IBs unless changes or unless it breaks tests. @andersonjacob, @mark-grimes can you please take care of it?

mark-grimes · 2014-01-31T16:49:40Z

Tests 3300, 3400, 4100, 4400, 40001, 50002, 60002, 4502, 4500, 5001 and 15001 pass okay, as well as 3400 with combinedCustoms.cust_2019WithGem. However, tests 50001 (ttbar FastSim ExtendedPhase2TkBE) and 60001 (ttbar FastSim ExtendedPhase2TkBEForward) seg fault with the same stack trace as #2232:

#5  0x00002aae35863c29 in CloseComponentsMerger<5u>::merge(MultiGaussianState<5u> const&) const () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol1/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-01-31-0200/lib/slc5_amd64_gcc472/pluginTrackingToolsGsfToolsPlugins.so
#6  0x00002aae3be543fd in MultiTrajectoryStateMerger::merge(TrajectoryStateOnSurface const&) const () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol1/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-01-31-0200/lib/slc5_amd64_gcc472/libTrackingToolsGsfTracking.so
#7  0x00002aae3be5257f in GsfTrajectorySmoother::trajectory(Trajectory const&) const () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol1/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-01-31-0200/lib/slc5_amd64_gcc472/libTrackingToolsGsfTracking.so
#8  0x00002aae35742f42 in KFFittingSmoother::fitOne(TrajectorySeed const&, std::vector<ConstReferenceCountingPointer<TransientTrackingRecHit>, std::allocator<ConstReferenceCountingPointer<TransientTrackingRecHit> > > const&, TrajectoryStateOnSurface const&, TrajectoryFitter::fitType) const () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol1/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-01-31-0200/lib/slc5_amd64_gcc472/libTrackingToolsTrackFitters.so
#9  0x00002aae5e9985f7 in TrackProducerAlgorithm<reco::GsfTrack>::buildTrack(TrajectoryFitter const*, Propagator const*, std::vector<std::pair<Trajectory*, std::pair<reco::GsfTrack*, PropagationDirection> >, std::allocator<std::pair<Trajectory*, std::pair<reco::GsfTrack*, PropagationDirection> > > >&, std::vector<ConstReferenceCountingPointer<TransientTrackingRecHit>, std::allocator<ConstReferenceCountingPointer<TransientTrackingRecHit> > >&, TrajectoryStateOnSurface&, TrajectorySeed const&, float, reco::BeamSpot const&, edm::RefToBase<TrajectorySeed>, int, signed char) () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol1/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-01-31-0200/lib/slc5_amd64_gcc472/libRecoTrackerTrackProducer.so
#10 0x00002aae5e91b612 in TrackProducerAlgorithm<reco::GsfTrack>::runWithCandidate(TrackingGeometry const*, MagneticField const*, std::vector<TrackCandidate, std::allocator<TrackCandidate> > const&, TrajectoryFitter const*, Propagator const*, TransientTrackingRecHitBuilder const*, reco::BeamSpot const&, std::vector<std::pair<Trajectory*, std::pair<reco::GsfTrack*, PropagationDirection> >, std::allocator<std::pair<Trajectory*, std::pair<reco::GsfTrack*, PropagationDirection> > > >&) () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol1/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-01-31-0200/lib/slc5_amd64_gcc472/pluginRecoTrackerTrackProducerPlugins.so
#11 0x00002aae5e919d45 in GsfTrackProducer::produce(edm::Event&, edm::EventSetup const&) () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol1/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-01-31-0200/lib/slc5_amd64_gcc472/pluginRecoTrackerTrackProducerPlugins.so

cmsbuild · 2014-02-03T11:37:52Z

+1
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2253/8/summary.html

cmsbuild · 2014-02-03T11:43:37Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_6_2_X_SLHC IBs unless changes (tests are also fine). @andersonjacob, @mark-grimes can you please take care of it?

ianna · 2014-02-03T14:43:17Z

@mark-grimes - I cannot reproduce the crash you get:

cmsrel CMSSW_6_2_X_SLHC_2014-02-03-0200
cd CMSSW_6_2_X_SLHC_2014-02-03-0200/src/
cmsenv
git cms-merge-topic boudoul:be5dfix_trapez
cd ../
mkdir relval
cd src/
scram b -j 10
cd ../relval/
runTheMatrix.py --what upgrade -l 50001  --command "--customise SLHCUpgradeSimulations/Configuration/combinedCustoms.cust_2019WithGem"

gives the result:

GEN,SIM,RECO,VALIDATION,ENDJOB
We have determined that this is simulation (if not, rerun cmsDriver.py with --data)
Step: GEN Spec: 
Loading generator fragment from Configuration.Generator.TTbar_Tauola_14TeV_cfi
Step: SIM Spec: 
Step: RECO Spec: 
Step: VALIDATION Spec: 
Step: ENDJOB Spec: 
globaltag = DES19_62_V8::All
customising the process with cust_2019WithGem from SLHCUpgradeSimulations/Configuration/combinedCustoms
Traceback (most recent call last):
  File "/afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-03-0200/bin/slc5_amd64_gcc472/cmsDriver.py", line 43, in <module>
    run()
  File "/afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-03-0200/bin/slc5_amd64_gcc472/cmsDriver.py", line 15, in run
    configBuilder.prepare()
  File "/afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-03-0200/python/Configuration/Applications/ConfigBuilder.py", line 2082, in prepare
    self.pythonCfgCode += self.addCustomise()
  File "/afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-03-0200/python/Configuration/Applications/ConfigBuilder.py", line 795, in addCustomise
    self.process=getattr(package,fcn)(self.process)
  File "/afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-03-0200/python/SLHCUpgradeSimulations/Configuration/combinedCustoms.py", line 62, in cust_2019WithGem
    process=cust_2019(process)
  File "/afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-03-0200/python/SLHCUpgradeSimulations/Configuration/combinedCustoms.py", line 56, in cust_2019
    process=customisePhase1Tk(process)
  File "/afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-03-0200/python/SLHCUpgradeSimulations/Configuration/phase1TkCustoms.py", line 31, in customise
    process=customise_Validation(process,float(n))
  File "/afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-03-0200/python/SLHCUpgradeSimulations/Configuration/phase1TkCustoms.py", line 93, in customise_Validation
    process.validation_step.remove(process.PixelTrackingRecHitsValid)
AttributeError: 'Process' object has no attribute 'PixelTrackingRecHitsValid'

cmsbuild · 2014-02-03T19:25:30Z

-1
When I ran the RelVals I found an error in the following worklfows:
50001 step1

runTheMatrix-results/50001_TTbar_14TeV+TTbarFSP2+HARVESTFSP2/step1_TTbar_14TeV+TTbarFSP2+HARVESTFSP2.log

60001 step1

runTheMatrix-results/60001_TTbar_14TeV+TTbarFSP2Forw+HARVESTFSP2Forw/step1_TTbar_14TeV+TTbarFSP2Forw+HARVESTFSP2Forw.log

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2253/14/summary.html

cmsbuild · 2014-02-03T19:28:20Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_6_2_X_SLHC IBs unless changes (but tests are reportedly failing). @andersonjacob, @mark-grimes can you please take care of it?

mark-grimes · 2014-02-04T14:50:05Z

Tested again on top of the recently merged #2265 and get the same result (as well as a known issue in harvesting for 2 tests). The GSF electron trajectory fitter crashes in FastSim for ExtendedPhase2TkBE and ExtendedPhase2TkBE. Will discuss in the Offline Release Planning meeting today whether the benefits of including in SLHC7 outweigh breaking FastSim.

davidlange6 · 2014-02-04T14:53:37Z

Mark - can you post the errors? (or point me to them)

On Feb 4, 2014, at 3:50 PM, Mark Grimes notifications@github.com
wrote:

Tested again on top of the recently merged #2265 and get the same result (as well as a known issue in harvesting for 2 tests). The GSF electron trajectory fitter crashes in FastSim for ExtendedPhase2TkBE and ExtendedPhase2TkBE. Will discuss in the Offline Release Planning meeting today whether the benefits of including in SLHC7 outweigh breaking FastSim.

—
Reply to this email directly or view it on GitHub.

mark-grimes · 2014-02-04T15:00:56Z

It's the same stack trace as comment number 5 (I cut some of the cruft, let me know if you want the full thing):

%MSG-e FatalSystemSignal:  GsfTrackProducer:electronGsfTracks  04-Feb-2014 14:25:16 CET Run: 1 Event: 9
A fatal system signal has occurred: segmentation violation

<trimmed>

#5  0x00002ab72552ac29 in CloseComponentsMerger<5u>::merge(MultiGaussianState<5u> const&) const () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/pluginTrackingToolsGsfToolsPlugins.so
#6  0x00002ab72bab83fd in MultiTrajectoryStateMerger::merge(TrajectoryStateOnSurface const&) const () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/libTrackingToolsGsfTracking.so
#7  0x00002ab72bab4503 in GsfTrajectoryFitter::fitOne(TrajectorySeed const&, std::vector<ConstReferenceCountingPointer<TransientTrackingRecHit>, std::allocator<ConstReferenceCountingPointer<TransientTrackingRecHit> > > const&, TrajectoryStateOnSurface const&, TrajectoryFitter::fitType) const () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/libTrackingToolsGsfTracking.so
#8  0x00002ab725409f27 in KFFittingSmoother::fitOne(TrajectorySeed const&, std::vector<ConstReferenceCountingPointer<TransientTrackingRecHit>, std::allocator<ConstReferenceCountingPointer<TransientTrackingRecHit> > > const&, TrajectoryStateOnSurface const&, TrajectoryFitter::fitType) const () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/libTrackingToolsTrackFitters.so
#9  0x00002ab74e71f5f7 in TrackProducerAlgorithm<reco::GsfTrack>::buildTrack(TrajectoryFitter const*, Propagator const*, std::vector<std::pair<Trajectory*, std::pair<reco::GsfTrack*, PropagationDirection> >, std::allocator<std::pair<Trajectory*, std::pair<reco::GsfTrack*, PropagationDirection> > > >&, std::vector<ConstReferenceCountingPointer<TransientTrackingRecHit>, std::allocator<ConstReferenceCountingPointer<TransientTrackingRecHit> > >&, TrajectoryStateOnSurface&, TrajectorySeed const&, float, reco::BeamSpot const&, edm::RefToBase<TrajectorySeed>, int, signed char) () from /tmp/grimes/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/libRecoTrackerTrackProducer.so
#10 0x00002ab74e6a3612 in TrackProducerAlgorithm<reco::GsfTrack>::runWithCandidate(TrackingGeometry const*, MagneticField const*, std::vector<TrackCandidate, std::allocator<TrackCandidate> > const&, TrajectoryFitter const*, Propagator const*, TransientTrackingRecHitBuilder const*, reco::BeamSpot const&, std::vector<std::pair<Trajectory*, std::pair<reco::GsfTrack*, PropagationDirection> >, std::allocator<std::pair<Trajectory*, std::pair<reco::GsfTrack*, PropagationDirection> > > >&) () from /tmp/grimes/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/pluginRecoTrackerTrackProducerPlugins.so
#11 0x00002ab74e6a1d45 in GsfTrackProducer::produce(edm::Event&, edm::EventSetup const&) () from /tmp/grimes/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/pluginRecoTrackerTrackProducerPlugins.so

@ianna says she can't reproduce it but it's the same as @cmsbuild gets (comment above referencing https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2253/14/summary.html). Note that there are thousands of this error beforehand:

%MSG-w BasicTrajectoryState:  GsfTrackProducer:electronGsfTracks  04-Feb-2014 14:25:16 CET Run: 1 Event: 9
curv error not pos-def
[          nan         nan         nan         nan         nan
           nan         nan         nan         nan         nan
           nan         nan         nan         nan         nan
           nan         nan         nan         nan         nan
           nan         nan         nan         nan         nan ]
%MSG

davidlange6 · 2014-02-04T15:03:54Z

which workflow #?
On Feb 4, 2014, at 4:00 PM, Mark Grimes notifications@github.com
wrote:

It's the same stack trace as comment number 5 (I cut some of the cruft, let me know if you want the full thing):

%MSG-e FatalSystemSignal: GsfTrackProducer:electronGsfTracks 04-Feb-2014 14:25:16 CET Run: 1 Event: 9
A fatal system signal has occurred: segmentation violation

#5 0x00002ab72552ac29 in CloseComponentsMerger<5u>::merge(MultiGaussianState<5u> const&) const () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/pluginTrackingToolsGsfToolsPlugins.so
#6 0x00002ab72bab83fd in MultiTrajectoryStateMerger::merge(TrajectoryStateOnSurface const&) const () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/libTrackingToolsGsfTracking.so
#7 0x00002ab72bab4503 in GsfTrajectoryFitter::fitOne(TrajectorySeed const&, std::vector<ConstReferenceCountingPointer, std::allocator<ConstReferenceCountingPointer > > const&, TrajectoryStateOnSurface const&, TrajectoryFitter::fitType) const () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/libTrackingToolsGsfTracking.so
#8 0x00002ab725409f27 in KFFittingSmoother::fitOne(TrajectorySeed const&, std::vector<ConstReferenceCountingPointer, std::allocator<ConstReferenceCountingPointer > > const&, TrajectoryStateOnSurface const&, TrajectoryFitter::fitType) const () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/libTrackingToolsTrackFitters.so
#9 0x00002ab74e71f5f7 in TrackProducerAlgorithmreco::GsfTrack::buildTrack(TrajectoryFitter const_, Propagator const_, std::vector<std::pair<Trajectory*, std::pair<reco::GsfTrack*, PropagationDirection> >, std::allocator<std::pair<Trajectory*, std::pair<reco::GsfTrack*, PropagationDirection> > > >&, std::vector<ConstReferenceCountingPointer, std::allocator<ConstReferenceCountingPointer > >&, TrajectoryStateOnSurface&, TrajectorySeed const&, float, reco::BeamSpot const&, edm::RefToBase, int, signed char) () from /tmp/grimes/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/libRecoTrackerTrackProducer.so
#10 0x00002ab74e6a3612 in TrackProducerAlgorithmreco::GsfTrack::runWithCandidate(TrackingGeometry const_, MagneticField const_, std::vector<TrackCandidate, std::allocator > const&, TrajectoryFitter const_, Propagator const_, TransientTrackingRecHitBuilder const_, reco::BeamSpot const&, std::vector<std::pair<Trajectory_, std::pair<reco::GsfTrack*, PropagationDirection> >, std::allocator<std::pair<Trajectory*, std::pair<reco::GsfTrack*, PropagationDirection> > > >&) () from /tmp/grimes/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/pluginRecoTrackerTrackProducerPlugins.so
#11 0x00002ab74e6a1d45 in GsfTrackProducer::produce(edm::Event&, edm::EventSetup const&) () from /tmp/grimes/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/pluginRecoTrackerTrackProducerPlugins.so

@ianna says she can't reproduce it but it's the same as @cmsbuild gets (comment above referencing https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2253/14/summary.html). Note that there are thousands of this error beforehand:

%MSG-w BasicTrajectoryState: GsfTrackProducer:electronGsfTracks 04-Feb-2014 14:25:16 CET Run: 1 Event: 9
curv error not pos-def
[ nan nan nan nan nan
nan nan nan nan nan
nan nan nan nan nan
nan nan nan nan nan
nan nan nan nan nan ]
%MSG

—
Reply to this email directly or view it on GitHub.

mark-grimes · 2014-02-04T15:11:46Z

Sorry, 50001 and 60001 (ttbar FastSim for ExtendedPhase2TkBE and ExtendedPhase2TkBEForward respectively). Since merging the HCal changes also get the step 4 harvesting errors in 3400 and 15001 but that's a separate expected issue.

ianna · 2014-02-04T15:22:03Z

@mark-grimes - correction - I cannot reproduce it with customization command. I did get the same crash without it. It would seem strange that a trajectory fitter cannot deal with rectangular bounds of a DetId.

mark-grimes · 2014-02-04T17:15:38Z

merge

Discussed in the offline release planning meeting and agreed to include for SLHC7 even though the FastSim crashes for these particular geometries. David L. was actually surprised the workflows ran beforehand.

fix tracker.xml

fix tracker.xml

0db7d9c

cmsbuild added a commit that referenced this pull request Feb 4, 2014

Merge pull request #2253 from boudoul/be5dfix_trapez

36557f9

fix tracker.xml

cmsbuild merged commit 36557f9 into cms-sw:CMSSW_6_2_X_SLHC Feb 4, 2014

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix tracker.xml #2253

fix tracker.xml #2253

boudoul commented Jan 30, 2014

cmsbuild commented Jan 30, 2014

ianna commented Jan 31, 2014

cmsbuild commented Jan 31, 2014

mark-grimes commented Jan 31, 2014

cmsbuild commented Feb 3, 2014

cmsbuild commented Feb 3, 2014

ianna commented Feb 3, 2014

cmsbuild commented Feb 3, 2014

cmsbuild commented Feb 3, 2014

mark-grimes commented Feb 4, 2014

davidlange6 commented Feb 4, 2014

mark-grimes commented Feb 4, 2014

davidlange6 commented Feb 4, 2014

mark-grimes commented Feb 4, 2014

ianna commented Feb 4, 2014

mark-grimes commented Feb 4, 2014

fix tracker.xml #2253

fix tracker.xml #2253

Conversation

boudoul commented Jan 30, 2014

cmsbuild commented Jan 30, 2014

ianna commented Jan 31, 2014

cmsbuild commented Jan 31, 2014

mark-grimes commented Jan 31, 2014

cmsbuild commented Feb 3, 2014

cmsbuild commented Feb 3, 2014

ianna commented Feb 3, 2014

cmsbuild commented Feb 3, 2014

cmsbuild commented Feb 3, 2014

mark-grimes commented Feb 4, 2014

davidlange6 commented Feb 4, 2014

mark-grimes commented Feb 4, 2014

davidlange6 commented Feb 4, 2014

mark-grimes commented Feb 4, 2014

ianna commented Feb 4, 2014

mark-grimes commented Feb 4, 2014