Core update

[jackaraz]

Context

This branch is dedicated to improving the C++ functionalities of MadAnalysis 5.
It mainly adapts std=c++11 and applies appropriate modifications for the
data structure in the updated c++ scheme. Additionally provides useful
functionality for the expert mode.

Description of the change and benefits

Interface updates

• Added extra locations for detection of global zlib libraries and header
  files on macOS.
• MadAnalysis 5 minimum running requirement is c++11. madanalysis/system/detect_gpp.py L85
  won't let users use old GCC versions anymore. This has room to update to c++14 already implemented
  and Ma5 will be aware when c++14 is present but we are currently not directly using it.
• Python version has been limited to a minimum of 3.8. There is no support for lower versions
  anymore, we can not guarantee the stability of the software.
• ⚠️ Crucial bugfix in pyhf wrapper showed up during compatibility tests. without this code will crash.

Updates in expert mode structure

• tools/SampleAnalyzer/Commons/Service/Utils.h provides various new functionalities
  for the expert mode:

filter: filters given reconstructed object with respect to ptmin, absetamax and absetamin

MAdouble64 ptmin = 20., etamax = 2.5;
std::vector<RecJetFormat> signaljets = filter(event.rec()->jets(), ptmin, etamax);

filter_select: Uses lambda function to filter a given reconstructed object

std::vector<const RecJetFormat*> SignalJets = filter_select(event.rec()->jets(), \
        [] (const RecJetFormat* jet) { return (jet->pt()>20. && jet->abseta()>2. && \
                                               jet->abseta()<4. && !jet->btag() && !jet->ctag()); });

OverlapRemoval: Standard overlap removal function input1 is cleaned from input2
with respect to an angular separation drmin:

SignalJets = OverlapRemoval(SignalJets, SignalElectrons, 0.2);

conditional_removal: Similarly cleans input1 from input2 but with respect to a
lambda function

SignalJets = conditional_removal(SignalJets, SignalElectrons, 
    [] (const RecJetFormat* jet, const RecLeptonFormat* lep)
        {
            return jet.dr(lep) > 0.2;
        };
);

• tools/SampleAnalyzer/Process/Analyzer/AnalyzerBase.h: Additional functionality to add
  hadronic and invisible particles. AddDefaultHadronic() and AddDefaultInvisible()
  automatically adds all the necessary particles to the analysis.

bool analysis::Initialize(
    const MA5::Configuration& cfg, const std::map<std::string, std::string>& parameters
)
{
    // Initializing PhysicsService for MC
    PHYSICS->mcConfig().Reset();
    AddDefaultHadronic();
    AddDefaultInvisible();
    return true;
}

• tools/SampleAnalyzer/Commons/Service/MCconfig.h: In order to prevent any drawback from
  default addition of the particles in the analysis MCconfig now provides RemoveHadronicId and
  RemoveInvisibleId function.

bool analysis::Initialize(
    const MA5::Configuration& cfg, const std::map<std::string, std::string>& parameters
)
{
    // Initializing PhysicsService for MC
    PHYSICS->mcConfig().Reset();
    AddDefaultHadronic();
    AddDefaultInvisible();
    PHYSICS->mcConfig().RemoveInvisibleId(1000022); // Make neutralino visible
    return true;
}

Both of these methods are included in the normal mode run as well (only in reco mode).

• AnalyzerBase.h - getOption and SetOptions routine: These functions allow analysis
  to accept command line options.

bool analysis::Initialize(const MA5::Configuration& cfg, const std::map<std::string,std::string>& parameters)
{
    // Initializing PhysicsService for MC
    PHYSICS->mcConfig().Reset();
    AddDefaultHadronic();
    AddDefaultInvisible();

    MAdouble64 value = getOption<MAdouble64>("my-value", 20.); // default value 20
    return true;
}

one can change the default value of value input through the command line as follows;

$ ./MadAnalysis5job ../Input/_sample.list --my-value=10.

• tools/SampleAnalyzer/Process/Core/SampleAnalyzer.cpp initialization of the analysis
  has been relocated to PostInitialize() so that analysis can have access to the output
  folder path through Output() option within Initialize function.

• ⚠️Signal region dependent reweighting:⚠️ as before SetCurrentEventWeight(weight) reweigths all the events with the same weight. Region-based reweighting added via
  SetRegionWeight(std::string name, MAfloat64 weight) where name stands for the region name and weight is the desired weight
  to be set for the event. Note that neither of these functions scales the weights they rather set the weight.

  • tools/SampleAnalyzer/Process/RegionSelection/RegionSelection.h: SetWeight is the per region weight setter and
    GetWeight is per region weight getter.
  • tools/SampleAnalyzer/Process/RegionSelection/RegionSelectionManager.cpp: Minor change; instead of using weight_
    now we are setting the cutflow weight via the per region weight getter shown above.
  • tools/SampleAnalyzer/Process/RegionSelection/RegionSelectionManager.cpp: redefinition of SetCurrentEventWeight and
    definition of new function SetRegionWeight.

Tests done for backwards compatibility

In PAD:

• ATLAS SUSY 2018 031
• CMS SUS 19 006
• ATLAS SUSY 2019 08
• CMS SUS 16 039

In Normal mode:

Ma5 card for normal mode test: $ ./bin/ma5 -R

set main.fastsim.package = fastjet
set main.fastsim.algorithm = antikt
import tag_1_pythia8_events.hepmc.gz
set defaultset.xsection = 0.00629
define invisible = -16 -14 -12 12 14 16
plot PT(j)
submit

New analysis initialization output mentioned above:

user.cpp:

...
MAbool user::Initialize(const MA5::Configuration& cfg,
                      const std::map<std::string,std::string>& parameters)
{
  // Initializing PhysicsService for MC
  PHYSICS->mcConfig().Reset();


  // definition of the multiparticle "hadronic"
  AddDefaultHadronic();
  // definition of the multiparticle "invisible"
  AddDefaultInvisible();
  PHYSICS->mcConfig().RemoveInvisibleId(1000022);
  PHYSICS->mcConfig().RemoveInvisibleId(1000039);
...

All tests are executed with proper data (300K events phase-space chosen for the analyses specifically).

Future directions

• tools/SampleAnalyzer/Commons/Service/Utils.h: this file is to be expanded with all functionality that
  has been regularly used within MadAnalysis 5.
• Although c++11 is a big step for MadAnalysis 5 we need to move forward and adapt to c++14 asap.
• Signal region dependent weighting is just a simple patch for cut-and-count analysis. In order to be able to use
  multi weights within Ma5 globally one needs to create an object-oriented Weight class and instead of manipulating
  doubles analysis should always scale the class itself. This will allow us to track all the weights at a given time
  and by giving them the ability to combine in certain ways we will be able to have multiweight treatment on the fly.

Drawbacks

• For each particle rec()->tracks() are asking for the charge of the particle. There are many generator specific
  particles in the particle history that are not defined within tools/SampleAnalyzer/particle.tbl. Thus Ma5 constantly
  issues warnings. This is not necessarily a problem but doesn't look good during execution. The cause of the problem
  is L231:
  

  madanalysis5/tools/SampleAnalyzer/Process/JetClustering/JetClusterer.cpp

  Lines 204 to 234 in 92ca66d

  	// Set up tracks as charged FS particles OR charged interstate particles with nonzero ctau
  	if (PDG->IsCharged(part.pdgid()) && part.mothers().size()>0)
  	{
  	// Minimum tracking requirement is around 0.5 mm see ref. 1007.1988
  	if (part.ctau() > 0. || PHYSICS->Id->IsFinalState(part))
  	{
  	// Reminder: -1 is reserved for the tracks
  	MCParticleFormat smeared_track = mySmearer_->Execute(&part, -1);
  	if (smeared_track.pt() > 1e-5)
  	{
  	RecTrackFormat * track = myEvent.rec()->GetNewTrack();
  	MALorentzVector trk_mom;
  	trk_mom.SetPtEtaPhiM(smeared_track.pt(),
  	smeared_track.eta(),
  	smeared_track.phi(),0.0);
  	track->setMomentum(trk_mom);
  	track->setD0(smeared_track.d0());
  	track->setDZ(smeared_track.dz());
  	track->setD0Approx(smeared_track.d0_approx());
  	track->setDZApprox(smeared_track.dz_approx());
  	MAdouble64 ctau = PHYSICS->Id->IsFinalState(part) ? 0.0 : part.mothers()[0]->ctau();
  	MALorentzVector new_vertex(part.mothers()[0]->decay_vertex().X(),
  	part.mothers()[0]->decay_vertex().Y(),
  	part.mothers()[0]->decay_vertex().Z(), ctau);
  	track->setProductionVertex(new_vertex);
  	track->setClosestApproach(smeared_track.closest_approach());
  	track->setMc(&(part));
  	track->SetCharge(PDG->GetCharge(part.pdgid()) / 3.);
  	}
  	}
  	}

  • Possible solution is to add quiet delivery for warnings.
• Signal region dependent weighting won't work on histograms. The modification is too simple. See above
  for possible universal weight treatment.

[jackaraz]

As discussed privately; there is an additional problem with

madanalysis5/madanalysis/interpreter/cmd_install.py

Lines 102 to 110 in 92ca66d

	elif ResuActi == 1 and not has_release:
	self.logger.warning(to_activate + " is not installed: installing it...")
	resu = installer.Execute(release)
	if resu:
	UpdatePaths()
	if not main.CheckConfig():
	return False
	return resu
	elif ResuActi == 0 and has_release and not pad:

return resu in L109 needs to be out of the previous if condition otherwise when Delphes installation fails, PAD will still be installed.

[BFuks]

Drawbacks

* For each particle `rec()->tracks()` are asking for the charge of the particle. There are many generator specific
  particles in the particle history that are not defined within `tools/SampleAnalyzer/particle.tbl`. Thus Ma5 constantly
  issues warnings. This is not necessarily a problem but doesn't look good during execution. The cause of the problem
  is `L231`:
  https://github.com/MadAnalysis/madanalysis5/blob/92ca66d789cffeba39ec227c6ca2fe895e5a0f48/tools/SampleAnalyzer/Process/JetClustering/JetClusterer.cpp#L204-L234

In principle, the warning should be printed only 10 times. This has to be checked. Maybe there is something wrong in there.

[jackaraz]

In principle, the warning should be printed only 10 times. This has to be checked. Maybe there is something wrong in there.

It might be only 10, but there are too many such particles, so I believe we can set up a quiet delivery, and it will print everything at the end of the analysis with their counts. We can set a bool silence = false so that it won't affect the rest of the code.