[py2py3] Migration at level scr/python/A/B/C - slice 6

[mapellidario]

Fixes #10127

Status

In development

Related PRs

• Previous migration PR: [py2py3] Migration at level scr/python/A/B/C - slice 5 #10262 (slice5, issue [py2py3] Migration at level scr/python/A/B/C - slice 5 #10116)
• Next migration PR: [py2py3] Migration at level scr/python/A/B/C - slice 7 #10284 (slice7, issue [py2py3] Migration at level scr/python/A/B/C - slice 7 #10131 )

Description

Run futurize and some manual changes on the first batch of src/python/A/B/C.

Use of "native-string" strategy:

• src/python/WMCore/JobSplitting/JobFactory.py: no from bultins import str here

division:

• src/python/WMCore/JobSplitting/ParentlessMergeBySize.py: assuming that

  self.currentJob.addResourceEstimates(jobTime = jobTime, disk = (jobSize+largestFile)/1024)

  can accept float arguments

Is it backward compatible (if not, which system it affects?)

It should be. (Any possible cause for errors will we reported here)

External dependencies / deployment changes

Requires python-future in both py2 and py3 environments.

[cmsdmwmbot]

Jenkins results:

• Unit tests: succeeded
  • 9 tests deleted
• Pylint check: failed
  • 64 warnings and errors that must be fixed
  • 7 warnings
  • 358 comments to review
• Pylint py3k check: succeeded
  • 0 errors and warnings that should be fixed
  • 0 warnings
  • 0 comments to review
• Pycodestyle check: succeeded
  • 586 comments to review
• Python3 compatibility checks: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/11206/artifact/artifacts/PullRequestReport.html

[mapellidario]

@amaltaro Since this slice involved src/python/WMCore/WMBS/Job.py, I run some unit tests in my oracle setup. This PR seems consistent with master, so far so good.

However, I noticed that the test

nosetests test/python/WMCore_t/WMBS_t/Job_t.py:JobTest.testSaveTransaction

is ok with mariadb but fails in oracle since at least 1.3.0. Is this to be expected?

logs

With code

        testJobA = self.createTestJob()
        ... # (changes to testJobA)
        testJobA.save()
        testJobB = Job(id=testJobA["id"])
        testJobB.loadData()
        assert testJobA["mask"] == testJobB["mask"], \
            "ERROR: Job mask did not load properly" # this assertion pass in both oracle and mariadb
        ... # (otherr changes to testJobA)
        testJobA.save()
        testJobC = Job(id=testJobA["id"])
        testJobC.loadData()

        print("testJobA: ", testJobA["mask"])
        print("testJobC: ", testJobC["mask"])
        assert testJobA["mask"] == testJobC["mask"], \
            "ERROR: Job mask did not load properly"

logs from mariadb setup (apart the int becoming long, these are identical and pass the assertion)

('testJobA: ', {'LastRun': 12, 'FirstRun': 11, 'inclusivemask': True, 'runAndLumis': {}, 'LastEvent': 8, 'FirstEvent': 7, 'LastLumi': 10, 'FirstLumi': 9})
('testJobC: ', {'LastRun': 12L, 'FirstRun': 11L, 'inclusivemask': True, 'runAndLumis': {}, 'LastEvent': 8L, 'FirstEvent': 7L, 'LastLumi': 10L, 'FirstLumi': 9L})

while, logs from oracle setup are

('testJobA: ', {'LastRun': 12, 'FirstRun': 11, 'inclusivemask': True, 'runAndLumis': {}, 'LastEvent': 8, 'FirstEvent': 7, 'LastLumi': 10, 'FirstLumi': 9})
('testJobC: ', {'LastRun': 6, 'FirstRun': 5, 'inclusivemask': True, 'runAndLumis': {}, 'LastEvent': 2, 'FirstEvent': 1, 'LastLumi': 4, 'FirstLumi': 3})

as if the second save does not behave. Do you need some more logs?

[amaltaro]

@mapellidario about the testSaveTransaction unit test:
#8372

So, I have no clue of what happens, but it seems to be somehow expected :-)

[amaltaro]

Changes look good Dario. Unless you have anything else to add here, I'll merge it tomorrow first thing in the morning.

[mapellidario]

Perfect! I consider this finished and ready to merge! Thanks!