test_openmm_dihedral failing

[zhang-ivy]

It seems like this test is failing all of a sudden, but I'm not sure why.

@mikemhenry @ijpulidos : could you investigate why?
I'm not sure if this is only happening in #860 or not, but the tests were passing fine for commit 4b72d38.

https://github.com/choderalab/perses/pull/860/checks?check_run_id=3942799861

=================================== FAILURES ===================================
_____________________________ test_openmm_dihedral _____________________________
[gw1] linux -- Python 3.9.7 /usr/share/miniconda/envs/test/bin/python

    def test_openmm_dihedral():
        """
        Test FFAllAngleGeometryEngine _internal_to_cartesian and _cartesian_to_internal are consistent with OpenMM torsion angles.
        """
        TORSION_TOLERANCE = 1.0e-4 # permitted disagreement in torsions
    
        # Create geometry engine
        from perses.rjmc import geometry
        geometry_engine = geometry.FFAllAngleGeometryEngine({'test': 'true'})
    
        # Create a four-bead test system with a single custom force that measures the OpenMM torsion
        import simtk.openmm as openmm
        integrator = openmm.VerletIntegrator(1.0*unit.femtoseconds)
        sys = openmm.System()
        force = openmm.CustomTorsionForce("theta")
        for i in range(4):
            sys.addParticle(1.0*unit.amu)
        force.addTorsion(0,1,2,3,[])
        sys.addForce(force)
        positions = unit.Quantity(np.array([
                [0.10557722, -1.10424644, -1.08578826],
                [0.0765,  0.1,  -0.4005],
                [0.0829, 0.0952, -0.2479],
                [-0.057,  0.0951, -0.1863],
                ]), unit.nanometers)
        atom_position = positions[0,:]
        bond_position = positions[1,:]
        angle_position = positions[2,:]
        torsion_position = positions[3,:]
    
        #atom_position = unit.Quantity(np.array([ 0.10557722 ,-1.10424644 ,-1.08578826]), unit=unit.nanometers)
        #bond_position = unit.Quantity(np.array([ 0.0765,  0.1  ,  -0.4005]), unit=unit.nanometers)
        #angle_position = unit.Quantity(np.array([ 0.0829 , 0.0952 ,-0.2479]) ,unit=unit.nanometers)
        #torsion_position = unit.Quantity(np.array([-0.057 ,  0.0951 ,-0.1863] ) ,unit=unit.nanometers)
    
        # Compute the dimensionless internal coordinates consistent with this geometry
        rtp, detJ = geometry_engine._cartesian_to_internal(atom_position, bond_position, angle_position, torsion_position)
        (r, theta, phi) = rtp # dimensionless internal coordinates
    
        # Create a reference context
        platform = openmm.Platform.getPlatformByName("Reference")
        context = openmm.Context(sys, integrator, platform)
        context.setPositions([atom_position, bond_position, angle_position, torsion_position])
        openmm_phi = context.getState(getEnergy=True).getPotentialEnergy()/unit.kilojoule_per_mole # this system converts torsion radians -> kJ/mol
        assert np.linalg.norm(openmm_phi - phi) < TORSION_TOLERANCE, '_cartesian_to_internal and OpenMM disagree on torsions'
    
        # Test _internal_to_cartesian by rotating around the torsion
        n_divisions = 100
        phis = np.arange(-np.pi, np.pi, (2.0*np.pi)/n_divisions) # _internal_to_cartesian only accepts dimensionless quantities
        for i, phi in enumerate(phis):
            # Note that (r, theta, phi) are dimensionless here
            xyz_atom1, _ = geometry_engine._internal_to_cartesian(bond_position, angle_position, torsion_position, r, theta, phi)
            positions[0,:] = xyz_atom1
            context.setPositions(positions)
            openmm_phi = context.getState(getEnergy=True).getPotentialEnergy()/unit.kilojoule_per_mole # this system converts torsion radians -> kJ/mol
            msg  = '_internal_to_cartesian and OpenMM disagree on torsions: \n'
            msg += '_internal_to_cartesian generated positions for: {}\n'.format(phi)
            msg += 'OpenMM: {}\n'.format(openmm_phi)
            msg += 'positions: {}'.format(positions)
>           assert np.linalg.norm(openmm_phi - phi) < TORSION_TOLERANCE, msg
E           AssertionError: _internal_to_cartesian and OpenMM disagree on torsions: 
E             _internal_to_cartesian generated positions for: -3.141592653589793
E             OpenMM: 3.141592653589793
E             positions: [[ 1.27477734  0.10606969 -1.09675394]
E              [ 0.0765      0.1        -0.4005    ]
E              [ 0.0829      0.0952     -0.2479    ]
E              [-0.057       0.0951     -0.1863    ]] nm
E           assert 6.283185307179586 < 0.0001
E            +  where 6.283185307179586 = <function norm at 0x7f1094d7a820>((3.141592653589793 - -3.141592653589793))
E            +    where <function norm at 0x7f1094d7a820> = <module 'numpy.linalg' from '/usr/share/miniconda/envs/test/lib/python3.9/site-packages/numpy/linalg/__init__.py'>.norm
E            +      where <module 'numpy.linalg' from '/usr/share/miniconda/envs/test/lib/python3.9/site-packages/numpy/linalg/__init__.py'> = np.linalg

perses/tests/test_coordinate_numba.py:105: AssertionError

[mikemhenry]

Looks like it is passing fine on master https://github.com/choderalab/perses/actions/runs/1356975445

It isn't failing on all the tests so I will re-run it as well

[ijpulidos]

Yes, I agree with @mikemhenry , it might just be some weird behavior from the CI environment or something like that. Because I cannot reproduce it locally after running it several times. I get the exact same behavior for tests before and after the changes.

[zhang-ivy]

Looks like it is passing fine on master https://github.com/choderalab/perses/actions/runs/1356975445

That run was from much earlier in the day. You can see that the test is failing on master now too:
https://github.com/choderalab/perses/actions/runs/1361421142

Could one of you dig into what might be wrong with the CI environment?

[ijpulidos]

@zhang-ivy This is very strange, that number being used in the failed assertion is exactly 2𝝅 🤔

[ijpulidos]

Okay, so clearly we are getting positve values for the openmm_phi, whereas we get negative for phi. I don't know why I cannot reproduce this locally. Any ideas about what a positive phi means here? Maybe that tells us something.

[mikemhenry]

Something weird is going on, looking at the last 2 nights of CI runs:

https://github.com/choderalab/perses/actions/runs/1365813609
https://github.com/choderalab/perses/actions/runs/1361421142

We are getting this error, but not consistently across python versions or openmm versions. So we need to dig into these tests and see if there is a source of randomness somewhere. I'm going to tag this into our current milestone so we can talk about it at our meeting today.

[jchodera]

The test is sensitive to tiny numerical errors, and should be changed from

assert np.linalg.norm(openmm_phi - phi) < TORSION_TOLERANCE, msg

to

# Compute absolute error across periodic boundary
delta = abs(openmm_phi - phi)
error = min(delta, 2*np.pi-delta)
assert error < TORSION_TOLERANCE, msg