Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[7.15](backport #29985) Add note to documentation about dashboard loading in Beats #29987

Closed
wants to merge 1 commit into from
Closed

[7.15](backport #29985) Add note to documentation about dashboard loading in Beats #29987

wants to merge 1 commit into from

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Jan 25, 2022 •
edited by zube bot
Loading

This is an automatic backport of pull request #29985 done by Mergify.

Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

  • @Mergifyio refresh will re-evaluate the rules
  • @Mergifyio rebase will rebase this PR on its base branch
  • @Mergifyio update will merge the base branch into this PR
  • @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

  • look at your merge queues
  • generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.com

@mergify mergify bot added the backport label Jan 25, 2022
@mergify mergify bot assigned kvch Jan 25, 2022
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 25, 2022
@elasticmachine
Copy link
Collaborator

elasticmachine commented Jan 25, 2022 •
edited by jenkins-beats-ci bot
Loading

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Reason: null

  • Start Time: 2022-01-28T15:13:20.702+0000

  • Duration: 145 min 22 sec

  • Commit: 111ecb3

Test stats 🧪

Test Results
Failed 11
Passed 19977
Skipped 1519
Total 21507

Test errors 11

Expand to view the tests failures

> Show only the first 10 test failures

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_109_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T07:46:54.000-02:00', 'old_value': '2021-03-08T07:46:54.000-02:00'}}}, full object: 
  {'log.offset': 760, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T07:46:54.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Error', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'cyberarkpas.audit.message': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'cyberarkpas.audit.issuer': 'Builtin', 'cyberarkpas.audit.desc': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'host.name': 'VAULT', 'event.severity': 7, 'event.code': '479', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'security warning - the signature hash algorithm of the vault certificate is sha1.', 'event.type': 'error', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T07:46:54.000-02:00', 'old_value': '2021-03-08T07:46:54.000-02:00'}}})

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_109_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_109_cyberarkpas>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/479_security_warning_the_signature_hash_algorithm_of_the_vault_certificate_is_sha1.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': '811f7213-5f39-4f79-88fb-dfd9e782af19', 'hostnam...y warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', ...}}, 'ecs': {'version': '1.11.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list):
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T07:46:54.000-02:00', 'old_value': '2021-03-08T07:46:54.000-02:00'}}}, full object: 
E             {'log.offset': 760, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T07:46:54.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Error', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'cyberarkpas.audit.message': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'cyberarkpas.audit.issuer': 'Builtin', 'cyberarkpas.audit.desc': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'host.name': 'VAULT', 'event.severity': 7, 'event.code': '479', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'security warning - the signature hash algorithm of the vault certificate is sha1.', 'event.type': 'error', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T07:46:54.000-02:00', 'old_value': '2021-03-08T07:46:54.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:228: AssertionError

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_111_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:48:07.000-02:00', 'old_value': '2021-03-08T02:48:07.000-02:00'}}}, full object: 
  {'log.offset': 598, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T02:48:07.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor DR Replication start', 'cyberarkpas.audit.message': 'Monitor DR Replication start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor DR Replication start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '310', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor dr replication start', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:48:07.000-02:00', 'old_value': '2021-03-08T02:48:07.000-02:00'}}})

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_111_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_111_cyberarkpas>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/310_monitor_dr_replication_start.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': '08b183f6-3cc6-4b4d-95cb-e0e27653650b', 'hostnam...cation start', 'issuer': 'Batch', 'message': 'Monitor DR Replication start', ...}}, 'ecs': {'version': '1.11.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list):
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:48:07.000-02:00', 'old_value': '2021-03-08T02:48:07.000-02:00'}}}, full object: 
E             {'log.offset': 598, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T02:48:07.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor DR Replication start', 'cyberarkpas.audit.message': 'Monitor DR Replication start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor DR Replication start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '310', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor dr replication start', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:48:07.000-02:00', 'old_value': '2021-03-08T02:48:07.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:228: AssertionError

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_125_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T03:00:20.000-02:00', 'old_value': '2021-03-08T03:00:20.000-02:00'}}}, full object: 
  {'log.offset': 598, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T03:00:20.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Auto Clear Users History end', 'cyberarkpas.audit.message': 'Auto Clear Users History end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Auto Clear Users History end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '289', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'auto clear users history end', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T03:00:20.000-02:00', 'old_value': '2021-03-08T03:00:20.000-02:00'}}})

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_125_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_125_cyberarkpas>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/289_auto_clear_users_history_end.log'
objects = [{'@timestamp': '2021-03-05T11:00:06.000Z', 'agent': {'ephemeral_id': 'dbd73b76-bf5d-49ac-bf75-167b39f762e1', 'hostnam... History end', 'issuer': 'Batch', 'message': 'Auto Clear Users History end', ...}}, 'ecs': {'version': '1.11.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list):
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T03:00:20.000-02:00', 'old_value': '2021-03-08T03:00:20.000-02:00'}}}, full object: 
E             {'log.offset': 598, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T03:00:20.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Auto Clear Users History end', 'cyberarkpas.audit.message': 'Auto Clear Users History end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Auto Clear Users History end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '289', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'auto clear users history end', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T03:00:20.000-02:00', 'old_value': '2021-03-08T03:00:20.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:228: AssertionError

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_133_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:48:07.000-02:00', 'old_value': '2021-03-08T02:48:07.000-02:00'}}}, full object: 
  {'log.offset': 592, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T02:48:07.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor DR Replication end', 'cyberarkpas.audit.message': 'Monitor DR Replication end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor DR Replication end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '311', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor dr replication end', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:48:07.000-02:00', 'old_value': '2021-03-08T02:48:07.000-02:00'}}})

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_133_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_133_cyberarkpas>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/311_monitor_dr_replication_end.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': '2d9a982b-ec79-49e2-a74f-d3f51f097f29', 'hostnam...eplication end', 'issuer': 'Batch', 'message': 'Monitor DR Replication end', ...}}, 'ecs': {'version': '1.11.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list):
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:48:07.000-02:00', 'old_value': '2021-03-08T02:48:07.000-02:00'}}}, full object: 
E             {'log.offset': 592, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T02:48:07.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor DR Replication end', 'cyberarkpas.audit.message': 'Monitor DR Replication end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor DR Replication end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '311', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor dr replication end', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:48:07.000-02:00', 'old_value': '2021-03-08T02:48:07.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:228: AssertionError

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_138_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:32:56.000-02:00', 'old_value': '2021-03-08T02:32:56.000-02:00'}}}, full object: 
  {'log.offset': 574, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T02:32:56.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor FW Rules end', 'cyberarkpas.audit.message': 'Monitor FW Rules end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor FW Rules end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '358', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor fw rules end', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:32:56.000-02:00', 'old_value': '2021-03-08T02:32:56.000-02:00'}}})

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_138_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_138_cyberarkpas>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/358_monitor_fw_rules_end.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': 'f2e6e2da-541d-4b5d-a4b0-73cb4282682e', 'hostnam...Monitor FW Rules end', 'issuer': 'Batch', 'message': 'Monitor FW Rules end', ...}}, 'ecs': {'version': '1.11.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list):
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:32:56.000-02:00', 'old_value': '2021-03-08T02:32:56.000-02:00'}}}, full object: 
E             {'log.offset': 574, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T02:32:56.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor FW Rules end', 'cyberarkpas.audit.message': 'Monitor FW Rules end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor FW Rules end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '358', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor fw rules end', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:32:56.000-02:00', 'old_value': '2021-03-08T02:32:56.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:228: AssertionError

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_141_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T03:00:20.000-02:00', 'old_value': '2021-03-08T03:00:20.000-02:00'}}}, full object: 
  {'log.offset': 604, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T03:00:20.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Auto Clear Users History start', 'cyberarkpas.audit.message': 'Auto Clear Users History start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Auto Clear Users History start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '288', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'auto clear users history start', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T03:00:20.000-02:00', 'old_value': '2021-03-08T03:00:20.000-02:00'}}})

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_141_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_141_cyberarkpas>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/288_auto_clear_users_history_start.log'
objects = [{'@timestamp': '2021-03-05T11:00:06.000Z', 'agent': {'ephemeral_id': 'b2684b15-a9ee-49f1-8057-35a98ffa7336', 'hostnam...tory start', 'issuer': 'Batch', 'message': 'Auto Clear Users History start', ...}}, 'ecs': {'version': '1.11.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list):
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T03:00:20.000-02:00', 'old_value': '2021-03-08T03:00:20.000-02:00'}}}, full object: 
E             {'log.offset': 604, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T03:00:20.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Auto Clear Users History start', 'cyberarkpas.audit.message': 'Auto Clear Users History start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Auto Clear Users History start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '288', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'auto clear users history start', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T03:00:20.000-02:00', 'old_value': '2021-03-08T03:00:20.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:228: AssertionError

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_143_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T03:41:01.000-02:00', 'old_value': '2021-03-08T03:41:01.000-02:00'}}}, full object: 
  {'log.offset': 0, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T03:41:01.000-02:00', 'file.path': 'Root\\Policies\\Policy-BusinessWebsite.ini', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.message': 'Retrieve File', 'cyberarkpas.audit.issuer': 'PasswordManager', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.file': 'Root\\Policies\\Policy-BusinessWebsite.ini', 'cyberarkpas.audit.safe': 'PasswordManagerShared', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Retrieve File', 'cyberarkpas.audit.desc': 'Retrieve File', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '51', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'retrieve file', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T03:41:01.000-02:00', 'old_value': '2021-03-08T03:41:01.000-02:00'}}})

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_143_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_143_cyberarkpas>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/legacysyslog.log'
objects = [{'@timestamp': '2022-03-08T03:41:01.000-02:00', 'agent': {'ephemeral_id': 'c6831631-f5bd-4bae-a4f4-37dd7199c915', 'ho...: 'Root\\Policies\\Policy-BusinessWebsite.ini', 'issuer': 'PasswordManager', ...}}, 'ecs': {'version': '1.11.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list):
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T03:41:01.000-02:00', 'old_value': '2021-03-08T03:41:01.000-02:00'}}}, full object: 
E             {'log.offset': 0, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T03:41:01.000-02:00', 'file.path': 'Root\\Policies\\Policy-BusinessWebsite.ini', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.message': 'Retrieve File', 'cyberarkpas.audit.issuer': 'PasswordManager', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.file': 'Root\\Policies\\Policy-BusinessWebsite.ini', 'cyberarkpas.audit.safe': 'PasswordManagerShared', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Retrieve File', 'cyberarkpas.audit.desc': 'Retrieve File', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '51', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'retrieve file', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T03:41:01.000-02:00', 'old_value': '2021-03-08T03:41:01.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:228: AssertionError

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_145_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:32:56.000-02:00', 'old_value': '2021-03-08T02:32:56.000-02:00'}}}, full object: 
  {'log.offset': 580, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T02:32:56.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor FW rules start', 'cyberarkpas.audit.message': 'Monitor FW rules start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor FW rules start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '357', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor fw rules start', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:32:56.000-02:00', 'old_value': '2021-03-08T02:32:56.000-02:00'}}})

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_145_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_145_cyberarkpas>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/357_monitor_fw_rules_start.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': '838c94a4-5dcb-447a-8913-15ceef52b6ae', 'hostnam...tor FW rules start', 'issuer': 'Batch', 'message': 'Monitor FW rules start', ...}}, 'ecs': {'version': '1.11.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list):
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:32:56.000-02:00', 'old_value': '2021-03-08T02:32:56.000-02:00'}}}, full object: 
E             {'log.offset': 580, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T02:32:56.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor FW rules start', 'cyberarkpas.audit.message': 'Monitor FW rules start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor FW rules start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '357', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor fw rules start', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:32:56.000-02:00', 'old_value': '2021-03-08T02:32:56.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:228: AssertionError

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_148_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T03:10:31.000-02:00', 'old_value': '2021-03-08T03:10:31.000-02:00'}}}, full object: 
  {'log.offset': 604, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T03:10:31.000-02:00', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.safe': 'PasswordManager_workspace', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Clear Safe History', 'cyberarkpas.audit.message': 'Clear Safe History', 'cyberarkpas.audit.issuer': 'PasswordManager', 'cyberarkpas.audit.desc': 'Clear Safe History', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '59', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'clear safe history', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T03:10:31.000-02:00', 'old_value': '2021-03-08T03:10:31.000-02:00'}}})

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_148_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_148_cyberarkpas>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/59_clear_safe_history.log'
objects = [{'@timestamp': '2021-03-04T19:25:02.000Z', 'agent': {'ephemeral_id': '967d67bd-1451-43a1-9de7-5b53d9578044', 'hostnam...r Safe History', 'iso_timestamp': '2021-03-09T09:00:47Z', 'issuer': 'Batch', ...}}, 'ecs': {'version': '1.11.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list):
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T03:10:31.000-02:00', 'old_value': '2021-03-08T03:10:31.000-02:00'}}}, full object: 
E             {'log.offset': 604, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T03:10:31.000-02:00', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.safe': 'PasswordManager_workspace', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Clear Safe History', 'cyberarkpas.audit.message': 'Clear Safe History', 'cyberarkpas.audit.issuer': 'PasswordManager', 'cyberarkpas.audit.desc': 'Clear Safe History', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '59', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'clear safe history', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T03:10:31.000-02:00', 'old_value': '2021-03-08T03:10:31.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:228: AssertionError

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_159_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:54:46.000-02:00', 'old_value': '2021-03-08T02:54:46.000-02:00'}}}, full object: 
  {'log.offset': 1113, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T02:54:46.000-02:00', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Set Password', 'cyberarkpas.audit.message': 'Set Password', 'cyberarkpas.audit.issuer': 'PVWAGWUser', 'cyberarkpas.audit.desc': 'Set Password', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '88', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'set password', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:54:46.000-02:00', 'old_value': '2021-03-08T02:54:46.000-02:00'}}})

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_159_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_159_cyberarkpas>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/88_set_password.log'
objects = [{'@timestamp': '2021-03-04T19:16:19.000Z', 'agent': {'ephemeral_id': '6300e20c-45f0-4158-8636-c982705490a8', 'hostnam...mp': '2021-03-10T09:11:52Z', 'issuer': 'PSMPApp_localhost.localdomain', ...}}, 'ecs': {'version': '1.11.0'}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list):
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:54:46.000-02:00', 'old_value': '2021-03-08T02:54:46.000-02:00'}}}, full object: 
E             {'log.offset': 1113, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2022-03-08T02:54:46.000-02:00', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Set Password', 'cyberarkpas.audit.message': 'Set Password', 'cyberarkpas.audit.issuer': 'PVWAGWUser', 'cyberarkpas.audit.desc': 'Set Password', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '88', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'set password', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2022-03-08T02:54:46.000-02:00', 'old_value': '2021-03-08T02:54:46.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:228: AssertionError

Steps errors 10

Expand to view the steps failures

generator-metricbeat-test - make -C generator/_templates/metricbeat test test-package
  • Took 4 min 39 sec . View more details here
  • Description: make -C generator/_templates/metricbeat test test-package
generator-metricbeat-test - make -C generator/_templates/metricbeat test test-package
  • Took 0 min 16 sec . View more details here
  • Description: make -C generator/_templates/metricbeat test test-package
generator-metricbeat-test - make -C generator/_templates/metricbeat test test-package
  • Took 0 min 16 sec . View more details here
  • Description: make -C generator/_templates/metricbeat test test-package
generator-beat-test - make -C generator/_templates/beat test test-package
  • Took 4 min 55 sec . View more details here
  • Description: make -C generator/_templates/beat test test-package
generator-beat-test - make -C generator/_templates/beat test test-package
  • Took 1 min 30 sec . View more details here
  • Description: make -C generator/_templates/beat test test-package
generator-beat-test - make -C generator/_templates/beat test test-package
  • Took 0 min 20 sec . View more details here
  • Description: make -C generator/_templates/beat test test-package
x-pack/filebeat-pythonIntegTest - mage pythonIntegTest
  • Took 34 min 23 sec . View more details here
  • Description: mage pythonIntegTest
x-pack/filebeat-pythonIntegTest - mage pythonIntegTest
  • Took 31 min 19 sec . View more details here
  • Description: mage pythonIntegTest
x-pack/filebeat-pythonIntegTest - mage pythonIntegTest
  • Took 31 min 26 sec . View more details here
  • Description: mage pythonIntegTest
Error signal
  • Took 0 min 0 sec . View more details here
  • Description: Error 'hudson.AbortException: script returned exit code 2'

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Genuine test errors 11

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

  • Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_109_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_111_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_125_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_133_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_138_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_141_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_143_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_145_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_148_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_159_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_301_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@mtojek mtojek added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label Jan 27, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 27, 2022
@kvch kvch force-pushed the mergify/bp/7.15/pr-29985 branch from 27f0624 to 111ecb3 Compare January 28, 2022 15:13
@mergify
Copy link
Contributor Author

mergify bot commented Jan 31, 2022

This pull request has not been merged yet. Could you please review and merge it @kvch? 🙏

2 similar comments
@mergify
Copy link
Contributor Author

mergify bot commented Feb 7, 2022

This pull request has not been merged yet. Could you please review and merge it @kvch? 🙏

@mergify
Copy link
Contributor Author

mergify bot commented Feb 14, 2022

This pull request has not been merged yet. Could you please review and merge it @kvch? 🙏

@kvch kvch closed this Feb 14, 2022
@mergify mergify bot deleted the mergify/bp/7.15/pr-29985 branch February 14, 2022 07:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@kvch kvch

Labels
backport Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@elasticmachine @kvch @mtojek