Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

113 Improve reporting of test failures for CQL tests #153

Closed
wants to merge 20 commits into from

Conversation

absurdfarce
Copy link
Collaborator

No description provided.

if len(compareOnlySet) > 0:
log.info("Statements in compare file {} but not in reference file {}:".format(comparePath, referencePath))
logCqlDigest(comparePath, compareOnlySet)
for digest in compareOnlySet:
log.info(compareDict[digest])
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not really sure the explicitly logging we're doing here is necessary now.

Given the following diff (to introduce a CQL mismatch):

diff --git a/python/adelphi/tests/integration/resources/cql-schemas/4.0-rc1.cql b/python/adelphi/tests/integration/resources/cql-schemas/4.0-rc1.cql
index dbe0a97..c875e18 100644
--- a/python/adelphi/tests/integration/resources/cql-schemas/4.0-rc1.cql
+++ b/python/adelphi/tests/integration/resources/cql-schemas/4.0-rc1.cql
@@ -6,7 +6,7 @@ CREATE TYPE IF NOT EXISTS ks_0.udt_0 (
 );
 
 CREATE TYPE IF NOT EXISTS ks_0.udt_1 (
-    fld_2 frozen<udt_2>,
+    fld_22 frozen<udt_2>,
     fld_3 frozen<map<frozen<udt_2>, frozen<udt_2>>>,
     fld_4 frozen<list<frozen<udt_0>>>,
     fld_5 frozen<tuple<frozen<udt_2>, frozen<udt_0>>>

the blocks above print the following in the log (as displayed by tox):

INFO     adelphi:cassandra_util.py:23 Creating schema on Cassandra cluster from file /tmp/tmpoBGk1E/base-schema.cql
INFO     adelphi:test_cql.py:61 Statements in reference file tests/integration/resources/cql-schemas/4.0-rc1.cql but not in compare file /tmp/tmpoBGk1E/4.0-rc1-stdout.out:
INFO     adelphi:test_cql.py:63 CREATE TYPE IF NOT EXISTS ks_0.udt_1 ( fld_22 frozen<udt_2>, fld_3 frozen<map<frozen<udt_2>, frozen<udt_2>>>, fld_4 frozen<list<frozen<udt_0>>>, f
ld_5 frozen<tuple<frozen<udt_2>, frozen<udt_0>>> );                  
INFO     adelphi:test_cql.py:66 Statements in compare file /tmp/tmpoBGk1E/4.0-rc1-stdout.out but not in reference file tests/integration/resources/cql-schemas/4.0-rc1.cql:
INFO     adelphi:test_cql.py:68 CREATE TYPE IF NOT EXISTS ks_0.udt_1 ( fld_2 frozen<udt_2>, fld_3 frozen<map<frozen<udt_2>, frozen<udt_2>>>, fld_4 frozen<list<frozen<udt_0>>>, fl
d_5 frozen<tuple<frozen<udt_2>, frozen<udt_0>>> ); 

Note, however, that we've now switched to using sets of CQL statements for the assertEqual() call below, so the test actually fails with the following message:

ests/integration/test_cql.py:70: in compareToReferenceCql                                                                                                                        
    self.assertEqual(set(referenceDict.values()), set(compareDict.values()))                                                                                                      
E   AssertionError: Items in the first set but not the second:                                                                                                                    
E   'CREATE TYPE IF NOT EXISTS ks_0.udt_1 ( fld_22 frozen<udt_2>, fld_3 frozen<map<frozen<udt_2>, frozen<udt_2>>>, fld_4 frozen<list<frozen<udt_0>>>, fld_5 frozen<tuple<frozen<ud
t_2>, frozen<udt_0>>> );'                                                                                                                                                         
E   Items in the second set but not the first:                                                                                                                                    
E   'CREATE TYPE IF NOT EXISTS ks_0.udt_1 ( fld_2 frozen<udt_2>, fld_3 frozen<map<frozen<udt_2>, frozen<udt_2>>>, fld_4 frozen<list<frozen<udt_0>>>, fld_5 frozen<tuple<frozen<udt
_2>, frozen<udt_0>>> );'  

It's not clear to me that the explicit logging we do here is anything but repetitive.

Base automatically changed from 115-breadth-first-testing to master June 21, 2021 19:23
@absurdfarce
Copy link
Collaborator Author

Closing in favor of https://github.com/datastax/adelphi/pull/159/files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make reporting/logging for Python integration test infrastructure a bit more sane
2 participants