Intermitent: java.lang.RuntimeException: Class com.amazonaws.athena.jdbc.AthenaDriver not found

[jake-barry]

Make sure these boxes are checked before submitting your issue - thank you!

• I have checked the superset logs for python stacktraces and included it here as text if any
• [x ] I have reproduced the issue with at least the latest released version of superset
• [x ] I have checked the issue tracker for the same issue and I haven't found one similar

Superset version: 0.24.0

Expected results: Athena query succeeds

Actual results: I'm able to connect to Athena, run queries, and build a dashboard. However, I get the below error every 1/4 - 3/4 times. For example, I have a dahsbaord with four panels and at least one and sometimes three of the pannels will say:

<bound method java.lang.RuntimeExceptionPyRaisable.message of RuntimeExceptionPyRaisable(<jpype._jclass.java.lang.RuntimeException object at 0x7fa8493be610>,)>

In /var/logs/messages I see the below messages. Seems like superset cannot find the athena driver, but only somtimes.

Steps to reproduce: Create a connection to Athena, create a dashboard with several panels based on athena queries. Refresh.

Jun 8 23:30:30 ip-10-0-2-225 superset-server: 2018-06-08 23:30:30,496:DEBUG:pyathenajdbc.cursor:SELECT table_name
Jun 8 23:30:30 ip-10-0-2-225 superset-server: FROM information_schema.tables
Jun 8 23:30:30 ip-10-0-2-225 superset-server: WHERE table_schema = 'kinesisiotdatalake'
Jun 8 23:30:30 ip-10-0-2-225 superset-server: 2018-06-08 23:30:30,498:INFO:root:SELECT "deviceparameter" AS "deviceparameter", count(deviceparameter) AS "count_deviceparameter"
Jun 8 23:30:30 ip-10-0-2-225 superset-server: FROM "kinesisiotdatalake"."ath_iot_raw_dl_prev_3_h_v" GROUP BY "deviceparameter" ORDER BY "count_deviceparameter" DESC
Jun 8 23:30:30 ip-10-0-2-225 superset-server: LIMIT 10000
Jun 8 23:30:30 ip-10-0-2-225 superset-server: 2018-06-08 23:30:30,504:INFO:root:Database.get_sqla_engine(). Masked URL: awsathena+jdbc://AKIAJPIHEH2BGF6VRH4A:XXXXXXXXXX@athena.us-west-2.amazonaws.com/kinesisiotdatalake?s3_staging_dir=s3://aws-athena-query-results-753350027352-us-west-2
Jun 8 23:30:30 ip-10-0-2-225 superset-server: 2018-06-08 23:30:30,508:DEBUG:botocore.session:Setting config variable for region to 'us-west-2'
Jun 8 23:30:30 ip-10-0-2-225 superset-server: 2018-06-08 23:30:30,508:DEBUG:botocore.session:Loading variable profile from defaults.
Jun 8 23:30:30 ip-10-0-2-225 superset-server: 2018-06-08 23:30:30,508:DEBUG:botocore.session:Loading variable region from instance vars with value 'us-west-2'.
Jun 8 23:30:30 ip-10-0-2-225 superset-server: 2018-06-08 23:30:30,512:DEBUG:botocore.session:Setting config variable for region to 'us-west-2'
Jun 8 23:30:30 ip-10-0-2-225 superset-server: 2018-06-08 23:30:30,512:DEBUG:botocore.session:Loading variable profile from defaults.
Jun 8 23:30:30 ip-10-0-2-225 superset-server: 2018-06-08 23:30:30,512:DEBUG:botocore.session:Loading variable region from instance vars with value 'us-west-2'.
Jun 8 23:30:30 ip-10-0-2-225 superset-server: 2018-06-08 23:30:30,514:ERROR:root:java.lang.RuntimeException: Class com.amazonaws.athena.jdbc.AthenaDriver not found
Jun 8 23:30:30 ip-10-0-2-225 superset-server: Traceback (most recent call last):
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib/python2.7/site-packages/superset/connectors/sqla/models.py", line 688, in query
Jun 8 23:30:30 ip-10-0-2-225 superset-server: df = self.database.get_df(sql, self.schema)
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib/python2.7/site-packages/superset/models/core.py", line 696, in get_df
Jun 8 23:30:30 ip-10-0-2-225 superset-server: df = pd.read_sql(sql, eng)
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib64/python2.7/site-packages/pandas/io/sql.py", line 416, in read_sql
Jun 8 23:30:30 ip-10-0-2-225 superset-server: chunksize=chunksize)
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib64/python2.7/site-packages/pandas/io/sql.py", line 1092, in read_query
Jun 8 23:30:30 ip-10-0-2-225 superset-server: result = self.execute(*args)
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib64/python2.7/site-packages/pandas/io/sql.py", line 983, in execute
Jun 8 23:30:30 ip-10-0-2-225 superset-server: return self.connectable.execute(*args, **kwargs)
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2074, in execute
Jun 8 23:30:30 ip-10-0-2-225 superset-server: connection = self.contextual_connect(close_with_result=True)
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2123, in contextual_connect
Jun 8 23:30:30 ip-10-0-2-225 superset-server: self._wrap_pool_connect(self.pool.connect, None),
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2158, in _wrap_pool_connect
Jun 8 23:30:30 ip-10-0-2-225 superset-server: return fn()
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 403, in connect
Jun 8 23:30:30 ip-10-0-2-225 superset-server: return _ConnectionFairy._checkout(self)
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 782, in _checkout
Jun 8 23:30:30 ip-10-0-2-225 superset-server: fairy = _ConnectionRecord.checkout(pool)
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 532, in checkout
Jun 8 23:30:30 ip-10-0-2-225 superset-server: rec = pool._do_get()
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1277, in _do_get
Jun 8 23:30:30 ip-10-0-2-225 superset-server: return self._create_connection()
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 350, in _create_connection
Jun 8 23:30:30 ip-10-0-2-225 superset-server: return _ConnectionRecord(self)
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 477, in init
Jun 8 23:30:30 ip-10-0-2-225 superset-server: self.__connect(first_connect_check=True)
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 667, in __connect
Jun 8 23:30:30 ip-10-0-2-225 superset-server: connection = pool._invoke_creator(self)
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 105, in connect
Jun 8 23:30:30 ip-10-0-2-225 superset-server: return dialect.connect(*cargs, **cparams)
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 410, in connect
Jun 8 23:30:30 ip-10-0-2-225 superset-server: return self.dbapi.connect(*cargs, **cparams)
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib/python2.7/site-packages/pyathenajdbc/init.py", line 66, in connect
Jun 8 23:30:30 ip-10-0-2-225 superset-server: driver_path, **kwargs)
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib/python2.7/site-packages/pyathenajdbc/connection.py", line 71, in init
Jun 8 23:30:30 ip-10-0-2-225 superset-server: jpype.JClass(ATHENA_DRIVER_CLASS_NAME)
Jun 8 23:30:30 ip-10-0-2-225 superset-server: File "/home/ec2-user/supersetagain/venv/local/lib64/python2.7/site-packages/jpype/_jclass.py", line 73, in JClass
Jun 8 23:30:30 ip-10-0-2-225 superset-server: raise _RUNTIMEEXCEPTION.PYEXC("Class %s not found" % name)
Jun 8 23:30:30 ip-10-0-2-225 superset-server: java.lang.RuntimeExceptionPyRaisable: java.lang.RuntimeException: Class com.amazonaws.athena.jdbc.AthenaDriver not found
Jun 8 23:30:30 ip-10-0-2-225 superset-server: 2018-06-08 23:30:30,531:DEBUG:root:#33[36m[stats_logger] (incr) queries#033[0m
Jun 8 23:30:31 ip-10-0-2-225 superset-server: 2018-06-08 23:30:31,022:DEBUG:botocore.session:Setting config variable for region to 'us-west-2'
Jun 8 23:30:31 ip-10-0-2-225 superset-server: 2018-06-08 23:30:31,022:DEBUG:botocore.session:Loading variable profile from defaults.
Jun 8 23:30:31 ip-10-0-2-225 superset-server: 2018-06-08 23:30:31,022:DEBUG:botocore.session:Loading variable region from instance vars with value 'us-west-2'.
Jun 8 23:30:31 ip-10-0-2-225 superset-server: 2018-06-08 23:30:31,039:DEBUG:pyathenajdbc.converter:{'REAL': 7, 'CLOB': 2005, 'TIME_WITH_TIMEZONE': 2013, 'LONGVARBINARY': -4, 'CHAR': 1, 'TINYINT': -6, 'BIGINT': -5, 'REF_CURSOR': 2012, 'DATE': 91, 'VARBINARY': -3, 'NULL': 0, 'REF': 2006, 'LONGNVARCHAR': -16, 'SMALLINT': 5, 'VARCHAR': 12, 'DISTINCT': 2001, 'NUMERIC': 2, 'NVARCHAR': -9, 'INTEGER': 4, 'ARRAY': 2003, 'JAVA_OBJECT': 2000, 'DATALINK': 70, 'SQLXML': 2009, 'DECIMAL': 3, 'OTHER': 1111, 'TIMESTAMP_WITH_TIMEZONE': 2014, 'BIT': -7, 'NCHAR': -15, 'BINARY': -2, 'STRUCT': 2002, 'DOUBLE': 8, 'LONGVARCHAR': -1, 'FLOAT': 6, 'BOOLEAN': 16, 'BLOB': 2004, 'ROWID': -8, 'TIME': 92, 'NCLOB': 2011, 'TIMESTAMP': 93}
Jun 8 23:30:31 ip-10-0-2-225 superset-server: 2018-06-08 23:30:31,039:DEBUG:pyathenajdbc.cursor:SELECT "deviceid" AS "deviceid",
Jun 8 23:30:31 ip-10-0-2-225 superset-server: COUNT(DISTINCT deviceparameter) AS "count_distinct_deviceparameter"
Jun 8 23:30:31 ip-10-0-2-225 superset-server: FROM "kinesisiotdatalake"."ath_iot_raw_dl_prev_3_h_v"
Jun 8 23:30:31 ip-10-0-2-225 superset-server: GROUP BY "deviceid"
Jun 8 23:30:31 ip-10-0-2-225 superset-server: ORDER BY "count_distinct_deviceparameter" DESC
Jun 8 23:30:31 ip-10-0-2-225 superset-server: LIMIT 10000
Jun 8 23:30:31 ip-10-0-2-225 superset-server: 2018-06-08 23:30:31,092:DEBUG:botocore.session:Setting config variable for region to 'us-west-2'
Jun 8 23:30:31 ip-10-0-2-225 superset-server: 2018-06-08 23:30:31,092:DEBUG:botocore.session:Loading variable profile from defaults.
Jun 8 23:30:31 ip-10-0-2-225 superset-server: 2018-06-08 23:30:31,092:DEBUG:botocore.session:Loading variable region from instance vars with value 'us-west-2'.
Jun 8 23:30:31 ip-10-0-2-225 superset-server: 2018-06-08 23:30:31,108:DEBUG:pyathenajdbc.converter:{'REAL': 7, 'CLOB': 2005, 'TIME_WITH_TIMEZONE': 2013, 'LONGVARBINARY': -4, 'CHAR': 1, 'TINYINT': -6, 'BIGINT': -5, 'REF_CURSOR': 2012, 'DATE': 91, 'VARBINARY': -3, 'NULL': 0, 'REF': 2006, 'LONGNVARCHAR': -16, 'SMALLINT': 5, 'VARCHAR': 12, 'DISTINCT': 2001, 'NUMERIC': 2, 'NVARCHAR': -9, 'INTEGER': 4, 'ARRAY': 2003, 'JAVA_OBJECT': 2000, 'DATALINK': 70, 'SQLXML': 2009, 'DECIMAL': 3, 'OTHER': 1111, 'TIMESTAMP_WITH_TIMEZONE': 2014, 'BIT': -7, 'NCHAR': -15, 'BINARY': -2, 'STRUCT': 2002, 'DOUBLE': 8, 'LONGVARCHAR': -1, 'FLOAT': 6, 'BOOLEAN': 16, 'BLOB': 2004, 'ROWID': -8, 'TIME': 92, 'NCLOB': 2011, 'TIMESTAMP': 93}

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.