Default mapping from ElasticsearchPublisher (TABLE_INDEX_MAP) doesn't work for me #2157
Unanswered
vpirogovrep
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
i try to import my metadata from apache atlas into amundsen via ElasticsearchPublisher like this
job_config = ConfigFactory.from_dict({
'extractor.atlas_search_data.{}'.format(AtlasSearchDataExtractor.ATLAS_URL_CONFIG_KEY): atlas_host,
'extractor.atlas_search_data.{}'.format(AtlasSearchDataExtractor.ATLAS_PORT_CONFIG_KEY): 21000,
'extractor.atlas_search_data.{}'.format(AtlasSearchDataExtractor.ATLAS_VALIDATE_SSL_CONFIG_KEY): True,
'extractor.atlas_search_data.{}'.format(AtlasSearchDataExtractor.ATLAS_USERNAME_CONFIG_KEY): atlas_user,
'extractor.atlas_search_data.{}'.format(AtlasSearchDataExtractor.ATLAS_PASSWORD_CONFIG_KEY): atlas_password,
'extractor.atlas_search_data.{}'.format(AtlasSearchDataExtractor.ENTITY_TYPE_KEY): 'Table',
'loader.filesystem.elasticsearch.{}'.format(
FSElasticsearchJSONLoader.FILE_PATH_CONFIG_KEY): extracted_search_data_path,
'publisher.elasticsearch.{}'.format(ElasticsearchPublisher.FILE_PATH_CONFIG_KEY): extracted_search_data_path,
'publisher.elasticsearch.{}'.format(ElasticsearchPublisher.FILE_MODE_CONFIG_KEY): 'r',
'publisher.elasticsearch.{}'.format(
ElasticsearchPublisher.ELASTICSEARCH_CLIENT_CONFIG_KEY): elasticsearch_client,
'publisher.elasticsearch.{}'.format(
ElasticsearchPublisher.ELASTICSEARCH_NEW_INDEX_CONFIG_KEY): elasticsearch_new_index_key,
'publisher.elasticsearch.{}'.format(
ElasticsearchPublisher.ELASTICSEARCH_DOC_TYPE_CONFIG_KEY): elasticsearch_doc_type_key,
'publisher.elasticsearch.{}'.format(
ElasticsearchPublisher.ELASTICSEARCH_ALIAS_CONFIG_KEY): elasticsearch_index_alias,
'publisher.elasticsearch.{}'.format(
ElasticsearchPublisher.ELASTICSEARCH_MAPPING_CONFIG_KEY): TABLE_INDEX_MAP})
but face an error
elasticsearch.exceptions.RequestError: RequestError(400, 'mapper_parsing_exception', 'Root mapping definition has unsupported parameters: [schema : {analyzer=simple, type=text, fields={raw={type=keyword}}}] [cluster : {analyzer=simple, type=text, fields={raw={type=keyword}}}] [description : {analyzer=simple, type=text}] [display_name : {type=keyword}] [column_descriptions : {analyzer=simple, type=text}] [programmatic_descriptions : {analyzer=simple, type=text}] [tags : {type=keyword}] [badges : {type=keyword}] [database : {analyzer=simple, type=text, fields={raw={type=keyword}}}] [total_usage : {type=long}] [name : {analyzer=simple, type=text, fields={raw={type=keyword}}}] [last_updated_timestamp : {format=epoch_second, type=date}] [unique_usage : {type=long}] [column_names : {analyzer=simple, type=text, fields={raw={normalizer=column_names_normalizer, type=keyword}}}] [key : {type=keyword}]')
my json looks like this
{"badges": [],
"cluster": "",
"column_descriptions": [],
"column_names": ["version", "key", "value"],
"database": "system", "description": null, "display_name":
"default.test_low_cor", "key": "system://.default/test_low_cor",
"last_updated_timestamp": 1685559885, "name": "test_low_cor",
"programmatic_descriptions": [], "schema": "default", "schema_description": "", "tags": [], "total_usage": 0, "unique_usage": 0}
please help
thx
Beta Was this translation helpful? Give feedback.
All reactions