Cannot Migrate DB after upgrade to 2.1.0

[tosto92]

Cannot Migrate DB after upgrade to 2.1.0

Request Type

Bug

Work Environment

Question	Answer
OS version (server)	Ubuntu
OS version (client)	16.04
Cortex version / git hash	2.1.0
Package Type	DEB
Elasticsearch Version	6.4.1

Problem Description

After the upgrade from 2.0.4 to 2.1.0 I can't update the DB.
When I click the button on the web interface I have a red notification in the bottom-left corner with "UserMgmtCtrl".
In cortex Logs I have

2018-09-27 10:19:41,035 [ERROR] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-24 - Migration fail org.elasticsearch.transport.RemoteTransportException: [node-1][127.0.0.1:9300][indices:admin/create] Caused by: java.lang.IllegalArgumentException: Rejecting mapping update to [cortex_2] as the final mapping would have more than 1 type: [artifact, dblist, data, audit, organization, report, job, worker, workerConfig, user] at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:407) at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:355) at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:279) at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$IndexCreationTask.execute(MetaDataCreateIndexService.java:461) at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:45) at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:639) at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:268) at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:198) at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:133) at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:624) at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:244) at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:207) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2018-09-27 10:19:41,036 [INFO] from org.thp.cortex.services.ErrorHandler in application-akka.actor.default-dispatcher-24 - POST /api/maintenance/migrate returned 400 org.elasticsearch.transport.RemoteTransportException: [node-1][127.0.0.1:9300][indices:admin/create] Caused by: java.lang.IllegalArgumentException: Rejecting mapping update to [cortex_2] as the final mapping would have more than 1 type: [artifact, dblist, data, audit, organization, report, job, worker, workerConfig, user] at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:407) at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:355) at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:279) at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$IndexCreationTask.execute(MetaDataCreateIndexService.java:461) at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:45) at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:639) at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:268) at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:198) at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:133) at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:624) at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:244) at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:207) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Update

I've added the Elasticsearch version (6.4.1).

[chrissommer]

We got the exact same problem - also with Ubuntu 16.04 and DEB Package.
When I try to Update, I receive the Notifcation "UserMgmtCtrl"

[nadouani]

Cortex is only compatible with ES 5.x, not 6.

[chrissommer]

Hi,

we are Running 5.6.12 - not 6.x

BR
Chris

[nadouani]

Oh sorry @csommerkapsch I was answering to @tosto92 who has updated his ES to 6.4.1.

Do you have the same logs?

[chrissommer]

I clicked the Update Button again, to get fresh Logs, still got the Error in the Corner, but now Cortex started migrating ...

I will wait for it to finish, and get back to you.

[chrissommer]

OK Cortex seems to work fine now - but in the Observable Overview all my Reports are gone.
Even if I rerun the Analyzers, the Reports are only shown inside the Observable Tab, not in the Overview.

Shall I Open a New Case @ TheHive for this?

BR
Chris

[tosto92]

I reverted to a snapshot with the correct ES version and now it works.

I've got the same problem on TheHive, the Reports seems gone, but if I click the observables I can see them.

but I think this is a TheHive problem so maybe we can open a new issue (or reopen this in the correct repository.

[To-om]

The mini-report problem (which is not related to Cortex by TheHive) is fixed by TheHive-Project/elastic4play#70.