Stream real time Tweets of current affairs like covid-19 using Kafka 2.0.0 high throughput producer & consumer into Elasticsearch using safe, idempotent and compression configurations. Aggregate the data and use it for further analytics.

Setup a Apache Kafka on Linux using JAVA (Gradle). Setting up producers to produce data into Kafka from wikimedia and then setting up consumers and pulling data into Open Search.

This application will help you to find relevant information within your tweets. Besides a basic search, it offers you the possibility to perform advanced searches and to personalize the results according to the preferences of a set of predefined users.