Skip to content

Latest commit

 

History

History
27 lines (18 loc) · 605 Bytes

README.md

File metadata and controls

27 lines (18 loc) · 605 Bytes

Project structure

  • flume contains the flume script
  • twitter-streams-producer-agent gets data from twitter and writes it into MapR streams (runs as agent)
  • spark-streams-consumer is a spark streaming app which aggretates data and writes it into OpenTSDB

Generate IDE files

gradle idea

 or

gradle eclipse

Build with

gradle clean build distZip

Run twitter-streams-producer-agent <config file> for Twitter agent Run spark-submit myjarfile.jar for Spark streaming app

Start Flume: /opt/mapr/flume/flume-1.6.0/bin/flume-ng agent -f conf/twitter_disk.conf -n TwitterDisk