GitHub - ella33/hadoopRepo: University project: inverted index developed in hadoop.

ella33 / hadoopRepo Public

Notifications You must be signed in to change notification settings
Fork 0
Star 0

University project: inverted index developed in hadoop.

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 6 Commits
src/main/java/org/apache/hadoop/examples		src/main/java/org/apache/hadoop/examples
target		target
README		README
pom.xml		pom.xml

Repository files navigation

------------------------
HDInsight Hadoop cluster
------------------------
SSH Login:

Run job:
a.yarn jar InvertedIndex-1.0-SNAPSHOT.jar org.apache.hadoop.examples.InvertedIndex /user/root/smallinput /user/root/output/iiout /user/root/smallinput/stop-words.txt
b. yarn jar InvertedIndex-1.0-SNAPSHOT.jar org.apache.hadoop.examples.InvertedIndex /user/root/input /user/root/output/iiout /user/root/input/stop-words.txt

See results:
hdfs dfs -cat /user/root/output/iiout/*

Remove output results:
hadoop fs -rm -r /user/root/output/iiout

----------------------
Command Line Arguments
----------------------

Example: 
(1) - input path
(2) - output path 
(3) - stop words file