-
Notifications
You must be signed in to change notification settings - Fork 0
crail
Animesh Trivedi edited this page Jun 13, 2017
·
2 revisions
Crail (www.crail.io) specific notes
What to do when this happens
~/hadoop-2.7.3$ ./bin/hadoop jar ~/HdfsBench-1.0.jar readRandomHeap $((128*1024*1024)) $((8*32)) crail://flex11-40g0:9060/sql/local-32gb
Picked up JAVA_TOOL_OPTIONS: -XX:+PreserveFramePointer
reading random file in heap mode crail://flex11-40g0:9060/sql/local-32gb
java.lang.IllegalArgumentException: Wrong FS: crail://flex11-40g0:9060/sql/local-32gb, expected: hdfs://flex11-40g0:9000
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:648)
at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:194)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:106)
at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305)
at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301)
at com.github.animeshtrivedi.HdfsIOBenchmark.HdfsIOBenchmark.readRandomHeap(HdfsIOBenchmark.java:281)
at com.github.animeshtrivedi.HdfsIOBenchmark.HdfsIOBenchmark.run(HdfsIOBenchmark.java:71)
at com.github.animeshtrivedi.HdfsIOBenchmark.HdfsIOBenchmark.main(HdfsIOBenchmark.java:483)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Solution:
Run either by making crail the default file system in the core-site.xml file or run natively with the classpath as
java -cp /home/demo/HdfsBench-1.0.jar:/home/demo/crail-deployment/hadoop-2.7.3/share/hadoop/common/lib/*:/home/demo/crail-deployment/hadoop-2.7.3/share/hadoop/common/*:/home/demo/crail-deployment/hadoop-2.7.3/share/hadoop/hdfs:/home/demo/crail-deployment/hadoop-2.7.3/share/hadoop/hdfs/lib/*:/home/demo/crail-deployment/hadoop-2.7.3/share/hadoop/hdfs/*:/home/demo/crail-deployment/hadoop-2.7.3/share/hadoop/yarn/lib/*:/home/demo/crail-deployment/hadoop-2.7.3/share/hadoop/yarn/*:/home/demo/crail-deployment/hadoop-2.7.3/share/hadoop/mapreduce/lib/*:/home/demo/crail-deployment/hadoop-2.7.3/share/hadoop/mapreduce/*:/home/demo/crail-deployment/hadoop//contrib/capacity-scheduler/*.jar:/home/demo/crail-deployment/crail//conf/:/home/demo/crail-deployment/crail//jars/* com.github.animeshtrivedi.HdfsIOBenchmark.HdfsIOBenchmark readRandomHeap 134217728 256 crail://flex11-40g0:9060/sql/local-32gb