Super User Asked by bharathyes on November 20, 2021
I am trying to run hadoop 3.2.0 on Ubuntu 18.10 running as a VM over Win10.
Want to execute a sample word count program to verify that the installation was successful and that hadoop has been properly configured.
When I try to run the included example jar by using command:
bharath@ubuntu:/usr/share/hadoop$ bin/hadoop jar
share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.0.jar grep input
output ‘dfs[a-z.]+’
I get this on the terminal:
2019-05-05 11:51:55,667 INFO impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties
2019-05-05 11:51:55,873 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
2019-05-05 11:51:55,873 INFO impl.MetricsSystemImpl: JobTracker metrics system started
2019-05-05 11:51:56,020 INFO mapreduce.JobSubmitter: Cleaning up the staging area file:/tmp/hadoop/mapred/staging/bharath705403707/.staging/job_local705403707_0001
ENOENT: No such file or directory
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:234)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:861)
at org.apache.hadoop.fs.ChecksumFileSystem$1.apply(ChecksumFileSystem.java:508)
at org.apache.hadoop.fs.ChecksumFileSystem$FsOperation.run(ChecksumFileSystem.java:489)
at org.apache.hadoop.fs.ChecksumFileSystem.setPermission(ChecksumFileSystem.java:511)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:676)
at org.apache.hadoop.mapreduce.JobResourceUploader.mkdirs(JobResourceUploader.java:658)
at org.apache.hadoop.mapreduce.JobResourceUploader.uploadResourcesInternal(JobResourceUploader.java:172)
at org.apache.hadoop.mapreduce.JobResourceUploader.uploadResources(JobResourceUploader.java:133)
at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:99)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:194)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1570)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1567)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1567)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1588)
at org.apache.hadoop.examples.Grep.run(Grep.java:78)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.examples.Grep.main(Grep.java:103)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
I am new to hadoop and even Java so I am unable to make much sense of what any of this means other than the fact that the program isn’t working and that certain files might be missing.
Can anyone point out what the issue is here or make suggestions to find what is causing this issue?
The key is that the input file location is on hdfs not on your local hard (on your computer). Thus, the address you entered is wrong and you should: First put the input file on hdfs with its command. Second run the right command to execute that file. BR, Neda
Answered by Neda Maleki on November 20, 2021
There seems to be an error with the command. Syntax is: hadoop jar jar-file class-with-Main input-file-hdfs output-dir-hdfs
Output dir cannot pre exist
Ex:
hadoop jar /opt/mapr/hadoop/hadoop-0.20.2/hadoop-0.20.2-dev-examples.jar wordcount /myvolume/in /myvolume/out
Answered by Suyash Limaye on November 20, 2021
Get help from others!
Recent Questions
Recent Answers
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP