MapReduce运行异常:Unknown protocol to name node: org.apache.hadoop.mapred.JobSubmissionProtocol
描述:job 配置错误
异常:
14/03/26 22:23:27 ERROR security.UserGroupInformation: PriviledgedActionException as:allen cause:org.apache.hadoop.ipc.RemoteException: java.io.IOException: Unknown protocol to name node: org.apache.hadoop.mapred.JobSubmissionProtocol
at org.apache.hadoop.hdfs.server.namenode.NameNode.getProtocolVersion(NameNode.java:149)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
Exception in thread "main" org.apache.hadoop.ipc.RemoteException: java.io.IOException: Unknown protocol to name node: org.apache.hadoop.mapred.JobSubmissionProtocol
at org.apache.hadoop.hdfs.server.namenode.NameNode.getProtocolVersion(NameNode.java:149)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
解决办法:
conf.set("mapred.job.tracker", "ip:端口");
端口和地址在:mapred-site.xml中
<property>
<name>mapred.job.tracker</name>
<value>ip:端口</value>
</property>
如果还无法解决:
注释掉conf.set("mapred.job.tracker", "ip:端口");
然后配置输入输出目录时,配上hdfs的路径
String[] ioArgs = new String[] { "hdfs://ip:端口/mr/sort/sort_in", "hdfs://ip:端口/mr/sort/sort_out" };
相关推荐
org.apache.hadoop.mapreduce.server.jobtracker org.apache.hadoop.mapreduce.server.tasktracker org.apache.hadoop.mapreduce.tools org.apache.hadoop.mapreduce.v2 org.apache.hadoop.mapreduce.v2.app....
包org.apache.hadoop.mapreduce的Hadoop源代码分析
org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected at org.apache...
标签:apache、mapreduce、common、client、hadoop、jar包、java、API文档、中英对照版; 使用方法:解压翻译后的API文档,用浏览器打开“index.html”文件,即可纵览文档内容。 人性化翻译,文档中的代码和结构保持...
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:133) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:437) at org.apache....
【SpringBoot】Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster报错明细问题解决后记 报错明细 IDEA SpringBoot集成hadoop运行环境,,本地启动项目,GET请求接口触发...
Maven坐标:org.apache.hadoop:hadoop-mapreduce-client-core:2.5.1; 标签:core、apache、mapreduce、client、hadoop、jar包、java、API文档、中文版; 使用方法:解压翻译后的API文档,用浏览器打开“index.html...
我在进行MapReduce本地测试的时候,遇见下面的报错: Exception in thread “main” java. lang . UnsatisfiedLinkError: org. apache . hadoop. io. nativeio. NativeIOSWindows . access0 (Ijava...
Hadoop源码 包含mapred
at org.apache.sqoop.mapreduce.JobBase.putSqoopOptionsToConfiguration(JobBase.java:392) at org.apache.sqoop.mapreduce.JobBase.createJob(JobBase.java:378) at org.apache.sqoop.mapreduce.ImportJobBase....
本节介绍如何编写基本的 MapReduce 程序实现数据分析。本节代码是基于 Hadoop 2.7.3 开发的。 任务准备 单词计数(WordCount)的任务是对一组输入文档中的单词进行分别计数。假设文件的量比较大,每个文档又包含...
标签:apache、mapreduce、jobclient、client、hadoop、jar包、java、中文文档; 使用方法:解压翻译后的API文档,用浏览器打开“index.html”文件,即可纵览文档内容。 人性化翻译,文档中的代码和结
本文介绍了用Java编写并运行第一个mapreduce作业的步骤及遇到的问题和解决方案。
IDEA中通过Java的API操作MapReducer报错org.apache.hadoop.io.nativeio.NativeIO$Windows...的解决办法(进来看一下)-附件资源
概念 MapReduce是hadoop分布式计算框架。 MapReduce意味着在计算过程中实际分为两大步,...import org.apache.hadoop.mapreduce.Mapper; // 案例:统计每一个单词出现的次数 // KEYIN - 这一行的偏移量 --- // VALUEIN
对应Maven信息:groupId:org.apache.hadoop,artifactId:hadoop-mapreduce-client-core,version:2.6.5 使用方法:解压翻译后的API文档,用浏览器打开“index.html”文件,即可纵览文档内容。 人性化翻译,文档...
Using the old API – org.apache.hadoop.mapred Going real — network monitoring data Getting and understanding the data Knowing the problems Solution approaches Approach 1 – Preaggregate the ...
hadoop mapreduce测试样例,文档,源码
Explore different approaches to making Solr work on big data ecosystems besides Apache Hadoop Improve search performance while working with big data A practical guide that covers interesting, real-...
赠送jar包:hadoop-mapreduce-...标签:apache、mapreduce、jobclient、client、hadoop、jar包、java、API文档、中英对照版; 使用方法:解压翻译后的API文档,用浏览器打开“index.html”文件,即可纵览文档内容。 人