从 windows eclipse 运行 Mapreduce(yarn) 时出错

Error while running Mapreduce(yarn)from windows eclipse(从 windows eclipse 运行 Mapreduce(yarn) 时出错)
本文介绍了从 windows eclipse 运行 Mapreduce(yarn) 时出错的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着跟版网的小编来一起学习吧!

问题描述

限时送ChatGPT账号..

我正在从我的 Eclipse 中运行 WordCount 程序.我尝试使用 Hadoop1.x 运行良好.在 hadoop2.x 上运行时遇到问题

I am running a WordCount program from my eclipse. I tried with Hadoop1.x it is running fine. Facing issue while running on hadoop2.x

我试过了1)将所有 xml 添加到我的类路径中.2)也尝试了conf.set(),在conf对象中设置xml属性.

i tried 1)added all xml into my classpath. 2)also tried conf.set(), setting xml properties in conf object.

还在日志中显示:-没有可用于容器 container_1394042163908_0573_01_000001 的日志

Also in logs it says :-No logs available for container container_1394042163908_0573_01_000001

  Application application_1394042163908_0573 failed 2 times due to AM Container for      appattempt_1394042163908_0573_000002 exited with exitCode: 1 due to: Exception from container-launch:
org.apache.hadoop.util.Shell$ExitCodeException: /bin/bash: line 0: fg: no job control
at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
at org.apache.hadoop.util.Shell.run(Shell.java:379)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerEx    ecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)`enter code here`
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
.Failing this attempt.. Failing the application.

推荐答案

我已经解决了这个问题,有一个jira可以解决它:https://issues.apache.org/jira/browse/MAPREDUCE-5655

I have figured out the issue, there is a jira for it:https://issues.apache.org/jira/browse/MAPREDUCE-5655

我刚刚添加了 YARNRunner.java &MRApps.java 到我的项目.它还需要在 windows 框上的 mapred-site.xml 中添加以下属性,以便作业启动器知道作业运行器将是一个 linux:

I just added YARNRunner.java & MRApps.java to my project . It also reqires to add the following property to mapred-site.xml on the windows box, so that the job launcher knows, that the job runner will be a linux:

<property>
<name>mapred.remote.os</name>
<value>Linux</value>
<description>Remote MapReduce framework's OS, can be either Linux or Windows</description>
</property>

而且 MapReduce 现在运行良好.

And MapReduce is running fine now.

同时将 org.apache.hadoop.util.Shell.java 复制到您的项目中.

Also Copy org.apache.hadoop.util.Shell.java into your project.

您可以注释掉下面这行,以消除 winutils.exe 错误.
throw new IOException("在 Hadoop 二进制文件中找不到可执行文件 " + fullExeName + ".");

You can comment out the below line,to remove the winutils.exe Error.
throw new IOException("Could not locate executable " + fullExeName + " in the Hadoop binaries.");

这篇关于从 windows eclipse 运行 Mapreduce(yarn) 时出错的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持跟版网!

本站部分内容来源互联网,如果有图片或者内容侵犯了您的权益,请联系我们,我们会在确认后第一时间进行删除!

相关文档推荐

Sending a keyboard event from java to any application (on-screen-keyboard)(将键盘事件从 java 发送到任何应用程序(屏幕键盘))
How to make JComboBox selected item not changed when scrolling through its popuplist using keyboard(使用键盘滚动其弹出列表时如何使 JComboBox 所选项目不更改)
Capturing keystrokes without focus(在没有焦点的情况下捕获击键)
How can I position a layout right above the android on-screen keyboard?(如何将布局放置在 android 屏幕键盘的正上方?)
How to check for key being held down on startup in Java(如何检查在Java中启动时按住的键)
Android - Get keyboard key press(Android - 获取键盘按键)