Datanodes that start and then disappear

[Bernard Butler]

Hi,

I'm new to Hadoop - so hope you can help with this problem.

I'm trying to set up a small (2-zone) hadoop cluster on Solaris.
start-dfs.sh runs without error, e.g., it prints the following to the
screen:

master: starting datanode, logging to ..
slave: starting datanode, logging to ..

All looks well. However, when I check the DFS admin web page on the
master (on port 50070) it says

------------
Cluster Summary
1 files and directories, 0 blocks = 1 total. Heap Size is 18.5 MB / 448 MB
(4%)
Capacity : 0 KB
DFS Remaining : 0 KB
DFS Used : 0 KB
DFS Used% : 0 %
Live Nodes : 0
Dead Nodes : 0

There are no datanodes in the cluster
------------

I had a look in the datanode logs and they were empty on both master and
slave.

Running netstat -an on the master shows that it is listening on ports
50070, 50090 and 54310 (I changed fs.default.name to avoid a port
conflict). The slave has no hadoop-related ports active although there is
a single com.sun.management.jmxremote process running.

FYI a single-node pseudo-distributed installation worked fine on the
master. I'm running hadoop-0.17.1. I did not run start-mapred.sh.

Advice/suggestions would be very welcome.

Thanks,
B Butler

[Xavier Stevens]

Is there a way to un-blacklist a node without restarting hadoop?

Thanks,

-Xavier

[Arun C Murthy]

Xavier,

Post by Xavier Stevens
Is there a way to un-blacklist a node without restarting hadoop?

Which blacklist are you talking about? Per-job blacklist of
TaskTrackers? Hadoop Daemons?

Arun

[Xavier Stevens]

I don't think this is the per-job blacklist. The datanode is still
running but the tasktracker isn't on the slave machine. Can I just
"start-mapred" on that machine?

-Xavier


-----Original Message-----
From: Arun C Murthy [mailto:acm-ZXvpkYn067l8UrSeD/***@public.gmane.org]
Sent: Thursday, August 14, 2008 1:43 PM
To: core-user-7ArZoLwFLBtd/SJB6HiN2Ni2O/***@public.gmane.org
Subject: Re: Un-Blacklist Node

Xavier,

Post by Xavier Stevens
Is there a way to un-blacklist a node without restarting hadoop?

Which blacklist are you talking about? Per-job blacklist of
TaskTrackers? Hadoop Daemons?

Arun

[lohit]

One way I could think of is to just restart mapred daemons.
./bin/stop-mapred.sh
./bin/start-mapred.sh


Thanks,
Lohit

----- Original Message ----
From: Xavier Stevens <Xavier.Stevens-***@public.gmane.org>
To: core-user-7ArZoLwFLBtd/SJB6HiN2Ni2O/***@public.gmane.org
Sent: Thursday, August 14, 2008 12:18:18 PM
Subject: Un-Blacklist Node

Is there a way to un-blacklist a node without restarting hadoop?

Thanks,

-Xavier