Discussion:
Datanode can't connect namenode with kerberos
ZongtianHou
2018-11-09 07:28:34 UTC
Permalink
Hi, everyone
I set up kerberos for the hdfs cluster, but after I start name node, then the datanode, In the namenode log file, it display the following error:

2018-11-09 15:09:38,725 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user datanode
4870 2018-11-09 15:09:38,725 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 8020, call org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol.versionRequest from 127.0.0.1:56409 Call#132 Retry#0: org.apache.hadoop.security.AccessControlException: Access denied for user datanode. Superuser privilege is required

The kerberos auth for name node and data node are both ok, can anyone see the problem here, any hint will be very appreciated.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-***@hadoop.apache.org
For additional commands, e-mail: user-***@hadoop.apache.org
Harinder Singh
2018-11-09 07:35:02 UTC
Permalink
I think you need a superuser to start the service. Is datanode a superuser?

Regards
Harinder
Post by ZongtianHou
Hi, everyone
I set up kerberos for the hdfs cluster, but after I start name node, then
2018-11-09 15:09:38,725 WARN
org.apache.hadoop.security.UserGroupInformation: No groups available for
user datanode
4870 2018-11-09 15:09:38,725 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 0 on 8020, call
org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol.versionRequest from
org.apache.hadoop.security.AccessControlException: Access denied for user
datanode. Superuser privilege is required
The kerberos auth for name node and data node are both ok, can anyone see
the problem here, any hint will be very appreciated.
---------------------------------------------------------------------
Attila Bukor
2018-11-09 07:37:56 UTC
Permalink
All HDFS processes (NameNodes, DataNodes, JournalNodes, FailoverControllers) to
run as the same user, e.g. “hdfs”.
Post by Harinder Singh
I think you need a superuser to start the service. Is datanode a superuser?
Regards
Harinder
Hi, everyone
2018-11-09 15:09:38,725 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user datanode
4870 2018-11-09 15:09:38,725 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 8020, call org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol.versionRequest from 127.0.0.1:56409 <http://127.0.0.1:56409/> Call#132 Retry#0: org.apache.hadoop.security.AccessControlException: Access denied for user datanode. Superuser privilege is required
The kerberos auth for name node and data node are both ok, can anyone see the problem here, any hint will be very appreciated.
---------------------------------------------------------------------
ZongtianHou
2018-11-09 07:45:18 UTC
Permalink
I run all process with the same user. And it should be the superuser since it start the namenode, is there some configuration I need to do to let the datanode become superuser?
Post by Attila Bukor
All HDFS processes (NameNodes, DataNodes, JournalNodes, FailoverControllers) to
run as the same user, e.g. “hdfs”.
Post by Harinder Singh
I think you need a superuser to start the service. Is datanode a superuser?
Regards
Harinder
Hi, everyone
2018-11-09 15:09:38,725 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user datanode
4870 2018-11-09 15:09:38,725 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 8020, call org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol.versionRequest from 127.0.0.1:56409 <http://127.0.0.1:56409/> Call#132 Retry#0: org.apache.hadoop.security.AccessControlException: Access denied for user datanode. Superuser privilege is required
The kerberos auth for name node and data node are both ok, can anyone see the problem here, any hint will be very appreciated.
---------------------------------------------------------------------
ZongtianHou
2018-11-09 08:09:08 UTC
Permalink
And another wired thing, when I try mkdir in hdfs, it show the following error,
$hadoop dfs -mkdir /user
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

mkdir: Permission denied: user=kousouda, access=WRITE, inode="/":namenode:supergroup:drwxr-xr-x

why the owner of the root dir is namenode. Other than the user start the namenode.
Post by ZongtianHou
I run all process with the same user. And it should be the superuser since it start the namenode, is there some configuration I need to do to let the datanode become superuser?
Post by Attila Bukor
All HDFS processes (NameNodes, DataNodes, JournalNodes, FailoverControllers) to
run as the same user, e.g. “hdfs”.
Post by Harinder Singh
I think you need a superuser to start the service. Is datanode a superuser?
Regards
Harinder
Hi, everyone
2018-11-09 15:09:38,725 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user datanode
4870 2018-11-09 15:09:38,725 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 8020, call org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol.versionRequest from 127.0.0.1:56409 <http://127.0.0.1:56409/> Call#132 Retry#0: org.apache.hadoop.security.AccessControlException: Access denied for user datanode. Superuser privilege is required
The kerberos auth for name node and data node are both ok, can anyone see the problem here, any hint will be very appreciated.
---------------------------------------------------------------------
ZongtianHou
2018-11-09 09:31:10 UTC
Permalink
The problem seem to be here, when I start datanode, the dnUser here is datanode other than the user start it, does anyone know how to determine the user?


2018-11-09 17:25:00,706 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 50475
815960 2018-11-09 17:25:00,706 INFO org.mortbay.log: jetty-6.1.26
815961 2018-11-09 17:25:00,842 INFO org.mortbay.log: Started ***@0.0.0.0:50475
815962 2018-11-09 17:25:00,845 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnUserName = datanode
815963 2018-11-09 17:25:00,845 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: supergroup = supergroup
Post by ZongtianHou
And another wired thing, when I try mkdir in hdfs, it show the following error,
$hadoop dfs -mkdir /user
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
mkdir: Permission denied: user=kousouda, access=WRITE, inode="/":namenode:supergroup:drwxr-xr-x
why the owner of the root dir is namenode. Other than the user start the namenode.
Post by ZongtianHou
I run all process with the same user. And it should be the superuser since it start the namenode, is there some configuration I need to do to let the datanode become superuser?
Post by Attila Bukor
All HDFS processes (NameNodes, DataNodes, JournalNodes, FailoverControllers) to
run as the same user, e.g. “hdfs”.
Post by Harinder Singh
I think you need a superuser to start the service. Is datanode a superuser?
Regards
Harinder
Hi, everyone
2018-11-09 15:09:38,725 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user datanode
4870 2018-11-09 15:09:38,725 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 8020, call org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol.versionRequest from 127.0.0.1:56409 <http://127.0.0.1:56409/> Call#132 Retry#0: org.apache.hadoop.security.AccessControlException: Access denied for user datanode. Superuser privilege is required
The kerberos auth for name node and data node are both ok, can anyone see the problem here, any hint will be very appreciated.
---------------------------------------------------------------------
Loading...