Question 214841 in Ubuntu Question posted in
The official Ubuntu documentation can be found here.

Ubuntu – Is there a way for fixing "ERROR: Cannot set priority of datanode process…" when start-dfs.sh?

I am installing apache hadoop 3.2.4, 4 nodes cluster with Java 8 on Ubuntu machines.

3 machines have 16GB RAM and 1 have 8GB RAM. All of them are 16GB CPU.

I have done most of the configuration but having trouble while getting datanode to start. This is the error:

hadoop@rai-lab-hdwk-01:~$ start-dfs.sh
Starting namenodes on [rai-lab-hdwk-01]
Starting datanodes
rai-lab-hapo-01: ERROR: Cannot set priority of datanode process 385305
rai-lab-hdwk-02: ERROR: Cannot set priority of datanode process 190218
rai-lab-hdwk-03: ERROR: Cannot set priority of datanode process 210343
Starting secondary namenodes [rai-lab-hdwk-01]
hadoop@rai-lab-hdwk-01:~$ jps
185158 NameNode
185551 Jps
hadoop@rai-lab-hdwk-01:~$

Can you help me fix this?

I have tried

sudo chown -R hadoop:hadoop /usr/local/hadoop/etc/hadoop/*

and

chmod -R 777 /usr/local/hadoop/etc/hadoop/*

to give read, write execute permissions but still hasn’t worked.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I have completed the installation succesfully and ran a mapreduce jobs. I had a lot to fix and thanks to the help of ChatGPT I was able to do it. Here is what I had to fix:

    1. yarn-site.xml had errors. I had to fix the sintax errors on this file.
    2. namenode cluster id and datanode cluster id had different ids. Since I reinstalled hadoop on the namenode, it started generating different cluster ids. So I had to manually make it the same. I deleted all the metadata and cluster ids from the datanodes. I did hdfs namenode format then I copied the cluster ID it generated and I manually put it in the hdfs-site.xml file to match
    3. A few other bugs I was able to find with logs and chatgpt.

  2. 0

    Could you please provide the log of the datanode? It can be found in the ${HADOOP_HOME}/logs directory.

    I suspect that the error may be due to the datanode not having the necessary permissions to write to the directory.
    ex.dfs.datanode.data.dir

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search