Yarn reporting nodes are full even though they have free space

Question

I have installed Hadoop and I'm trying to use it as a Pseudo-Distributed system in my own machine. I have also set up Yarn as the resource manager. However, when I try to run any job, I can see that it is stuck as Accepted. Checking the status of my only node tells me that it is Unhealthy with the following message:

1/1 local-dirs usable space is below configured utilization percentage/no more usable space [ /tmp/hadoop-usr/nm-local-dir : used space above threshold of 90.0% ] ; 1/1 log-dirs usable space is below configured utilization percentage/no more usable space [ /Users/usr/Documents/hadoop-3.3.1/logs/userlogs : used space above threshold of 90.0% ]

From what I understand, the issue is that the disk associated to the specific node is over 90% filled. However, when I check my log directory, I can see that it is empty so why would its usage be above the threshold? Also, I can see that nm-local-dir has 0B of usage. I would appreciate any help with this issue.

Answer 1

I figured out my issue. All the responses that I found mentioned that the issue was that there was not enough space in the disk associated to the DataNode. In my case, my disk is the one on my Mac. Therefore, to verify if I didn't have any space, I checked on the Storage section in the About this Mac menu. According to this, I had plenty of space in my disk. However, when I went to check on Disk utilities, it turns out that most of this free space was purgeable. My guess is that HDFS does not see this purgeable space as free space and it reports it as being occupied. I ended up restarting my Mac and this freed up some of the purgeable space enough for HDFS to finally see enough free space to run my jobs.

While this fixed my issue, I am curious as to what other solutions could be. I know I can always increase the percentage disk utilization limit but I'm wondering if I can just create a new partition and associate it to my HDFS system? I would appreciate any guidance regarding this.