Find where inodes are being used

So I received a warning from our monitoring system on one of our boxes that the number of free inodes on a filesystem was getting low.

df -i output shows this:

Filesystem       Inodes  IUsed    IFree IUse% Mounted on
/dev/xvda1       524288 422613   101675   81% /

As you can see, the root partition has 81% of its inodes used.

 

basically an inode is used for each file on the filesystem. So running out of inodes generally means you’ve got a lot of small files laying around. So the question really becomes, “what directory has a large number of files in it?”

In this case, the filesystem we care about is the root filesystem /, so we can use the following command:

find / -xdev -printf '%h\n' | sort | uniq -c | sort -k 1 -n

This will dump a list of every directory on the filesystem prefixed with the number of files (and subdirectories) in that directory. Thus the directory with the largest number of files will be at the bottom.

In my case, this turns up the following:

   1202 /usr/share/man/man1
   2714 /usr/share/man/man3
   2826 /var/lib/dpkg/info
 306588 /var/spool/postfix/maildrop

So basically /var/spool/postfix/maildrop is consuming all the inodes.

 

 

For example, the most upvoted answer is this one:

for i in `find . -type d `; do echo `ls -a $i | wc -l` $i; done | sort -n

If we change this instead to

for i in `find . -xdev -type d `; do echo `ls -a $i | wc -l` $i; done | sort -n

even though /mnt/foo is a mount, it is also a directory on the root filesystem, so it’ll turn up in find . -mount -type d, and then it’ll get passed to the ls -a $i, which will dive into the mount.

The find in my answer instead lists the directory of every single file on the mount. So basically with a file structure such as:

/foo/bar
/foo/baz
/pop/tart

we end up with

/foo
/foo
/pop

So we just have to count the number of duplicate lines.

 

this is reposted from here at the asker’s behest:

du --inodes -S | sort -rh | sed -n \
        '1,50{/^.\{71\}/s/^\(.\{30\}\).*\(.\{37\}\)$/\1...\2/;p}'

And if you want to stay in the same filesystem you do:

du --inodes -xS

Here’s some example output:

15K     /usr/share/man/man3
4.0K    /usr/lib
3.6K    /usr/bin
2.4K    /usr/share/man/man1
1.9K    /usr/share/fonts/75dpi
...
519     /usr/lib/python2.7/site-packages/bzrlib
516     /usr/include/KDE
498     /usr/include/qt/QtCore
487     /usr/lib/modules/3.13.6-2-MANJARO/build/include/config
484     /usr/src/linux-3.12.14-2-MANJARO/include/config

NOW WITH LS:

Several people mentioned they do not have up-to-date coreutils and the –inodes option is not available to them. So, here’s ls:

ls ~/test -AiR1U | 
sed -rn '/^[./]/{h;n;};G;
    s|^ *([0-9][0-9]*)[^0-9][^/]*([~./].*):|\1:\2|p' | 
sort -t : -uk1.1,1n |
cut -d: -f2 | sort -V |
uniq -c |sort -rn | head -n10

If you’re curious, the heart-and-soul of that tedious bit of regex there is replacing the filename in each of ls's recursive search results with the directory name in which it was found. From there it’s just a matter of squeezing repeated inode numbers then counting repeated directory names and sorting accordingly.

The -U option is especially helpful with the sorting in that it specifically does not sort, and instead presents the directory list in original order – or, in other words, by inode number.

And of course -1 is incredibly helpful in that it ensures a single result per line, regardless of possibly included newlines in filenames or other spectacularly unfortunate problems that might occur when you attempt to parse a list.

And of course -A for all and -i for inode and -R for recursive and that’s the long and short of it.

The underlying method to this is that I replace every one of ls’s filenames with its containing directory name in sed. Following on from that… Well, I’m a little fuzzy myself. I’m fairly certain it’s accurately counting the files, as you can see here:

% _ls_i ~/test
> 100 /home/mikeserv/test/realdir
>   2 /home/mikeserv/test
>   1 /home/mikeserv/test/linkdir

This is providing me pretty much identical results to the du command:

DU:

15K     /usr/share/man/man3
4.0K    /usr/lib
3.6K    /usr/bin
2.4K    /usr/share/man/man1
1.9K    /usr/share/fonts/75dpi
1.9K    /usr/share/fonts/100dpi
1.9K    /usr/share/doc/arch-wiki-markdown
1.6K    /usr/share/fonts/TTF
1.6K    /usr/share/dolphin-emu/sys/GameSettings
1.6K    /usr/share/doc/efl/html

LS:

14686   /usr/share/man/man3:
4322    /usr/lib:
3653    /usr/bin:
2457    /usr/share/man/man1:
1897    /usr/share/fonts/100dpi:
1897    /usr/share/fonts/75dpi:
1890    /usr/share/doc/arch-wiki-markdown:
1613    /usr/include:
1575    /usr/share/doc/efl/html:
1556    /usr/share/dolphin-emu/sys/GameSettings:

I think the include thing just depends on which directory the program looks at first – because they’re the same files and hardlinked. Kinda like the thing above. I could be wrong about that though – and I welcome correction…

DU DEMO

% du --version
> du (GNU coreutils) 8.22

Make a test directory:

% mkdir ~/test ; cd ~/test
% du --inodes -S
> 1       .

Some children directories:

% mkdir ./realdir ./linkdir
% du --inodes -S
> 1       ./realdir
> 1       ./linkdir
> 1       .

Make some files:

% printf 'touch ./realdir/file%s\n' `seq 1 100` | . /dev/stdin
% du --inodes -S
> 101     ./realdir
> 1       ./linkdir
> 1       .

Some hardlinks:

% printf 'n="%s" ; ln ./realdir/file$n ./linkdir/link$n\n' `seq 1 100` | 
    . /dev/stdin
% du --inodes -S
> 101     ./realdir
> 1       ./linkdir
> 1       .

Look at the hardlinks:

% cd ./linkdir
% du --inodes -S
> 101

% cd ../realdir
% du --inodes -S
> 101

They’re counted alone, but go one directory up…

% cd ..
% du --inodes -S
> 101     ./realdir
> 1       ./linkdir
> 1       .

Then I ran my ran script from below and:

> 100     /home/mikeserv/test/realdir
> 100     /home/mikeserv/test/linkdir
> 2       /home/mikeserv/test

And Graeme’s:

> 101 ./realdir
> 101 ./linkdir
> 3 ./

So I think this shows that the only way to count inodes is by inode. And because counting files means counting inodes, you cannot doubly count inodes – to count files accurately inodes cannot be counted more than once.