The file is gone, the space is not
The disk reports 54% used and 4.0G taken, but the directory you just cleared lists nothing at all. Both readings are correct. On Linux a file is only truly released when the last process holding it open lets go - until then the name is gone from the directory while the data stays on disk. Listing open files whose name is already deleted finds the holder, here one process sitting on 536870912 bytes that no longer has a name. Stopping that process, or restarting the service, returns the space immediately. This is why deleting more files does not help when a log was rotated out from under a running daemon.
The directory is empty. Half a gigabyte is still gone.
Your disk fills up. You find the big file, you delete it, and the usage does not move. So you delete more, and it still does not move.
The disk reports four gigabytes used, fifty-four percent. The directory you just cleared lists nothing at all. Both are true.
On Linux a file is only really gone when the last program using it lets go. Until then the name disappears but the data stays.
One command lists every open file whose name is already deleted.
There it is. One process, still holding half a gigabyte that no longer has a name. Stop that process, or restart the service, and the space comes back immediately.
Before you delete anything else, ask which program is still holding the old one.
#linux#security#selfhosted#devops#sysadmin
❯ Ask which program is still holding the file you deleted.