Surviving — and Preventing — “Root Filesystem Full” Disasters on RHEL 9 (with ESXi & LVM)

“Nothing focuses the mind like a VM that refuses to boot because / is 100 % full.”
— Every systems administrator, at least once

Running Red Hat Enterprise Linux 9 under VMware is wonderfully boring — right up to the morning the guest won’t finish booting because the root filesystem ran out of space.
This article walks through:

1. Why a full / stops the boot process.
2. Two fast recovery paths (same‑disk resize or add‑a‑second‑disk).
3. A surgical “make room now” approach when you just need a few MB to get past the emergency shell.
4. Hard‑won prevention tips so it never happens again.

Everything here assumes the default RHEL 9 layout on ESXi:

/dev/sda1   EFI, 600 MiB   (vfat)
/dev/sda2   /boot, 1 GiB   (xfs)
/dev/sda3   LVM PV         → rhel/root (xfs) + rhel/swap

1 Why an out‑of‑space root kills the boot

• dracut hands control to systemd only after it can switch‑root and populate /run, /dev, and various state files. All of those live on the root filesystem—even during early‑boot.
• Once systemd starts and hits the tiny journal files it needs to write, any ENOSPC (no space left on device) triggers the emergency shell.
• If you’re on XFS, you can’t mount read‑only and…