Strictly speaking, it's the Waymo Driver, the company's automated driving system, that's classified as Level 4. Waymo's published safety research describes it as an SAE Level 4 automated driving system, and that classification explains why an empty driver's seat doesn't automatically make a Waymo Level 5.
There are a total of six SAE automation levels, numbered 0 through 5. The National Highway Traffic Safety Administration (NHTSA) starts its breakdown at Level 0, where the human does all the driving even if safety systems can briefly intervene. Level 1 can continuously assist with steering or acceleration and braking, while Level 2 can handle both at the same time. In each case, the driver remains responsible for the car.
Level 3 changes who's doing the driving. The system can handle the entire driving task under certain conditions, but the human must be available to take over when requested. At Level 4, that requirement disappears. Once the automated system is operating within its intended conditions, the occupants are passengers rather than backup drivers.
This is also why Waymo can describe its service as fully autonomous without claiming SAE Level 5. Its cars can complete rides without anyone sitting behind the wheel, but that ability still has defined operating limits.