Yeah, I assumed if they eat, they drink. And there was a lot of sheet sent to the Void, so if it's frequent enough, sustainable food and drink source isn't too much of an issue.
And it wouldn't really be something that they needed to show on screen but it wouldn't have hurt to see them trying to hunt down some food. Ultimately it's fine they didn't even address it since Loki and Sylvie weren't there for long.