Withdrawals can be orchestrated by someone in the system with a balance. Storing funds on a server, should ideally be avoided, as it will make the server a target for hackers. A hot wallet may be a solution but suffers from a similar limitation. A more optimal solution, would be to have withdrawal requests sent to trusted notaries, who then verify the size and the request. After verification, the notary can process the withdrawal based on cold storage, or a hardware wallet.
Withdrawals happen in the reverse way to deposits. However, it requires the owner of the HD wallet to process via cold storage. A fee market for competitive withdrawals can emerge, or it could be a community service. Ultimately it should become more cost effective to spend funds via solid pay, than withdrawing to lower layers.