The tokens are held by the bridge on the L1 Ethereum network from the time they are bridged over to Optimism, until they are claimed from the bridge after the fault proof window (seven days at present) passes.
So tokens that are “in limbo” during withdrawal are on the L1 Ethereum network. Their Optimism equivalents have already been burned, but they can’t be claimed yet because of the fault challenge window.