The deposit sits in a brokerage or lawyer's trust account, not in the seller's pocket, specifically so neither side can access it unilaterally while a dispute is unresolved. Releasing it requires both parties' agreement or a legal ruling, not just one side deciding they are owed it.
Beyond losing the deposit, walking away after conditions are removed without a valid contractual reason can expose a buyer to a further claim for damages if the seller resells for less, since the contract is binding at that point, not optional. Reviewing condition dates with a lawyer before removing them typically costs far less than the $800 to $1,500 that lawyer charges for the whole file.
Joel Dyck makes sure a buyer understands exactly which conditions are still open and what removing each one actually commits them to before they sign off. Get your hand-reviewed valuation from Joel Dyck.