I think the Kents and Gloucesters have already been downsized into "cottages".

Edward's lease is periodically reviewed, and his rent adjusted accordingly, and Anne owns her home. It's possible that Charles will try to turf them out the suite of rooms they use when they have engagements in London. If he does so, they may decide that Charles and Camilla can take over their London commitments.