The issue here is that switching and resigning councillors through the Appointment menu doesn't apply the 'Fired from Council' opinion modifier, which disallows you rehiring them for a few years. With it active it's way more risky to do the 'council shuffle', as you might trap yourself with powerful vassals who can't be on the council. This issue is recorded in the database and will be fixed in an upcoming patch.