Also, in CK1 your realm could get invaded by a massive foreign army, and a couple of lords within your realm would do nothing while the invaders tried to take over the whole kingdom. Sure, some nobles might prefer to bide their time or take advantage of a chaotic situation, but they wouldn't just let themselves be conquered or have their lands trampled and ruined by raging hordes from who-knows-where.
That's the wrong way to look at it. If they hate their current lord enough, they won't see it as a foreign ravager, but as, at very worst, a status quo scenario. Why should they fight to defend the legal rights of somebody they don't want to rule them?
Granted, they will get annoyed if their own castle is besieged, but their garrison does fight back then.