OK, point taken. Should have rebelled against that heretic king
Didn't always have the ability. Or they could be staying and taking the infidel liege's wrath on themselves to spare their people, such as when
King Demetre II of Georgia went soo the Khagan, knowing that he wouldn't likely return. There should be examples among Russian princes too.
From the point of view of Catholic theology, it's actually a very poor idea morally to start a war you can't win, especially if this means losing the lives of you and your folks. This was sometimes blurred by a crusading spirit when people thought they'd all die as martyrs but a Catholic count or small duke without any chance of winning against his new muslim or pagan liege would be well within his right to try to work out a way to find a way of living with the new power, himself (i.e. the count or duke) remaining as the guarantor of his people still being allowed to stay Christians and stay alive at the same time. For Orthodox, I guess King Demetre II and certain Russian princes are examples of similar thinking. They actually went and sacrificed their lives to avoid a war that would destroy their respective peoples. Going to war would have meant destruction of the towns, execution or mutiliation of crowds of their subject, selling into slavery, forced conversions (barring Mongols) etc.
I believe people from your religious group should stay. This includes the remnants of Greek or Armenian nobility when the crusading Franks come and take the higher titles.
An especially striking example is when a Mongol power or a sultanate from Africa uses an invasion CB on an Italian republic. In later starts, this can actually mean the Ilkhanate sieging Caffa and other holdings in the Crimean peninsula and the Doge of Genoa surrendering himself as a Mongol vassal thereafter, bringing Genoa in Italy under the Mongols along with himself. The same could be done against the Serene Doge of Venice, who could use his king-tier title but remain the doge (duke) of Venice all the same, now under a muslim or mongol power, just because somebody messed with Crete.
Now, if you launch a holy war there, you don't get to free the humiliated Doge. He and all the mayors, bishops, barons etc. get wiped. I suppose you could engineer an alliance and wait for them to rebel but the opportunity may never present itself. They can refuse your advances for this or that reason (desiring a better alliance among them) and then simply not rebel. Not to mention that your children and siblings aren't unlimited and uncles and aunts don't give alliances once your ruler changes.
Also, imagine the Fatimids somehow win an invasion vs Aquitaine where the French king surrendered with a small war score (e.g. because he had England on him). Perhaps 2-3 counties in Toulouse go to the Sultan or the other muslim ruler who won, who becomes king, the former French vassal dukes becoming his vassals. The Pope calls a crusade. Probably the entire kingdom is Christian and just 2-3 counties are in the demesne of the muslim king, all the dukes are still Occitan or French Catholics. If you win that crusade, all of them are removed, regardless how long they'd ruled the lands. This shouldn't be happening.