There might be a continent modifier, but there is also a step-function distance modifier. Take Catholic in Iberia as any Catholic and compare AE values between France, Italians outside HRE, Bosnia, Poland, and Sweden. None of these are Iberian culture group, all are same religion. But one will have significantly more AE than the rest, which will have different values between each other.
The presence of colonies/distant territories will sometimes nullify this distance modifier completely, despite that the distance between target and outraged is otherwise huge. But the game does not treat distance this same way for other mechanics, hence the internal inconsistency.
If he didn't have colonies in Indonesia, taking land in Egypt would be meaningless to the Indonesians. But because he does have a random colony there they now care as much as if they were in charge of the holy cities, possibly minus that flat -10. This is not a sensible interaction.