Expansionist Zealotry privilege is still unclear

[RMS Oceanic]

As part of experimenting with new Byzantine strategies I decided to take the privilege for my opening war against Epirus, and sure enough it worked as advertised. I have since vassalised and converted Epirus, and now the Privilege is giving me the negative parts, despite being at war with a heretic or heathen merely months ago. I went into the code to take a look:

conditional_modifier = {
        trigger = {
            NOT = {
                any_known_country = {
                    NOT = { religion = root }
                    OR = {
                        war_with = root
                        truce_with = root
                    }
                }
            }
            NOT = { has_country_modifier = recently_force_converted_protagonist_timer }
            NOT = { has_country_modifier = recently_annexed_other_religion_timer }
        }
        modifier = {
            church_loyalty_modifier = -0.1
            land_morale = -0.05
        }
        is_bad = yes
    }
    penalties = {
    }
    ai_will_do = {
        factor = 0
    }

So even though I've recently fought a Heretic, because they're no longer a heretic the privilege "forgets" this. I'm wondering if there's a way to address this loophole.