HRE Lands and Outside Parties

So, I had this event pop up where the Emperor was telling me, hey bud, how bout you give me that land you took. Of course I promptly tell him no, and find I lose stability. I'm not sure why I, as a foreign power, have to give anything to him or suffer a stability hit. If its such a big deal to retain the borders of the HRE, then have him bring his armies and take those lands if he wants. Also, if this is a repeating event, it should really be changed. Its amazing to me how uniting Germany gets progressively harder with every patch-expansion. Its near impossible now without a huge amount of luck and many many many years of waiting.