I don't think much would need to be changed. Local autonomy will handle the issue of rapid annexation, sorties will prevent unrealistic carpet sieging. I'd also increase costs of war by increasing attrition suffered the further you are from owned provinces (not controlled!). The idea is that making big gambles like annexing Austria should be a high risk/high reward venture - if you win, hurray, extra stuff! If you lose, hurray, all your neighbours declare war on you in your moment of weakness and low manpower and gold! Also, annexing Austria whole will give you enough AE to trigger a European coalition and because there's no limit to their demands, you'd be destroyed. If you think you can beat everyone, well then as the saying goes, shoot for the moon, and if you miss, you'll die in space alone of asphyxiation.
A greater problem is diplomacy. One of the most glaring problems of the diplomacy AI is that it cannot comprehend that allying the biggest fish and allowing it to stomp it's neighbours is a terrible idea. Example: Spain allying France. France is already a powerhouse, and Spain is allowing it's very powerful neighbour to become even stronger as it moves into the HRE. Anyone would understand that as soon as the HRE, France's biggest rival, is dealt with, Spain is next. The AI does not, and this is the greatest hurdle to the abolishment of warscore. The AI should be able to assess the relative strengths of regional alliances and try to either find an alliance from which it can benefit most (e.g. enemies of neighbours) or, if none exist, find one that maintains a balance of power between the alliances. An additional diplomatic option would be needed, where you could propose to ally with a state and simultaneously break alliance with another, so that switching alliances is more seamless.
I'd also add a "threat" opinion malus, which would take into account the AI's (or it's alliance's) power compared to another and if the difference is too great would penalize relations, effectively breaking old alliances and preventing new alliances with the threatening state from forming. This would be separate from AE and would not trigger coalitions on it's own, but would obviously play a substantial role if the threatening state began expansion. For example, if England was allied to France, Spain and the HRE, but France was only allied with England, France would see England as a threatening state because of it's allies combined forces and would want another ally to balance it out, say Denmark or the Ottos. But Denmark hates England and will only accept France's alliance if they break their alliance with England. Switching immediately would open France to England's return cores CB, so France first works to ally the Ottos, then switches England's alliance for Denmark's, establishing regional balance. This would create a volatile and dynamic diplomatic stage where every decision could cascade into a power struggle and jockeying to find a proper ally.
