Yes you do... partly, Kronoz.
The .exe changes (changes to the engine, i.e. combat, TC, etc) take effect immediately. Changes to events, scenarios, ministers, leaders, AI and such only take effect after a new game
The .exe changes (changes to the engine, i.e. combat, TC, etc) take effect immediately. Changes to events, scenarios, ministers, leaders, AI and such only take effect after a new game