There is actually a setting you can choose that weakens or switches off invasions from NPC factions. We added it because we thought people who player competitive PvP might get annoyed by sudden NPC invaaions messing with the balance of who won.
If you go into advanced world settings, I think it's under Game Flow when you set up the game.
That's the thing, I had that setting in my most recent game, but because I refused their demand they declared war on me