The system does suck, it's a universally accepted fact.
I never have a problem with adventurers unless my realm is extremely unhealthy. They are sacks of gold waiting for me to banish and confiscate, as far as I'm concerned.
Adventurer warning? Let's plan what to spend that 500 gold on in two years.
The last time I had an adventurer actually win was when I was in the middle of a civil war and being victimized by a crusade at the same time. And even then, he just occupied some crap and won by ticking war score. I could have beaten the guy eventually if I could have gotten armies over there.
Hell, I turned LoR off and ran a game with no retinues and I still never got in trouble with any adventurers, including the time both my banished brothers launched attacks within months of each other.