How do I have to use Vasalls and Trade Cities in order to counter the limited state-amount of a Merchant republic?
If I am around the mediterranean sea, are there 20 like `mandatory` provinces that not to state would be a mistake?
Not really if you turn an area into a state - all provinces in that area will become full cores and count towards that 20 provinces limit.
The best is probably to just turn a few areas close to your capital into full cores so its just at 20 provinces. Then play tall. Develop those provinces into powerhouses with 30+ development each by the end of the game.
20 provinces with 30 development each is 600 development. Note that provinces that are territories dont count towards the 20 province limit so you can have plenty of those. Add to that in vassals and you can become the #1 great power of the world.