Distance and your Mercantilism/FT slider. And there's an extra penalty in foreign (at least) CoTs if you can't trace a route through your own territory from your capital to the sea, or if there's no route through the CoT owner's territory from the CoT to the sea.
A CoT in my current game is justa few provinces away. I can reach the sea through my own land, from my capital. The CoT can reach the sea through the owner's lands. Still the cost is sky-high. Why?:/