Your reasoning is correct but EU4 is a game focused on abstractions rather than a simulation, in a simulation type game (REF :I:R; Vicky 2) the devs should focus on historical accuracy.
Think of EU4 mechanics more as a "RISK" game and based on balance while still having regional mechanics for flavor beeing loosly based on some historical aspect of the region,then you'll see that the reasoning for not having isolate culture groups with just a couple of provinces makes a lot more sense.
As an exemple play with one of the small (dev wise) isolated culture group (Byzantine, tibet, some african ones, korean....) while some of them can be very strong regardless the boost given by having a big "brother culture" amount of clay around you is big boost IMHO. Look at the Manchu, once they're intégrated into the Chinese sphere of culture they gain a massive power as having land with accepted culture is a big boost to the lands values.