If you have counts as vassals that are part of the same Duchy, click on a county, then in the lower left corner of your screen you get a box, in that box the Kingdom and Duchy the county belongs to are named, as well as the Kingdom and Duchy's Coats of Arms. Click on the Duchy coat of arms, and you should get a new box with four options at the top, the leftmost of them being "create". If there's no create, then it already exists, and you should go to it's current ruler (if it's not you), right click on them, select "transfer vassalage" and select the vassal that owns the county. Doing so will make the Duke like you more and rid you of a vassal. If you can create it, do so, and then right click the portrait of one of your vassals and choose the option "give landed title" and give them the duchy. The other counts in the duchy will become vassals of the count.
Do note that doing this costs gold every time you do it. Another option is to crown yourself emperor of the Holy Roman Empire (I think you should own enough for that) and then give away Kingdom titles, which basically does the above but on a larger scale. That does leave you with very strong vassals however, who you'll need to keep happy once they no longer have a relationship bonus for being granted a kingdom.
EDIT: Adding to above post, you can also just transfer the vassalage of a ton of counts to a single duke, that might put them over the vassal limit, but as long as you're not over it, hey, what do you care?
Also, if you're indeed gavelkind it might be a good idea to either switch to some form of elective (primogeniture or ultimogeniture might be hard to achieve at this point in the game) or instead form the empire; you'll lose some kingdoms to brothers on succession, but they'll be your vassals still.
That way you can keep these counties in your realm.
