It depends... on the tax laws and your relations with the counts and duke.
The default tax rate on feudal vassals is zero, so you probably aren't getting anything from the counts or their sub-holdings anyway. Their holdings pay them, and since they owe you zero percent, you get nothing. Create a duchy and it goes up one chain and you still get... nothing.
If you raised your feudal tax law and the Duke's tax law is zero, then yes by creating the Duke you would see a net reduction in tax since he is collecting only from his demesne and none from his vassals...
edit - except raising the tax law could hurt relations, which might or might not reduce taxes more than you gain... then it starts getting complicated I guess, since that all starts to depend on diplo stat, relative traits, and so on. I bet someone did the math to see which way is optimal.
Thank you. I have raised the feudal tax (there is no way the 30 or so counts get to keep all their gold!), and having now tested it by transferring the duchy of Apulia to my heir, it seems that yes, creating and handing out duchies will hurt me economically. According to the ledger, the counts in Apulia do no longer pay tax to me.
Wouldn't it be best to not ever create new duchies, and then, just before the game ends, create as many as you can to gain the instant prestige/score?