Are you talking about per capita production or rough plantation-wide production? Because, as you said, it can be in the end far more productive in absolute because you have more people working a same patch of land for more time with longer shifts and a almost industrial organization of labour, but as someone who lives on a country where latifundia to this day dominates i can assure you that much productive capacity is wasted because neither the plantation owners nor the workers have much incentive to make complete use of their lands, since the slaves won't gain anything from it and the plantation owners don't need absolute efficiency. I'm sure the crack of the whipe can drive someone to work harder, but small-scale sabotage and lack of willingness to work where also constants of slave economies all throughout the world.
On the issue of taxation, you don't use it only to pay for infraestructure and education, you also need to pay the army and your bureaucracy, vital functions of any State since its inception, and most rulers in history wanted to expand their power, wich implies the expansion of these tools of state, wich implies more need for taxes. After all, the raising of taxes that led to american and french revolutions surely wasn't made to pay teachers or roadbuilding.
per capita...large plantations had an economy of scale (and whipping) that the early attempts at sharecropping could not replicate. Planters were a competitive lot and had every incentive to maximize their land so as to maximize display of wealth. If there was one regret the planters held over Black slavery, it was that they could not squeeze an equal amoint out of the poor white trash.
As for military expenses, there was a state militia, but that was not a large expense and state bureaucracy cost little as there was not a lot of formal administration and that which existed was part of a good-ole-boy network that paid for itself in bribes/kickbacks. Federal expenses came largely from tariffs, which largely fell on those wealthy enough for imported goods. Do remember that there was not a large standing army in USA at the time