Sorry for the colloquial tone
While I agree it should be discussed for alliance members too (Uncle Sam did build a ton of qualtiy infrastructure in Europe, including top notch military facilities in free Germany during the cold war), those questiosn should IMHO be distinct. To be simulationnist and realistic, I ll say that the french taxpayer who disagree on anything but welfare expenses whould have had objections to building freely a portion of the Maginot line in Belgium.
And to be nitpicking I would observe, as a sharp eagle I am, that neither Belgium nor Poland (the two wallable countries you rightfully identified) are part of the Allies coalition soon enough for the issue to matter anyways.
If you really are this fearfull, excluding land forts from the allied buildables, at least during peacetime, could be an option. While excluding mid tier infra is NOT.
How would you exploit it on puppets ? UK using its scarce IC to build a Maginot line between Mumbai and Kolkota from 1933 to 1939, while definitely not realisitic, doesnt sound me as a terrifying exploit breaking the game balance. And Royal Navy financing a level 8 military port on the eastern indian coast, on the other hand, sounds me as a plausible defense strategy for a colonial power wanting to keep its ability to confront a rising japanese power in the far eastern seas.