You probably should build civilian factories to start with. With mid-late game tech a single civilian factory can build a military factory in less then 1 year or a civilian factory in less then two years.There are lots of things to consider. The US, for example, will be trading a lot (and will receive lots of Civ factories), so they can focus on building equipment to lend-lease to other people before they actually enter the war.