The total population is not a good indicator of military manpower. AND....all manpower is not created equal. There is a difference between a Royal Marines commando and a supply clerk. Almost anybody can fill out a supply form, but climbing that wall with 50 kgs on your back is a specialized skill that not everyone is up to. Also, things like the level of medical care and education in a country can mean the difference between someone who is capable of military service and someone who is not.
(completely ficticous) Example: ten years ago, I broke my leg. The local witch doctor did a rain dance to drive out the evil spirits and my leg eventually healed.....but at a 45 degree angle. My twin brother broke his leg on the same day and had surgery to implant a metal plate which held his leg together, and is now a (slow) marathon runner.
More to the point.....
Most of the European states involved in WW2 had significant combat (i.e. men/males) loses in WW1. This is most true of France (as a proportion of total population). In France, the result of the lack of men (i.e. fathers/breeding males) led to a shortage of males available for military service, called "The hollow years" (IIRC) starting in the early 30's. This was one of the contributing factors to France's decision to build the Maginot line. Other nations (including Germany and the successor states of the Russian Empire) also had this, but to a lesser degree. The issue with the question is that you could have some domestic policy to increase population enacted on 1 Jan 1936 (start of the game), but by 1945, the resulting "population increase" would be at best 8 years old. The only real way to increase numbers available to the military is to do one or more of the following:
1-lower physical standards so that those with minor (or maybe major) disabilities would then serve (Germany did this)
2-lower the age of recruitment (Germany did this)
3-include both genders in the military (Soviets did this more than anyone else, but the allied powers also did this to some extent-at least in non-combat roles)
4-draft foreigners into your military (Germany did this)
5-increase efficiency in your economy so that less manpower is required for production (pretty much everyone tried this, to varying degrees of success)
6-mobilize foreign workers for your economy to free up native manpower (Germany did this-pls don't use this to start a discussion on a banned subject)
7-include women (and maybe children) in the workforce (pretty much everyone did this except Germany, unless you count Germany's doing ++++++ (a banned subject))
8-decrease the manning level/size of your units in the field (Germany did this)
9-shut down sectors of your economy that don't matter to the war effort (pretty much everybody did this to some extent)
One of the manpower problems that you should run into if HOI4 models it correctly, is that as you increase the number of factories, and put the economy on a "wartime basis" (i.e. running round the clock production shifts), this will eat into your manpower. It takes 3 times the number of workers to run a 24 hour shift as to run a 8 hour shift. Figure 1000 workers per factory for regular production (just a number I'm throwing out), jumping to war production would take a serious hit on your available manpower unless other things are done to fix this.