When a province falls into your hands, the previous garrison will have been wiped out. It will then take you a period of time to replenish the garrison. If your army leaves the province (or is defeated in battle) and and enemy besieges it, the garrison will be at whatever level it is when the siege starts. This could make it super-easy for the province to be recaptured, so you need to devote some thought (and forces) to protecting your gains if you want to keep them.
To answer the inevitable follow-up question...no, you can't drop troops from your regiments into the garrison. The garrison has to be replenished via the monthly regain rate (I forget what that is precisely, but it takes it a while to get back to full strength).