Two ways, need the shift key for both:
First way. Select all the armies in the province, then while holding shift down, click on the armies you don't want to merge in the panel on the right until you have only the armies you do want to merge.
Second way, a bit less fun. Do the same thing as the first way, just clicking on the little bars underneath each army on the map, instead of their representation in the right window. Much harder to click, and you don't know what army is what usually.
First way. Select all the armies in the province, then while holding shift down, click on the armies you don't want to merge in the panel on the right until you have only the armies you do want to merge.
Second way, a bit less fun. Do the same thing as the first way, just clicking on the little bars underneath each army on the map, instead of their representation in the right window. Much harder to click, and you don't know what army is what usually.