Oh so I have two planets with sedentary pops. Apparently the planet from which people emigrate got -3.4 pop growth penalty from emigration, while another one receives +2.95. So I lose 0.25 pop growth. I presume that it's caused due to the fact, that sedentary multiplier is not applied to emigration, but only imigrant (at least that's what pop-up shows).