I agree that the map would look better with an integrated Silesia. However, as the Silesian provinces are part of the Holy Roman Empire and aren't Bohemian cores, annexing them would make our relationship with the empire suffer greatly and severe sanctions would be imposed by the emperor, which would for example lead to much decreased tax income from all of our provinces. This is a SRI mechanic.
I am of course aiming to dissolve the HRE at some point, but that will still take some time. Until that time, Silesia must be spared.