As far as I know, everything on the Paradox webshop is a Steam code, but certainly all EU4 expansions are as it's a Steam-only game. You can see a Steam icon next by "platform" on all the Steam games, just in case there are any non-Steam codes in there.