Slavery almost came back in the 20s under the name of the liberal eugenics, political policies that would restrict the poor from making any decisions about their life, marriages, employment, everything controlled by the state, by the depression for vast numbers of people where they lived and work was controlled by chartered charitable organisations. They abolished serfdom and so invented homelessness and then they rounded up the homeless and gave them the choice of jail or the workhouse and if you ran away from the workhouse the police would collect you and drag you back and you could never leave unless you had the money to buy yourself out of it. How is that not slavery?
Indeed the Liberal option which would have happened had the first world war not was worse than slavery as it was based on the 'scientific' principles that poverty and criminality could be eliminated by stopping the poor from breeding, that the poor were a separate race rather than an economic class. Both Centre parties in britain supported it and in america the theory continued to hold full support until well after the second world war.
Theres no reason Slavery wouldnt come back, you can just be sure it wouldnt come back under the same name.
And secondly Serfdom was nothing akin to slavery as the serf had rights and the master responsibilities and severe penalties should be fail to fulfil them. However by the C19th the upper-classes had gained to much wealth off the death of it to ever allow it to return and no serf alive could remember it, not even the remainents as it was gone long before his fathers generation so Serfdom is not a plausible policy for Victoria II, no less than the colonisation of mars would be.
That said certainly in the uncivilised world Slavery could be abolished and then return as slavery with a new governments. Or potential anywhere should the situation be bad enough, except for england of course.