The South frequently raised the question, "If you end slavery, then what is to become of the millions of former slaves?" If there had been a reasonable, attainable answer to that, slavery might have been ended much earlier (though replaced with some sort of wage-garnishing share-cropping which would be scarcely better).
Which is what essentially actually did happen de facto, might I add.