The remaining United States would most likely have its black population continue to gradually assimilate culturally, and the existing racial and cultural divisions and animosities probably wouldn't exist to any significant degree. Basically, without the Reconstruction period, and the last vestiges of slavery in the north allowed to either die of more natural causes or be settled under more amicable terms than gunpoint, the polarized opposite extreme views wouldn't have developed and entrenched so heavily. Race probably wouldn't continue to be a significant issue past around WWI, in the same manner that it's not one today in Europe.
The CSA, however, would further ingrain its beliefs and legislate them into stone, leading to a long-term continuation of all of the inherent problems long after the rest of the world had moved on. This would continue to be a sticking point for any renewal of relations between north and south, and possibly the fuel waiting for a spark to ignite a second conflict. In that potential situation, unless the south had made large scale purchases of armaments and materials despite nearly world-wide condemnation (as an agricultural economy, it wasn't in a good position to industrialize on its own), the north would be in relatively even better shape than the first time around. The early southern victories in ACW I, before the far greater northern economic muscle historically tipped the balance the opposite way, wouldn't happen the second time around (ACW II) against a better prepared north facing obsolete southern equipment left over from the first war (much of which was produced in the north before that war).