Give him gold and a title. And if necessary give him land. I've had an heir leave because I kept denying him land.
Really? Because it always tells me, it will say things like, Thorvold went to so and so county and why. IE His wife became regant, or to press claims. Once I had a sucession crisis, I made it a point to watch all messages and why people left. It should pop up and pause game when your heir leaves though, for other family members it just pops into important message box. You might need to reset your message boxes, but you should always know where your heir is, the game does pretty good for me on that, now that I read every message.
Are you using a mod? I checked the message settings but I didn't find anything.