An early retreat should trigger it afaik. Are you sure you have the game rule turned on ?
Oh yes, definitely on. The game needs it even though its implementation isn't as good as EU4's. It was an all retinue army, it happened twice in a row and I lost like 4K men including some wiped out retinues - I kept getting attacked in a neighboring province and the shattered retreat arrow path was not displayed. My inbred Emperor was not leading the troops - he usually is but not this guy.