There is prestige, fame and level of fame (prestige, prestige_experience and prestige_level) If I decline a call to war I lose fame, if I attack an ally or break a truce I lose a level of fame and pay prestige. Gaining prestige gives the same amount of fame and at certain thresholds you get levels of fame.
What I am surprised about is that you can call in allies even if you don't have any prestige to spend. I thought it would follow the same rules as declaring a normal war where you can spend fame at an inflated price if you cannot pay the prestige cost.