To add to this, the fact that it takes fleets YEARS to cross an empire means that two separate fleets can easily conquer big swathes of territory before the doom-stack rolls in.
And once it does you can easily, if you plan things carefully just constantly avoid fighting it by either hiding behind a strong enough defense stations (half fleet + defence station can easily beat doom-stacks at least before the late-game, but by then its a gatewayfest anyway) or having a fleet full of corvettes (which the doom stack can't catch up to). Meanwhile the other fleet is free to do whatever it wants. I have repeatedly beaten empires with large doom-stacks by employing that strategy.
Add gateways or any form of unrestricted quick travel and this all goes up in flames.
So while I agree with the OP to the extent that sometimes allies seem useless there have to be countermeasures against gateways (eg destroying them with a swift fleet of corvettes). Also the AI should learn how to use its fleet properly and realize that a defensive pact with someone 3+ years away isn't really worth it.
1. Even on mid game, when piracy pop out, it's so weak that I can just build 3 to 4 corvette from the nearest shipyard to deal with them
2. The problem with improving the A.I choice of allies is that which threat should it prioritize? Because it might ally someone opposite of the player direction because it can also has a threat in that region. Also, they can't always pick someone close by or adjacent due to the variety of ethic government in the galaxy. So should they prioritize having the player as their biggest threat? Or should the dev improve on their threat assessment by adding more considerations? I'm not a programmer but I think the latter is hard to do, especially if you want to do it without allowing the A.I to cheat (like vision to the fog of war, etc
3. I agree, if they do bring gateway travel as earlier tech, they should bring it with limitation. Cooldown is one way to do it or another way to do it is like I said, to make it like a one way mass relay.
4. As for doomstack, in the early to mid game, assuming your opponent upgrade their starbase, you will need the full force of your force limit to crash through it anyway. In fact this is true even for the AI. When they split their fleet, each of those fleet can be stopped easily by your starbase. But when they bring their two fleets together, then it can crash through my bastion. If you notice, fleet can win againts starbase whose fleet power are like 1 to 2 k above them. So, assuming equivalent star nations, both side would still need their doomstack to actually win the war. But to be honest, I don't think this is a problem. Bringing your fleet together to one single offensive is a strategic choice, and it's one not without a risk. You are leaving your other flank without a fleet and Paradox A.I always has the tendency to pounce at such opportunity. (Especially if you set their aggression to max) That's why, whatever solution to the mobility problem, in regard to allies, must be done in such a way that would retain the risk of the doomstack strategy. It's hard balancing act but hey, if it's easy, the dev would have done it already.
As for the highway idea, I'm still exploring that idea in my mind. The problem is, it alsohas the possibility of becoming way too useful, and therefore reducing the risk of doomstack strategy again. If implemented, it will have to have the right kind of limitation.