1. Stop engaging their navy if you do not have naval superiority. EU has always been defiicient when it comes to naval warfare, and although EU4 has improved the basic issue intrinsic to the combat engine remains the same: AI has very broad sight when it comes to naval unit deployments, and he also has good algorithms telling him when to move to engage and when to move away. Unless you are (a) very attentive; (b) at least somewhat lucky; and/or (c) have some other advantage in terms of modifiers, naval trad, admirals, etc., engaging a nation/alliance/ coalition with anything less than 2:1 gross naval numerical superiority (and AT LEAST tech parity if not a slight edge) is a very risky business. The problem is of course that naval "battles" (like land battles): (d) drag on for ridiculously long time periods (e) still (despite changes made between EU3 and EU4) have a relatively high rate of vehicle destruction and relatively low rate of engagement avoidance/retreat. The fact that "land battles" (actually campaigns) can drag on for weeks and months is only a breach of reality if you insist on regarding them as true "battles" and refuse to acknowledge that they are actually campaigns. If they are campaigns then the fact that the engagement(s) can drag on and on for weeks and months is not so much a problem.
But with naval battles, this doesn't really work. Naval campaigns were much more fluid than a representation as one long drawn out engagement in which every single ship from the two opposing sides comes together in one "province" for one battle royale in which 30% or more of one sides navy gets sunk/captured and the other side may suffer anywhere from zero to comparable casualties.
What all of this means is that: it is foolish to send out small fleets, and in general you want to desperately avoid naval engagements unless you have at least tech/idea parity and numerical superiority (2:1 for all ship classes that might be engaged or ideally 2:1 with the best ship class you can bring to bear, meaning either heavies for ocean engagements or oar-ships for inland seas).
Unfortunately this isn't really how naval warfare worked, even during this time period, but ah well . . . no one game engine can provide everything. At least in EU4 navies are not effectively irrelevant as they basically were in EU3 and the inclusion of the retreat mechanic does bring the naval battles a lot closer to something like reality. The way the maths work in naval combats has also seemingly been improved TREMENDOUSLY! Still, all of these improvements are being built on top of a real-time combat engine that is just simply wrong when it comes to naval battles.
2. Look at how many ocean going vessels (heavies, lights and transports) they have in their Navies ledger page (unless of course it is tenable and advantageous for you to engage them in the med or some other inland sea where a fleet of oar-propelled galleys and galleases would be better). Build two times that many Heavy ships.
Do not set this fleet, or any portion of it to sea until it is fully formed. This will probably mean a long time of building, because you'll want to restrict construction to a small number of very closer coastal provinces where there is a limited risk that portions of your fleet will get engaged and massacred while you are forming it up into a "Mega Deathstack fleet." In fact, having more like 3:1 numerical superiority is preferable because of how the AI seems to judge its chances and how good he is a skedaddleing when he thinks he is outmatched.
If for example the AI has 20 Heavies, 20 Lights and 20 transports, you might consider 50 heavies to be enough to beat anything he'll bring to bear in an ocean battle, even if he manages to bring his entire navy into the fight before it is over. 60 heavies vs 20/20/0/20 (and assuming parity in all other respects) would certainly seem to be enough.
However, if you are floating around a death stack of 60 ships, you are going to have a hard time engaging him. He will run, and unless you have a substantial advantage in speed he will generally get a away. Apart from gaining some advantage in maneuverability, you can defeat this aspect of the AI's decision making by building even MORE ships, say 80 or 100 heavies, and deploying them as two or three fleets.
AI sees your fleet of 30 or 40 heavies and maybe he'll 'think' to himself, 'Okay, I can take that with my fleet of 20/20/0/20.' Then of course, once the 'battle' ensues you'll have a couple weeks to get your other stack of 30 or 40 there to tip the balance in your favor numerically. As long as the battle hasn't already gone badly against you, this can lead to victory, and at least cause his navy to go into dock beaten up and demoralized, and depleted of a certain number of ships.
This is the only consistent and relatively foolproof way I have found to ensure naval victory. Certainly with a lot of attention, some acceptance of risk, and intense micro-managing/pausing, it is possible to accomplish the same sort of decisive battle even without numerical superiority. But if on level or weaker technological/tactical footing, the greater numbers gambit is the safest, though obviously very costly and time intensive.
In this respect, the game is relatively accurate. The navy who could put more ships and higher quality ships/crews into each of its squadrons/fleets was the one who could dominate (meaning Great Britain) and even when leaders in rival nations understood this, and had literally generations to try to overcome it, the sheer cost of keeping up with, much less surpassing England's naval superiority (numerical and qualitative) was more than really anyone ever managed until the 20th century (except Spain to some degree). So while the 'battles' side of the naval game is IMHO rather lacking, and I think tends to promote a very 'risk-averse, keep your ships in port' approach unless you have dramatic numerical superiority, the overall historical and grand strategic way it tends to play out is fine.
3. Once you have your grand Armada(s) built, then start building your transport fleet(s). You'll want to transport as many soldiers as possible in as short a time frame as possible. However, it sounds like you might suspect Spain has few regiments left? You can check this in the ledger too. Basically you want to land with enough troops to ensure that your invasion of Spain does not fail. If it fails then all that time and investment is lost. Thus I tend to, again, argue that a 2 or 3 to 1 numerical advantage is advisable. If you land with a 0.75:1 first wave, or even a 1:1 first wave, with the second half in a second wave that will be several weeks or months behind, that right there could ruin your invasion of Spain. Better to land a first wave of 1.75 or 2 to 1 and a backup 2nd wave of 1.5 to 2 to 1. Then you WILL win (assuming you re not behind in Mil Tech and have properly formulated armies).
4. Siege Spain to 100% WS.
5. Force them to release Aragon or some other nastiness.
ADDIT: and one final issue that your situation demonstrates: do not try to play the global colonialist game if you cannot compete on the naval warfare playing field. While the situation does sound a bit absurd, the truth seems to be that, Spain's navy dominates the seas. Thus, even though you have an advantage in land war, and have defeated them in a string of land campaigns, you lack the naval power to extend your land capabilities into the Spanish homeland.
