Is conquering North Africa worth it, or will it just bring RR and drag down my research?
All of the above.
The most reasonable answer that I can offer, is that it depends on what nation you are, and how large and/or fiscally secure you are.
If you are an OPM with a port - I wouldn't do it. If you are a blob in the HRE, go for it. If you are a non-HRE Eurasian nation, then maybe. The majority of North African (and you might as well add Middle Eastern, Eurasian Horde, etc. etc.) provinces are not wealthy, so there will be reduced income during RR, and even after RR subsides, you won't *quite* have a revenue windfall. Additionally, you will have to divide your forces between your home nation and your new conquests - which will probably mean building many more troops (spending ducats), even if you avoided having regiments destroyed during your conquests. Then there is the significant cost (in ducats and RR) of conversion, if you choose to do that. So conquering those non-threatening lands is generally not an easy source for more ducats. Vassalizing your neighbors, province improvements, and trading can offer more benefits in the short term.
There are distinct short-term advantages to the early parts of conquering the infidels; namely the *fighting*. Crusade gives nice bonuses for as long as you can drag out your warfare. Unam Sanctum is a really nice way to take a lot of land (with the wrong religion!) for a small amount of infamy.
In my Transylvania game, I'm quickly falling behind in tech - even though I've managed to join the HRE recently - largely because I slacked off in province improvements. Gobbling up what is left of the Ottomans (and their allies, and their neighbors) makes sense because it adds to my Force Limits and manpower. To be victorious against other members of the HRE (and even some Middle Eastern nations who invested in tech instead of warfare), I have to outnumber them by at least 2 to 1, until I can Westernize.