Ivory on Madagascar is incorrect.

[Trin Tragula]

Walrus ivory has always been a exported ornament from Greenland.

And is in fact the reason for the chance to get ivory there.
As for Madagascar, Ivory, gold and slaves are the main exports from the mainland, it's not that improbable to have a center for their export on an island off the coast

 

[Trin Tragula]

I'm not really sure "an island off the coast" is an adequate description of Madagascar though. What you're describing sounds more like Zanzibar.

Islands come in many sizes. Madagascar was firmly a part of the Swahili trade world even in 1444, with small communities of Shirazi traders here and there. I still think it's not outside the realm of possibility for anyone setting up shop there to trade in the things that people traded in in that general area.