There is no problem for Ceylon. The problem exits for IJN (or any other nation) using Ceylon as a forward base to patrol the Indian Ocean. We can expect such fleets to come into port after a month and demand a huge amount of supplies and oil.
Ceylon will not have that - therefore no full replenishment will probably occur even weeks later... unless owner of fleet runs a special supply convoy to Ceylon, or gifts Ceylon the needed oil/supplies, or long ago set up very favorable trade for Ceylon so that it has managed to stockpile enough. This is in marked contrast to the situation that would exist if owner of fleet also controlled Ceylon. In that case it is a normal convoy, and one simply turns the convoy up to 100, or 200, or a ridiculous 500 merchant ships if you want... and at midnight you will have all the supplies/oil the replenishing fleet needs stockpiled in Ceylon.... and now it takes only a day more for that fleet to take on same.
The above situation needs to be compared to the special convoys (bright green) that can be set up to replenish player's units based at other countries - such as an independent or puppet Ceylon. In my experience, those bright green convoys do not turn up to a very large amount to quickly transfer huge amounts of oil/supplies. As the bright green special convoys are the only convoy option if player does not control Ceylon, expect a replenishment time of a week or more instead of a couple days.
So the problem is: Fleet is not ready for continued action until a week or more later. This creates a serious lack of patrol coverage that can result in enemy penetrating the patrol screen. The real problem will be when a RN CTF starts sinking Japanese TPs around Indonesia because patrolling of the Indian Ocean failed due to poor replenishment at Ceylon. Of course, you could base the fleet back at Rangoon, but then patrolling the Red Sea to insure no Allied fleet can leave the Med that way is impossible. After all, that was the whole point in taking Colombo. But its not worth much unless it comes with an appropriate means of re-supply.
Regular trade is not a favorable option since several fleets will vary replenishment needs, and 3-4 fleets coming at near same time will still wipe out Ceylon's stockpile established by guess work trading amounts set up over a long time estimate.
Open Negotiations works - unless you can't do any diplomatic action until allowed. So you might have 2nd fleet sitting a week waiting for minister to arrive because you just did open negotiations for earlier fleet (but didn't notice other fleet also heading for port). Such oversights easily happen in war, but if you control Ceylon these kinds of screws ups are not even encountered.