The thing about the Chinese "not expanding because their neighbors were barbarians" is that this seems to raise some awkward questions about the actual extent of Chinese expansion. In many respects, the China we see today is not far removed from its greatest extents; it was not always so large. The Miao tribes are probably among the most noteworthy, as they were not fully suppressed until the Qing Dynasty in spite of being in what is today "China" in spite of their conquest in the 13th century. When a solid economic or defensive benefit could be found in decisively defeating an enemy and expanding the grasp of Chinese governance, it is amazing how little the notion of "barbarians" emerges as a counter-argument and how much it appears as an argument in favor; the fate of the Dzungar Khanate against the Qing and, long before that, the Han against the Xiongnu, the Sui against the Goguryeo, or the overall tremendous military dynamism of the Tang. The Silk Road was the heartbeat of trade with the west, and that shows in Chinese military adventurism westward through the Gansu Corridor into the Tarim Basin. By contrast, Taiwan or the Philippines are peripheral islands home to peripheral polities, and relatively peaceful and/or poor ones at that; conquering them would require a significant military investment for not much additional return, unless you're a (to borrow and tweak the statement used earlier by another in this thread) map-painter or head-counter. Rather, Taiwan only became important when it became a potential threat to the Qing as the base of a Ming remnant, and the domination of trade in the Southeast never called for much more than the establishment of a few fortified trade bases in the Philippines by the Song.
Geographically, though, it is noteworthy to see how China actually did grow - every single one of its borders continued to spread away from the original core lands around the Yellow River until it reached some geographic obstacle, rolling over almost every military obstacle in their path. In the East, obviously, this was the Pacific. In the North, these are the vast steppes as well as the Gobi Desert and Altay and Yanshan Mountains: regions of exceptionally low population densities, few valuable resources (until the modern era), and obdurate tribes who were not worth the military cost of extracting tribute. To the West, the Taklamakan Desert, and beyond it and its chain of peripheral oases that formed the north and south branches of the Silk Road, the Pamirs, the Tian Shan, and the Kunlun Mountains all pose even greater obstacles which the Tang learned in particular were nearly insurmountable when it came to projecting military force into Central Asia beyond. To the South, Chinese civilization overran the Yue tribes of the Yangtze in the time of the Qin and Han, and kept right on going until they reached the jungles of Indochina and the mountains of Dali and Dian Kingdoms in the Yunnan. This pattern does not strike me as that of a non-expansionist, complacent and self-centered polity. Even the Ming, that late dynasty to which this position has been most often attributed, embarked on military campaigns in Yunnan, Vietnam, and Manchuria, the former of which was most successful, as well as multiple punitive expeditions against the Mongols and Oirats.
That said, a decline in military opportunism once these borders had been reached may not solely be to some sort of belief in one's superiority. By the same sort of broad-tarring brush, it could just as easily be attributed to this simple question - when one, as a ruler, has attained a hegemonic domination over a tremendous part of the world and is surrounded only by petty fiefs that largely exist because they are not worth the cost of conquering, what is the greatest threat to one's power? Essentially, the greatest threats to China once it attained its far east hegemony became not external, but rather internal threats. In its greatest manifestation being civil war, such as that in the Warring States or Three Kingdoms periods, internal unrest was destroyed more dynasties than the northern barbarians. The Qin fell before the Han, who themselves fell to internal unrest in the Three Kingdoms before eventual reunification. Even the conquest of China by the so-called foreign dynasties (the Jin, Yuan, and Qing for examples) tended to occur in times of already-present internal weakness, rather than sparking internal weakness themselves (famously, the Qing conquest took advantage of the self-styled founder of the Shun dynasty, siding with Ming remnants to crush him at Shanhai Pass before defeating the remaining other pretenders in detail - they would never have made the attempt had Wu Sangui not abandoned the northern fortifications in order to stop Li Zicheng). In light of this, the major military threat to China was, once it reached these "natural borders", in fact its own military. Any ambitious general would have no lack of inspirations in Chinese history to seize power in any time of political weakness. Certainly, sending such ambitious generals to establish some distant "Commandery of the [Cardinal Direction]" was an option, but it was also an option that, as the Jin learned from Qiao Zong who founded the Western Shu, could backfire if said general came home with the glory of conquest and a powerful army loyal to him. It was much safer to pare the military back and occasionally organize a retaliatory strikes if a few of the bordering northern nomadic or semi-nomadic tribes became too restless. While this did not stop foreign adventurism, it certainly marked limitations in the successes it could achieve, as can be seen from the net effect of the Ming campaign in Vietnam: in spite of early successes resulting in the incorporation of the region as a Chinese province, a series of decisive defeats during the Lam Son uprising brought Ming attempts to bring the region to heel more or less permanently to a close. By contrast, the Ming expeditions into Manchuria and Mongolia, with a goal to maintain the security of the nascent dynasty against a potential attack by Yuan sympathizers/Mongols, was completely successful in this aim, but neither attempted to achieve any more centralized political domination of the regions beyond the usual tributary relationships, and the former was not successful in completely pacifying the Wild Jurchen of the Amur.
EDIT: Not taiga. That's a bit too far northward for where the Chinese failed.
