It's sort of a chicken-and-egg question. What came first? the fruit or the color orange? Well all evidence points to the fruit. The word orange appears in the English language as early as the 13th century, but refering to the fruit. The earliest reference to orange as a color is found in the 16th century.
The color orange was previously called geoluhread, which just meant yellow-red. Geoluhread comes from Old English or Middle English. Orange is said to have no true rhyme, and many funny poems have been inspired and written about it. The word, however, does have half-rhymes or near rhymes and some proper nouns that rhyme with it.