The earliest use of the word ‘orange’ in the English language refers to the fruit and not the color. Before the English speaking people were introduced to the fruit, the color now known as orange was referred to as ‘yellow-red’ (geoluread in Old English) or ‘red-yellow’, depending on the hue.
Orange as a word originates from a Dravidian language and was then borrowed and adapted by many languages like Sanskrit and Old French before it reached the English language. The earliest recorded use of the word in English dates from the 13th century. The color was then named after the fruit and the first recorded use of orange in that context was only in the 16th century.
Over the ages it has always been a bit of a standing challenge to find a single English word that rhymes with the word orange. It is accepted that there is, in fact, no true rhyme for orange. There are, however, a couple of words that are half rhymes like hinge, lozenge, syringe, flange, Stonehenge, or porridge.
As rhyme is not only based on how a word looks, but also on how it sounds when spoken, sporange (a variant of sporangium) has often been mistaken for a rhyme candidate, but the stress falls on the second syllable, not the first. There is just no word that rhymes with orange!