This common misconception likely has to do with the fact that humans often get goosebumps when they are cold. Goosebumps are caused by the contraction of small muscles at the base of each hair. These contractions pull the hair shaft erect, which makes the individual hairs appear to be longer.
Hair tends to grow about a half-inch per month, regardless of the temperature.
(Sources: 1, 2)