Goldfish were originally grayish green or brown.

Goldfish are just the domesticated version of carp. Natural genetic mutations caused some of them to have gold skin, and the Chinese used to raise them as pets and specifically breed the gold-tinted mutant carp, during the Tang Dynasty.

The Prussian carp is the wild version of modern goldfish. It's usually gray, but some of the early mutations that led to modern goldfish still occur, causing yellow-tinted carp.
(source)