While this seems extremely odd to us as humans, in fact most hooved animals have square / rectangular pupils. Scientists have discovered that the reason goats have these eyes is to allow them to see in a wider circle around them.
Because of their pupils they can see up to 340 degrees while humans are limited to 210! Square pupils also help goats see better at night because they can be closed more during the day to restrict light. Interestingly, another animal with square pupils is the octopus!