Since the colonial land grant of 1664, Liberty Island was entirely located on the New Jersey side of the border. When the province of New Jersey was separated from the province of New York in 1674, it was argued that Staten Island belong to New Jersey.
Ever since then there has been a number of border disputes about who can control Liberty Island. Until 1997, the status quo was a very complicated solution: The over-water land of Liberty Island belonged to New York, while the submerged parts belonged to New Jersey. Complicated no?
In 1997, New Jersey further complicated things. They argued that the 1834 compact that had produced the current arrangement only granted the natural over-water land to New York.
This proved to be complicated because a lot of construction was done over artificial land that was infilled later. To solve this complicated problem, both states now share jurisdiction over the island. However, the fact remains: The Statue of Liberty is actually located completely within New Jersey.