I was searching for an image on Flickr today, and found a photo that reminded me so much of this mysterious structure, I had to post the link here. It's of an observatory (Oak Hill, Derryfield Park in Manchester), which you can actually enter and go to the top.
The mystery structure looks similar, except that it looks like just the base, in the same granite octagonal shape. Could the top have deteriorated (wooden?), or perhaps the project was abandoned before completion?
Weston Obsesrvatory on Flickr
Another link says "Constructed in 1896-97, the building is named for James A. Weston, who left the city $5,000 to build an observatory for "the advancement of science, for educational purposes, and for the use, enjoyment, benefit, and mental improvement of the inhabitants of the city of Manchester." The man who grew up on a local farm became the first native to serve as both mayor (four times) and
governor (two terms)... His home was on the grounds of what is now Wagner Park, fitting for the mayor who also
created an extensive parks and cemetery beautification program."
So, it would fit that he could have built another near GOVERNOR's Island, right?