If you go to the Belknap county deed website at:
http://www.nhdeeds.com/belk/web/start.htm
and search on your family name, you can trace the ownership of the property. I found the current owner pretty easily. Even though it is public information, I don't want to put his name here. Also it looks like somewhere along the way the address changed or your family owned two properties on the island.
I know someone that bought two houses on that end of the island and tore one down, but it doesn't seem to be related to your property.