I'm not sure there is right answer here, mainly it will come down to your expected experience when you
"use our phones when we're up" and budget...personally I would struggle with 100mb, as I suspect the real speed would be even less in reality. That said, nothing wrong with trying to save money.
Check out this comparison too:
https://broadbandusa.ntia.doc.gov/about/speed-simulator