So moving to a properly structured CRM like Copper/Salesforce/Hubspot/Pipedrive will result in either throwing away much of your data or hiring a data cleanser to take a ton of junk data and attempting to merge it all and fix it all.
With 43North, we spent at least 40 hours cleansing their data to move them over to Copper. Streak was a huge mistake and I had majorly regretted recommending it back in the day to them.
Their team also had slight frustration because things like mail merge weren't as "simple" with Copper, so they got used to some of the convenience, though at a major cost. John in ABC department could move so quickly doing things, but at the expense of having a complete mess of data disconnected from the rest of the team. Is it worth it? Well, you tell me — you're not building company IP when you do that, you're trying to do things quickly with duct tape.
All that being said, when you get to tens of thousands of records, your Gmail is going to be loading so slowly which leads to other frustrations. It's a good temporary solution for a small team that is okay with moving quick and breaking things, but you'll have to move off of it in the future with scale, and when you do, companies like us will recommend starting from scratch as that is easier than it will be to fix what already exists.