Part of the purpose of a CRM is not just storing contact records (that's the most bare-bones use-case of a CRM) but rather to be a central hub of communication across your company.
You email a customer, and someone else on your team emails that same customer. How do you know this happened? You don't when using Notion as a CRM. So customers may get 2-3 emails from different team members and it's a mess. We've heard scenarios of teams "copy and pasting" emails from clients into a Notion document - after trying to keep up with it for a few weeks, it not only fell by the wayside because this is a horrible time-sink.
A proper CRM is going to ingest emails automatically, count the number of interactions you've had with someone, and nudge you when you're not doing a good job at staying on top of leads. Notion as a CRM on the other hand will not be able to handle this type of communication tracking.
Using Notion as a CRM won't show you email history, making it more likely for you to email the same leads as your team members, resulting in messy experience.

