There are many ways you could segment your market for your marketing and sales motions: from industries to personas to company size to geographies (and for B2C companies, major demographics like age, gender, etc come into play). The questions you need to ask are "Do these groups of customers have fundamentally different needs for our product?" and "Would we acquire these groups of customers in different ways?". Wherever the differences are greatest, you'll want to start there.
Another key consideration is resourcing: do you have enough people to create focus areas among your marketing/sales/post-sales teams? Do you have enough bandwidth to create personalized messaging and campaigns? Do you have sufficient budget to split paid campaigns into different segments?
Market research data or industry reports may be able to tell you what some of the largest groups are that you should go after, but you'll also inherently start to learn this when you start running marketing campaigns & selling.
For example, it was immediately apparent to our sales team that institutional investors (hedge funds, equity analysts) spoke about research differently and had different objectives than insights and marketing professionals. Then we started to notice that B2B SaaS companies had different research challenges targeting niche groups of professionals than consumer goods companies doing research with the general population. As our teams scaled, this is where we made distinctions and split up our sales and marketing teams into "pods" to focus accordingly. Market research will come into play again when we need to make a strategic decision for which vertical to take on next and how to position ourselves differently.
It's also worth noting that you may use a totally different framework for customer segmentation to structure your post-sales teams. For example we have segments for self-serve vs sales-assisted customers. Self-serve customers are segmented by subscription type, and sales customers are segmented by spend level to determine the level of support/service they can receive so that our team can scale their efforts.