How many visitors do I need for Core Web Vitals score?

Nobody knows. Tests I conducted in the past pointed out that your site needs around 400 pageviews a month per device (mobile, tablet, desktop), using Chromium to get a Core Web Vitals origin summary when using PageSpeed Insights. Do note that this might be the absolute minimum.

When a site doesn't have enough visitors, chances are you need to find other ways to rank higher. Do note: improved pagespeed will still impact conversion positively, no matter the amount of visitors or passing the Core Web Vitals assessment.