That is not what herd immunity is. It does not remove the disease from the population, it just limits its spread to the point where it doesn't have a high likelihood of spreading to the group herd. The term was coined from during a cow early fetus death problem, and at some point was related to the human population. Measles, which has been considered "eradicated" still has 1000+ cases each year due to people refusing to vaccinate. The other issue is often like-minded people (anti-vaxxers for example) will go to the same churches, etc., so even if the population in general is largely immune, there can still be a spread.
For many years, adults had herd immunity from being exposed to diseases that childen could still get.
Chicken pox is a good example of a virus that we as a population had herd immunity to prior to vaccination. That doesn't mean people still couldn't get it that never had it, but the likelihood was low. The more contagious a virus is, the more people will need to have had it (vaccines not withstanding) in order to achieve herd immunity. Interestingly, since the chicken pox vaccine is relatively new, we still don't know if those that received the vaccine will still be subject to getting shingles.
Many highly contagious viruses, especially those susceptible to mutation, never allow us to truly get to herd immunity (think influenza). The flu kills on average 0.5% of the population. That number would be higher without a flu shot. That is probably where we'll end up with for Corona. Without the vaccine the large majority of the population will need to get it to have any sort of herd immunity. Thankfully there is a vaccine. Hopefully it can be distributed fast enough to slow down mutations that are resistant to the vaccine.
Thank you, well done, you saved me some typing!