Well, I wouldn't say you've 'let his teeth get bad' meegs, more his teeth have got to that stage from diet and lack of natural cleaners, that's all.
You can have the vet remove the left over plaque, and then keep an annual check on it each time you go in fo vaccinations or whatever...
Or you can remove it yourself. A good toothbrush, or even a cloth and wiping hard depending on the degree it has formed. I wouldn;'t suggest you use a tooth scaler unless you know how to use one for fear of injuring the dog's mouth, I'm sorry.
But sounds likethe bones have helped get rid of the majority of it, so that's great.