it sounds like a tough call - in hindsight, maybe you would have done things differently, but when faced with the dog's owner, you had to hand them back and as others have said, the guy may very well be going to get the dog to a vet - he at least came looking for them.
Funny thing happened to me yesterday that has had me agonising all night and day afterwards too - I'll tell you the story, it might help to know that we all find ourselves in questionable situations trying to help animals.
Yesterday I saw a loose staffy cross on my way home from a three hour hike with three of my dogs. We were all pretty hot and tired, but the pup was friendly and I was able to catch it pretty easily. I asked around the industrial area and nobody claimed him, so I set off with the little guy capering around my dogs, intending to walk straight past home and go to the vet to see if he was microchippped.
A block later, a guy pulled up in a truck and said it was his dog, which I was initally thrilled about. Until he went on to say the dog "always runs away and doesn't listen", etc. He claimed it was a pit bull, bred by a friend of a friend, but it was more like a white staffy cross. He then asked if I wanted the dog. My first reaction was "I already have seven, four of whom are rescues". He then kept me talking, clearly positive about the dog in one sense, but also at a loss. We discussed training, exercise, the fact that a seven month old pup is a lot of work, etc.
He then commented, I think as proof of his good ownership that he feeds him twice daily and that he had just this morning put round-up on the dog to kill fleas. As you can imagine, I did a double take, to which the guy then said he had washed it off anyway. There was a greasy stain all over the dog's back, but he seemed okay. I was horrified, to put it mildly. By now, I was thinking I had to get the dog away from the guy and get in touch with some friends who foster and rehome dogs, so I offered to take the dog. The guy now seemed unsure and instead asked me for my number in case he changed his mind and really did want to give the dog away. He said that I had given him some good suggestions and he would keep trying. I said I was glad he had washed the round-up off the dog because it wasn't safe and said I could find a home for the dog if he wanted to ring me.
I walked off and I don't think it's likely I will hear from the guy now.
Part of me wishes I had just said yes to his offer of the dog in the first place and then I could worry about what to do with it later. But I took on a seventh dog two weeks ago under urgent circumstances and I have been telling myself not to get sucked into anymore sob stories as I am at my limit in terms of time and money.
Of course I worry that the dog will end up handed off to someone with as little idea as his current owner, or being hit by a car after getting out again, or just at the pound/lost (the guy admitted he is not microchipped).
But, the guy did seem to like the dog a lot and maybe he will now feel encouraged to do a bit more.
All we can do is what is in our power at that moment, I guess.
Hopefully in both our situations, we have tweaked the conscience of the person responsible for that dog's well-being.