That's right. If your city has visible homeless population, it also has shelters, soup kitchens, churches that help them, etc.

Donate to some of those organizations, and only ever hand out the list of all places where to get food/shelter/counseling.