My neighbor was the spokesman for the New Orleans police department. He gave a pretty detailed explanation of how illegal guns are used, and where they come from.

Most of them come from purchases at gun shows (as laws are considerably laxer on private sales). The serial numbers are filed off, then they're hidden in abandoned buildings by a person who rents the guns out to be used in murders. The same illegal guns are typically used in numerous murders, making it more difficult to trace them back to any particular person, if they are found.