Did the swarm trap start out with any comb in it? Probably the bees were driven out by the SHB. Not much you can do about it. If you did have old comb in it, just don't leave any with pollen or honey, that will attract them.
I burned a couple of frames of SHB larvae this year too, squished almost hundreds, and vacuumed and drowned another several hundred. Some satisfaction, but surely not as much as a nice booming hive would have been :-x
Rick