It depends on why they are acting aggressive. I can't see that taking four frames of honey would be causing it, especially since I imagine your flow is still pretty strong. Could it be something environmental that irritated them? Did someone mow around the hives lately, or could an animal be messing with them? How strong is the hive? Could they be experiencing robbing? Do they have a newly mated queen, who perhaps mated with some nasty drones? Maybe it's an older queen, but she's just working on the sperm of a nasty drone or two at the moment? Any more information you could give us would be helpful to try and diagnose it. How did they look last inspection? Anything changed in their environment? Things like that.