If you can, and the sky's are clear (Blue) I would do it now. Sometimes the bees will do that if they have a new queen going out to make her mating flights. I think they do it so that if there is a preditor in the area waiting for a nice, slow, plump queen, they saturate the area to protect her as she takes off.
I had it happen. Saw a warm start, I sat and waited until the queen came out and I caught her and put her in a nuc. The swarm would not follow her into the box. I added some bees to the nuc and then ended up with the original hive being queenless.
Look for capped queen cells while you are in there. Never destroy all of your Q cells. You will probably end up Q less.
Jim