Sounds like queen wasps. I get them overwintering each year in my workshop and they can emerge anytime from Feb onwards. Last spring I killed about 5 of them over a few months - each one will make a nest and can produce scores of wasps. I even found one which had overwintered in a metronome - it was still sleeping when I found it so was easier to dispatch!
I've only once had a hornet in the house - make a deep, loud sound like a motor scooter, bigger than a queen wasp, and the bits which on a wasp are black are a deep brick red on a hornet. Latin for Hornet is vespa - perhaps why the motor scooter was called that???!