Must I Destroy The Nest??

I had yellow jacket wasps nest in a metal rectangular budgie nest box that I'd left on a ledge inside my greenhouse one year. No damage to the box proper, including the hardwood nest tray on the bottem, but I had some fun cleaning out all the old wasp nesting material come winter once the colony was dead! Pretty messy... I don't, however, believe that yellow jackets would deliberately destroy drywall, aside for some initial experimental chewing on the paper exterior. And if that interior portion of the drywall sheeting did get exposed, well, I can see the humidity within the nest proper causing a bit of localized crumbling unless the wasps saw fit to re-paper the 'walls' again beforehand more to their liking. Likely a bit of staining, too. Baby wasp poop and such. Personally, as long as your wasps have plenty of room to expand their nest and aren't so cramped that they feel forced to start chewing holes if only to try and expand their living space, I'd put up with them myself in a like situation and just accept any slight cosmetic damage incurred. They DO make terrific natural pest-control groundskeepers who will diligently work your yard and nearby surroundings as long as the good weather holds. I never did see a single insect pest inside my greenhouse that year when the wasps were in residence, not even an aphid.

Wasps do chew wood when collecting the material to make their nests, but I've observed that it's usually old wood, with a surface that's starting to lift, peel or fuzz up a bit. My local wasps most love my old fence planks and a certain old wooden door directly exposed to the outside elements. When they mix the greyish materials they collect in my yard with scrapings from a neighbour's red-stained garden hut, it makes for some really prettily patterned nests at times!

Can't say much about your whole colony surviving your winter as I'm in a climate zone with hard, defined winters that kill off all but overwintering queens every year. So, at most, our local wasps only have time to build nests that get about American football-sized before dying off and having to start all over. I don't know how big the colonies could become in more southerly regions or whether they're even self-perpetuating or 'designed' to collapse and die off on a roughly annual basis.

In closing, I'm glad you're not keen on killing your wasps and fine with having them around. I've always found yellow jackets to be relatively non-aggressive, as wasps go, and they seem smart enough to distiguish between individual humans and recognize those who become familiar to them and who mean them no harm. I never had any trouble with my greenhouse wasps, even when I had to work within a few feet of their nest, and aside from the occasional nest guardian lifting off to hover in front of my face for a moment and check me out or the odd forager landing on my long hair and hunting through it, presumably for bugs(!), they never bothered me. I would never have let a stranger, someone new to the wasps, go into the greenhouse, though.

I can tell you that insects only grow as long as they are larvae (wasps, beetles, butterflies, flies...) or nymphs (dragonflies, cicadas, stinkbugs, earwigs...). Once they've pupated/had their last molt and have become an adult insect, they don't grow anymore. The specimen you found will likely be some other species. If you take a picture and post in in the ID section, someone will surely help you out.

I assume your yellow jackets, like European wasps, may cause some minor damage to your walls, shave nearby wooden planks to obtain building material for their nest, kill lots of other insects for food and near the end of summer develop an interest in your lemonade. As long as their nest is not in a place where you may come into contact with it, I wouldn't bother.