Modeling DTS buildings for polysoup

Now that torque has polysoup collision, it's definitely much easier for a lone artist to use the same tool for buildings and organics. And as the lone artist for my project, I have some questions for those of you who may have done buildings in a non brush-based 3d package (blender, 3ds max, XSI ,etc).
I have a lot of experience with modeling organics in blender, but almost none modeling buildings and architecture.
So what's the general approach to doing this? Since I have only ever done buildings in QuArK, the first thing I tried to do in blender was make each wall a separate object, and use each object like a brush. This is kind of a problem though when it comes to removing internal faces and UV texturing.
The second thing I tried was to draw the floorplan for the building in a vector graphics app (inkscape) and then import it and exturde the walls upward. The problem with this approach is the roofing can get really nasty, and you have to manually face all of the floor and wall sections.

So whats the trick? does anyone do this? I could do just fine if the building didn't need any interior space (like RTS buildings), but for my project it does.

Thanks in advance,
Justin