I'm back to trying out some friction-fit assembly methods. I've got some 6" walls right now that assemble with slots on the sides and two on the top and bottom. The tabs that hold them together slip into the walls and hopefully out of sight. Floors can be permanently attached, or rested in place, I think. Tabs come in (currently) four flavors - "I", "T", "L" and "+" shapes. I might make a variant of the "T" tabs with a small lip rather than two of the full tabs. That'd be for resting floors on, and holding removable upper walls in place.
It's getting late. I hope to try to build a roof and truss tomorrow to see if it all holds together!