The building has, of course, changed in many ways over the years. The West Wing wasn't built until the turn of the 20th century, while the Oval Office followed eight years later and the East wing was majorly altered in 1946, barely a few years before Truman's renovations. But I don't see where there is any indication in any of its history that the White House needs a ballroom that is twice the size of the main building.