The video was filmed at a training center for firemen, located in Wards Island, New York.
"Make a hole with a gun perpendicular to the name of this town in a desktop globe."
This line is based on a Pogo comic, in which a character shoots a hole through a globe to find out what is on the exact other side of the world.
"Water spirals the wrong way out the sink."
In general, water drains counter-clockwise in the Northern Hemisphere, and clockwise in the Southern. This might or might not be due to the Coriolis Effect, which doesn't really affect sinks. (If anyone has any information on this, I'd like to hear/see it. I've heard contradictory things about the Coriolis Effect and drains.)
"All alone at the '64 World's Fair."
From the Early Years Handbook:
"The World's Fair-- a now-defunct world-famous bash celebrating technology & invention-- was a huge event in its heyday, drawing thousands upon thousands of wide-eyed spectators to observe the festival. John Linnell visited the '64 World's Fair as a child. Obviously, it made a profound impression on him...Flansburgh has been known to hunt all '64 World's Fair items obsessively, recently proudly noting that he had finally located the '64 World's Fair souvenir record."
"Eighty dolls yelling 'small girl after all.'"
The actual lyrics were "It's a small WORLD after all." The dolls that sing this can now be found at Walt Disney World, but they apparently had a brief gig at the World's Fair. In some live versions, the Johns have replaced the usual bridge with "It's a small girl after all," sung to the tune of "It's A Small World."
"I don't want the world. I just want your half."
This line apparently came from an argument about money. The Johns did not actually see it painted on a bridge, although the song has apparently inspired some TMBG-related graffiti.