Olivia Tremor Contol's original line-up was childhood friends Bill Doss, Will Cullen Hart, and Jeff Mangum. As this trio, the band released their debut vinyl EP, California Demise, on the Elephant 6 label in 1994. Shortly after, Jeff left the band to concentrate his efforts on Neutral Milk Hotel, though he returned several times throughout the band's existence to lend his talents, both in studio and on stage. OTC and NMH toured together a lot in the mid/late 90s, and would frequently join in during each other's sets.
In his first appearance on stage in years, Jeff joined the Olivia Tremor Control during their show on August 2, 2005 at New York's Bowery Ballroom. He helped sing a song in the middle of the set, "I Have Been Floated", and returned again for the encore, taking lead vocals on the song "Shaving Spiders". The crowd went nuts for him, and he delivered an intense performance. The next night (same venue), Jeff again guested on "I Have Been Floated", and played drums on the encore, "Memories Of Jacqueline 1906".