Vocalist and organist Victoria Legrand, who graduated from Vassar College in 2003, and guitarist Alex Scally, who graduated from Oberlin College in 2004, formed the band in 2004 after meeting in Baltimore's indie rock scene, producing music composed largely of organ, programmed drums, and slide guitar. Of the origins of the band name Scally said: "We’d been writing music, and we had all these songs, and then there was that moment where you say ‘what do we call ourselves?’ We tried to intellectualize it, and it didn’t work. There were different plant-names, Wisteria, that kind of thing. Stupid stuff. But, once we stopped trying, it just came out, it just happened. And it just seemed perfect. " In an interview with Pitchfork, Legrand addressed their being a two-member status thus: "[I]t's a way to challenge ourselves: What do you do when it's just the two of you? . . . [O]ne of the reasons this has been such a fulfilling experience for me is that with two people, it's so much easier to achieve things that feel exciting and new. "