We present an empirical framework to study segregation that bridges the empirical literature on residential choice and the theoretical literature on neighborhood segregation. The former literature is based upon equilibrium empirical models of disaggregated choices, whereas the latter literature is concerned with the aggregate phenomenon of segregation, which is often studied theoretically in disequilibrium. Our framework explicitly allows for incomplete information, moving costs, and for the disaggregated households’ choices to be observed out of equilibrium. We also propose novel instrumental variables that exploit the logic of a dynamic choice model and can be constructed with no additional data requirements. A simulation procedure aggregates these choices to characterize the dynamic process of segregation. We illustrate our framework with an analysis of racial segregation of White, Black, Hispanic and Asian homeowners in the San Francisco Bay Area from 1990- 2004. We find that all homeowners react highly heterogeneously to neighbors of different races. Black and Hispanic segregation would increase by around 17% in the absence of any external shocks to the housing market, but White and Asian segregation would increase by only around 7%. Moving costs play a central role in keeping segregation at bay by maintaining a mismatch between the desired and the current neighborhoods of many households. This mismatch is mostly sustained by neighborhood amenities other than racial composition.