Thanks to collaboration with The Creating Digital History class at New York University, Pastmapper now features the complete census records for West 9th Street in Manhattan from 1880 through 1940, as well as building history information for the structures on the street over time. Every person, along with their occupation, sex, age, nationality, and other details, are included in an easily navigable map, with clickable building outlines linked to lists of the residents from each year. Over 3700 individual listings have been compiled, and the residents reveal the story of this tony block.
Census entries and building history data have never been presented together in a historical mashup like this, but the result is a higher contextual understanding than either dataset would provide on its own. I’m eager to explore the potential of this presentation method for other cities too.
Check out the map here.
Learn more about the project in the earlier Pastmapper blog post here, or visit the Creating Digital History site here. Many thanks are due to the students of the class, their professor Cathy Moran Hajo, PhD, and here in San Francisco to Brian Mount for essential help in building new tools with the Google Maps API.
Some more detail on the project follows:
Making Sense of the Census
The census wasn’t really intended to be put on a map.
Census records are useful for tracking trends within large swathes of population. They can help reveal demographic shifts from the national level down to region, state, county, and city level. And although the data exist all the way down to the level of individuals, it’s actually very difficult to use it that way. Using census data to research a particular building, for instance, is a terribly complex process. Why? Because of the way census data were gathered.
Individual census employees, called enumerators, were assigned geographic areas, called enumeration districts. Enumerators were given a period of time in which to record all the residents who lived in their assigned district. The census records they kept were like big blank notebooks, so their handwritten entries are sorted not by building, but by the sequential order in which the enumerator spoke to each person. An enumerator might have walked part of a street, then skipped over to another street, only to return to complete the first street weeks later. The result is that one sheet might be neat and complete, recorded in order and reporting building residents in a geographical order (this is rare), whereas another might contain fragments of resident records from different buildings, with their remainders scattered throughout other sheets. Building numbers are recorded in the individual entries, but the sheets themselves are labeled with, and easily sorted by, their enumeration district number.
Because of this, enumeration districts have become the de facto geographic lens for viewing census data. District borders change from census to census, but there are clear maps for each census year, and any analysis of data (occupation, age, sex, nationality) can be made reasonably useful using these districts. Breaking down the data by another geographic factor, such as a building number, would require careful transcription and assignment of each person to the appropriate building. Building numbers aren’t recorded in each person’s entry, but are instead only written once for the first person listed for a building. This means that the original handwritten pages must be inspected in order to make the right assignments of people to buildings.
The first step was to transcribe the census, recording every detail of every listing into a spreadsheet. This laborious task was divided up among students of the Creating Digital History class, and without this hard work, the project would not have been possible at all.
Next came creation of the maps and the building outlines. Unlike the 1853 San Francisco map, the West 9th Street map required that listings be shown as more than just dots on a map; they needed to be shown as residents within a building. By importing current-day building shapes from the New York City OpenData site, I was able to create KML shapes for buildings that still exist, then add them to the map for every year following their construction date. For buildings that have since been demolished, I relied upon the traced building outlines from the NYPL’s Map Warper output of an 1854 real estate map. Piecing together the details and deciding which building outline corresponded to which address was a job of detective work. Clues came from the Greenwich Village Historical Survey of 1969 and from the census records themselves. Curiously, the census confirmed the existence of at least three ‘ghost buildings’, or structures that were built after 1854 but demolished before 1969, preventing them from being captured in either of my other sources of information. These ghost buildings are still being added to the map, and will appear after I’ve conducted further research to confirm their likely building footprints.
Several of the students identified a problem with the 1853 San Francisco map that I hadn’t considered. In the original interface, there was no automatic link between a change to the map year and a change to the directory data (the dots) shown on the map. In other words, a user could change both the map and the data independently of one another. This meant that locations of old businesses could be viewed on a modern-day map, providing a contextual orientation between time periods. But several users were confused to see an 1853 listing on a 2013 map, seeing it as a mistake, or worse, inferring an impossible continuity spanning 160 years.
Visually, the solution seemed simple – just link the map with the data. But actually solving it was a big challenge, and involved digging deep into the Google Maps API to find a way to force the map data to change whenever the map layer changed. I was quickly stumped. Luckily, uber-coder Brian Mount came to the rescue, offering several hours of his time to solve the problem. Thanks to Brian, the map now works the way many users expected it should; changing the year now automatically changes the active data later. Data from 1880 will only appear on a map of 1880, etc.
Scaling the Project
The methods developed during this project involve a lot of work, but they can be easily duplicated and repeated for other regions. The NYU students who transcribed data for the project know very well how tedious the task can be. And even after receiving the invaluable transcriptions, my process of cleaning up the data and cross-referencing it with building data was another huge undertaking.
Soon I’ll be posting a step-by-step guide to the process. I am openly soliciting ideas for scaling this process to include more regions. In the meantime, for anyone interested in doing more work with the West 9th information, the raw data for the West 9th Street census project are here as a comma-separated (CSV) file*:
*Keep in mind that the file contains some annotations for unresolved items. Also, many spreadsheet programs have a hard time with 19th-century dates, so the recording dates for 1880 may display as dates in 1980, etc. Please direct any questions about the data to me at firstname.lastname@example.org