Tag Archives: Maps

Who Picked Up The Check?

Adventures in Data Exploration

In November 2012 the United States Postal Service reported a staggering deficit of $15.9 billion. For the historian, this begs the question: was it always this bad? Others have penned far more nuanced answers to this question, but my starting point is a lot less sophisticated: a table of yearly expenses and income.


US Postal Department Surplus (Gray) or Deficit (Red) by Year

So, was the postal department always in such terrible fiscal shape? No, not at first. But from the 1840s onward, putting aside the 1990s and early 2000s, deficits were the norm. The next question: What was the geography of deficits? Which states paid more than others? Essentially, who picked up the check?

Every year the Postmaster General issued a report containing a table of receipts and revenues broken down by state. Let’s take a look at 1871:


1871 Annual Report of the Postmaster General – Receipts and Expenditures

Because it’s only one table, I manually transcribed the columns into a spreadsheet. At this point, I could turn to ArcGIS to start analyzing the data, maybe merging the table with a shapefile of state boundaries provided by NHGIS. But ArcGIS is a relatively high-powered tool better geared for sophisticated geospatial analysis. What I’m doing doesn’t require all that much horsepower. And, in fact, quantitative spatial relationships (ex. measurements of distance or area) aren’t all that important for answering the questions I’ve posed. There are a number of different software packages for exploring data, but Tableau provides a quick-and-dirty, drag-and-drop interface. In keeping with the nature of data exploration, I’ve purposefully left the following visualizations rough around the edges. Below is a bar graph, for instance, showing the surplus or deficit of each state, grouped into rough geographic regions:


Postal Surplus or Deficit by State – 1871

Or, in map form:


Postal Surplus (Black) or Deficit (Red) by State – 1871

Between the map and the bar graph, it’s immediately apparent that:
a) Most states ran a deficit in 1871
b) The Northeast was the only region that emerged with a surplus

So who picked up the check? States with large urban, literate populations: New York, Pennsylvania, Massachusetts, Illinois. Who skipped out on the bill? The South and the West. But these are absolute figures. Maybe Texas and California simply spent more money than Arizona and Idaho because they had more people. So let’s normalize our data by analyzing it on a per-capita basis, using census data from 1870.


Postal Surplus or Deficit per Person by State – 1871

The South and the West may have both skipped out on the bill, but it was the West that ordered prime rib and lobster before it left the table. Relative to the number of its inhabitants, western states bled the system dry. A new question emerges: how? What was causing this extreme imbalance of receipts and expenditures in the West? Were westerners simply not paying into the system?


Postal Receipts and Expenditures per Person by Region – 1871

Actually, no. The story was a bit more complicated. On a per-capita basis, westerners were paying slightly more money into the system than any other region. The problem was that providing service to each of those westerners cost substantially more than in any other region: $38 per person, or roughly 4-5 times the cost of service in the east. For all of its lore of rugged individualism and a mistrust of big government, the West received the most bloated government “hand-out” of any region in the country. This point has been driven home by a generation of “New Western” historians who demonstrated the region’s dependence on the federal government, ranging from massive railroad subsidies to the U.S. Army’s forcible removal of Indians and the opening of their lands to western settlers. Add the postal service to that long list of federal largesse in the West.

But what made mail service in the West so expensive? The original 1871 table further breaks down expenses by category (postmaster salaries, equipment, buildings, etc.). Some more mucking around in the data reveals a particular kind of expense that dominated the western mail system: transportation.


Transportation Expenses per Person by State (State surplus in black, deficit in red) – 1871

High transport costs were partially a function of population density. Many western states like Idaho or Montana consisted of small, isolated communities connected by long mail routes. But there’s more to the story. Beginning in the 1870s, a series of scandals wracked the postal department over its “star” routes (designated as any non-steamboat, non-railroad mail route). A handful of “star” route carriers routinely inflated their contracts and defrauded the government of millions of dollars. These scandals culminated in the criminal trial of high-level postal officials, contractors, and a former United States Senator. In 1881, the New York Times printed a list of the ninety-three routes under investigation for fraud. Every single one of these routes lay west of the Mississippi.


Annual Cost of “Star” Routes Under Investigation for Fraud – 1881 (Locations of Route Start/End Termini)

The rest of the country wasn’t just subsidizing the West. It was subsidizing a regional communications system steeped in fraud and corruption. The original question – “Who picked up the check?” – leads to a final cliffhanger: why did all of these frauds occur in the West?

A Dissertation’s Infancy: The Geography of the Post

A history PhD can be thought of as a collection of overlapping areas: coursework, teaching, qualifying exams, and the dissertation itself. The first three are fairly structured. You have syllabi, reading lists, papers, classes, deadlines. The fourth? Not so much. Once you’re advanced to candidacy there’s a sense of finally being cut loose. Go forth, conquer the archive, and return triumphantly to pen a groundbreaking dissertation. It’s exhilarating, empowering, and also terrifying as hell. I’ve been swimming through the initial research stage of the dissertation for the past several months and thought it would be a good time to articulate what, exactly, I’m trying to find. Note: if you are less interested in American history and more interested in maps and visualizations, I would skip to the end.

The Elevator Speech

I’m studying communications networks in the late nineteenth-century American West by mapping the geography of the U.S. postal system.*

The Elevator-Stuck-Between-Floors Speech

From the end of the Civil War until the end of the nineteenth century the US. Post steadily expanded into a vast communications network that spanned the continent. By the turn of the century the department was one of the largest organizational units in the world. More than 200,000 postmasters, clerks, and carriers were involved in shuttling billions of pounds of material between 75,000 offices at the cost of more than $100 million dollars a year. As a spatial network the post followed a particular geography. And nowhere was this more apparent than in the West, where the region’s miners, ranchers, settlers, and farmers led their lives on the network’s periphery. My dissertation aims to uncover the geography of the post on its western periphery: where it spread, how it operated, and its role in shaping the space and place of the region.

My project rests on the interplay between center and periphery. The postal network hinged on the relationship between its bureaucratic center in Washington, DC and the thousands of communities that constituted the nodes of that network. In the case of the West, this relationship was a contentious one. Departmental bureaucrats found themselves buffeted with demands to reign in ballooning deficits. Yet they were also required by law to provide service to every corner of the country, no matter how expensive. And few regions were costlier than the West, where a sparsely settled population scattered across a huge area was constantly rearranged by the boom-and-bust cycles of the late nineteenth century. From the top-down perspective of the network’s center, providing service in the West was a major headache. From the bottom-up perspective of westerners the post was one of the bedrocks of society. For most, it was the only affordable and accessible form of long-distance communication. In a region marked by transience and instability, local post offices were the main conduits for contact with the wider world. Western communities loudly petitioned their Congressmen and the department for more offices, better post roads, and speedier service. In doing so, they redefined the shape and contours of both the network and the wider geography of the region.

The post offers an important entry point into some of the major forces shaping American society in the late nineteenth century. First, it helped define the role of the federal government. On a day-to-day basis, for many Americans the post was the federal government. Articulating the geographic size and scale of the postal system will offer a corrective to persistent caricatures of the nineteenth-century federal government as weak and decentralized. More specifically, a generation of “New Western” historians have articulated the omnipresent role of the state in the West. Analyzing the relationship between center and periphery through the post’s geography provides a means of mapping the reach of federal power in the region. With the postal system as a proxy for state presence, I can begin to answer questions such as: where and how quickly did the state penetrate the West? How closely did it follow on the heels of settler migration, railroad development, or mining industries? Finally, the post was deeply enmeshed in a system of political patronage, with postmasterships disbursed as spoils of office. What was the relationship between a communications network and the geography of regional and national politics?

Second, the post rested on an often contentious marriage between the public and private spheres. Western agrarian groups upheld the post as a model public monopoly. Nevertheless, private hands guided the system’s day-to-day operations on its periphery. Payments to mail-carrying railroad companies became the department’s single largest expenditure, and it doled out millions of dollars each year to private contractors to carry the mail in rural areas. This private/public marriage came with costs – in the early 1880s, for instance, the department was rocked by corruption scandals when it discovered that rural private contractors had paid kickbacks to department officials in exchange for lavish carrying contracts. How did this uneasy alliance of public and private alter the geography of the network? And how did the department’s need to extend service in the rural West reframe wider debates over monopoly, competition, and the nation’s political economy?

Getting Off The History Elevator

That’s the idea, at least. Rather than delve into even greater detail on historiography or sources, I’ll skip to a topic probably more relevant for readers who aren’t U.S. historians: methodology. Digital tools will be the primary way in which I explore the themes outlined above. Most obviously, I’m going to map the postal network. This entails creating a spatial database of post offices, routes, and timetables. Unsurprisingly, that process will be incredibly labor intensive: scanning and georeferencing postal route maps, or transcribing handwritten microfilmed records into a database of thousands of geocoded offices. But once I’ve constructed the database, there are any number of ways to interrogate it.

To demonstrate, I’ll start with lower-hanging fruit. The Postmaster General issues an annual report providing (among other information) data on how many offices were established and discontinued in each state. These numbers are fairly straightforward to put into a table and throw onto a map. Doing so provides a top-down view of the system from the perspective of a bureaucrat in Washington, D.C. For instance, by looking at the number of post offices discontinued each year it’s possible to see the wrenching reverberations of the Civil War as the postal system struggled to reintegrate southern states into its network in 1867:

Post Offices Discontinued By State, 1867
(Source: Annual Report of the Postmaster General, 1867)

The West, meanwhile, was arguably the system’s most unstable region. As measured by the percentage of its total offices that were either established or discontinued each year, states such as New Mexico, Colorado, and Montana were continually building and dismantling new nodes in the network.

Post Offices Established or Discontinued as a Percentage of Total Post Offices in State, 1882
(Source: Annual Report of the Postmaster General, 1882)

Of course, the broad brush strokes of national, year-by-year data only provide a generalized snapshot of the system. I plan on drilling down to far more detail  by charting where and when specific post offices were established and discontinued. This will provide a much more fine-grained (both spatially and temporally) view of how the system evolved. Geographer Derek Watkins has employed exactly this approach:

Screenshot from Derek Watkins, “Posted: U.S. Growth Visualized Through Post Offices” (25 September 2011)

Derek’s map demonstrates the power of data visualization: it is compelling, interactive, and conveys an enormous amount of information far more effectively than text alone. Unfortunately, it also relies on an incomplete dataset. Derek scraped the USPS Postmaster Finder, which the USPS built as a tool for genealogists to look up postmaster ancestors. The USPS historian adds to it on an ad-hoc basis depending on specific requests by genealogists. In a conversation with me, she estimated that it encompasses only 10-15% of post offices, and there is no record of what has been completed and what remains to be done. Derek has, however, created a robust data visualization infrastructure. In a wonderful demonstration of generosity, he has sent me the code behind the visualization. Rather than spending hours duplicating Derek’s design work, I’ll be able to plug my own, more complete, post office data into a beautiful existing interface.

Derek’s generosity brings me back to my ongoing personal commitment to scholarly sharing. I plan on making the dissertation process as open as possible from start to finish. Specifically, the data and information I collect has broad potential for applications beyond my own project. As the backbone of the nation’s communications infrastructure, the postal system provides rich geographic context for any number of other historical inquiries. Cameron Ormsby, a researcher in Stanford’s Spatial History Lab, has already used post office data I collected as a proxy for measuring community development in order to analyze the impact of land speculation and railroad construction in Fresno and Tulare counties.

To kick things off, I’ve posted the state-level data I referenced above on my website as a series of CSV files. I also used Tableau Public to generate a quick-and-dirty way for people to interact with and explore the data in map form. This is an initial step in sharing data and I hope to refine the process as I go. Similarly, I plan on occasionally blogging about the project as it develops. Rather than narrowly focusing on the history of the U.S. Post, my goal (at least for now) is to use my topic as a launchpad to write about broader themes: research and writing advice, discussions of digital methodology, or data and visualization releases.

*By far the most common response I’ve received so far: “Like the Pony Express?” Interestingly, the Pony Express was a temporary experiment that only existed for about eighteen months in 1860-1861. In terms of mail carried, cost, and time in existence, it was a tiny blip within the postal department’s operations. Yet it has come to occupy a lofty position in America’s historical memory and encapsulates a remarkable number of the contradictions and mythologies of the West.

AAHC Recap (Afternoon)

The first session of the afternoon at the AAHC conference was by Patrick Murray-John as he discussed “Interlinking Who, What, Where, When, Why, and How We Teach into a Giant EduGraph.” He gave an entertaining presentation that described the benefits and challenges of a semantic web within education. One story stood out in particular as an example for the need for better efforts in the field. Two courses might use Frankenstein as an assigned text: a class in the chemistry department that examined the portrayal of scientists in literature and an introductory English literature course. Unfortunately, the only people who know that both classes are using the same text is the school book store. If the course syllabi were located within a strong semantic web, there could be tremendous potential for interdisciplinary discovery and interaction within a university.

The second session of the afternoon was “Mapping Our Archives,” by Tim Sherratt of the National Archives of Australia. Sherratt gave an incredible presentation, as he demonstrated his work with the records of Australian servicemembers during the First World War in Mapping Our Anzacas. First up, he used Google Earth to show how he mapped where each of the 300,000 plus soldiers were from. It was remarkably powerful, as he used the platform dynamically in order to visualize the origins and density of this segment of the population. In particular, I was struck at how effectively it took advantage of the interface of Google Earth, and its re-population of point data when you zoom in, so even as you get more and more detailed, it seems like the dots just keep multiplying. The map was one point of entry for users of the database, with another critical point of entry being the opportunity for users to upload multimedia to a “scrapbook” of individual servicemembers profiles. Although some curators expressed (standard) anxiety about losing credibility and content control, they were overwhelmed at the number and quality of contributions from “laypeople,” such as this one of Donald Addison:


Tim also shared his crafting and tweaking of the database for the servicemembers’ records. Each record was scanned, and many of them included photographic sets of the individuals along with vitals and descriptions of their service. He then used CoolIris (which I discussed in one of my first blog posts) to create a virtual gallery of these archival documents, as you could move along a 3-D wall with rather haunting profile photographs and corresponding information (almost like an early 20th-century version of Facebook). He also showed a similar application of CoolIris for other archival records, including those of immigrants and laborers. I’ve always been somewhat skeptical of CoolIris as a visualization technique, always thinking it did little more than create a kind of cool way to zoom around and look at pictures. But Tim substantially changed my mind by demonstrating its power as an interface – without a doubt, being able to see an array of documents displayed in front of you, seeing the faces of Chinese immigrant children looking out at you, was a singularly powerful use of archival material that significantly enhanced the viewer’s interaction with them.

Tim’s presentation got rave reviews from its audience members, and I think I speak for all of them when I say that his use of dynamic technical interfaces (which included not only Google Earth and CoolIris, but also Tumblr and Greasemonkey scripts) was one of the most elegant, powerful, and accessible examples of bringing archival material to life that I’ve seen yet. Tim offered a fantastic glimpse into what is possible when you combine the power of digital applications with the subtlety and thoughtfulness of historical exploration.

The final session of the afternoon was a presentation from Susan Garfinkel, Judy Graves, and Jurretta Heckscher from the Library of Congress. Although I was a little fried from seven hours of absorbing and processing a host of new thoughts and ideas, I always love supporting our friends at the LOC, who have put a lot of time, thought, and effort into advancing and experimenting with digital history. Their presentations brought up several interesting points. First, they stressed that a series of surveys have shown that faculty members are less than enthusiastic about actually utilizing digitization tools and resources that their libraries provide. Additionally, the LOC has decided to expand their earlier initiative to post photographic content on Flickr Commons by posting audio and video content to iTunes and YouTube. The presentation reinforced my optimism that our national library has the courage to pursue such a variety of digital initiatives.

I drove out of George Mason’s campus feeling exhausted and inspired. The conference did a superb job of bringing together and enhancing a community of digital humanists (made even stronger by the constant Twittering activity during the day). This community isn’t necessarily representative of either the general public or even the wider historical community – in particular, I was struck at the rather overwhelming whiteness of the conference participants. I think more efforts could be made to encourage a wider diversity of perspectives and backgrounds within the field of digital history (a subject for another blog post). Nevertheless, the range, depth, and passion with which so many people are pursuing their projects gives me a strong sense of hope. I know that over the next several years I will undoubtedly struggle to balance the demands of training in a traditional graduate history program with my own passion for digital methodology and exporation. But events such as the AAHC conference bolster my faith in the incredible support system of fellow digital humanists that I can lean on as I walk down that path.

Digging for Treasure

When I embarked on a summer research project in 2006, I was lucky enough to have the chance to tag along with an archaeological team each Friday. The team, led by Lucianne Lavin and Marc Banks, was excavating the property of Venture Smith, whom I was researching. There was red tape galore – the site was on the property of a nuclear power plant that was in the process of being decommissioned. But through Lucianne and Marc’s kindness, I got a real taste for the grittier and messier cousin of history: archaeology.

I learned a lot about the discipline. Chief among these realizations was that I was not very good at it. My first day I learned how to dig a 30cm square by 1 meter deep test pit. Dripping with a potent combination of sweat and bug spray, I gamely attempted to dig straight down in even, 10cm increments, pouring the shovelfuls of dirt and stones onto my excavating partner’s sifter so she could look for artifacts. Needless to say our test pit gradually took on a fun-house mirror appearance, especially in comparison to the other, perfectly dug pits around us.

My partner suggested we switch off so she could fix some of my mistakes. I was relieved, until I realized that I had to sift dirt while looking for any kind of material culture. Easier said than done. She carefully showed me the difference between a piece of fire-baked pottery and a normal pebble. I nodded understandingly, confidently lifted up the sifter, and promptly realized I was at an utter loss to tell the difference between a rock and pottery shard. Over the weeks, I got slightly better – to the point where I was throwing out more rocks than pottery, but only just. They finally learned their lesson and ended up having me map out each boulder, tree, and test pit on the small section of property using a tape measure and a series of graphing sheets.

Despite my incompetence, I managed to gain a deep appreciation for both the immense challenges and rewards of archaeological research. There is something truly thrilling to be standing and digging on the very spot where your subject of historical research lived. Although I get the same thrill from a manuscript or document, they lack the tangible reality of a material artifact you excavate in the field. Marc and Lucianne’s analysis led to a series of insights into my research that I never would have gained – they discovered the remnants of a dry dock that implied extensive river trade and activity, a faded cart path that likely corresponded to one mentioned in a land deed, and the foundation of a two-bay house and at least two other structures on the homestead. All of these were exciting and valuable contributions.

On the other hand, I was met with the frustrations of what I saw as broad speculation. Perhaps it was due to the fact that I couldn’t tell the difference between a rock and a nail, but hearing theoretical conjecture about Venture Smith’s life and activities based on tiny bits of pottery raised up red flags for the careful, source-oriented historian in me. I am guessing it is simply another form of source evaluation, one that I am far less comfortable or adept with. Just as I would judge a historical record based on its author, writing style, and context, I’m sure Marc and Lucianne bring to bear an equally careful evaluation of material artifacts.

Finally, I can’t resist linking to an article from the Economist’s Technology Quarterly – titled “Armchair Archaeology.” The article discusses how archaeologists are using satellite imagery such as GoogleEarth to plan expeditions, identify sites, and do surprisingly complex analysis. One cool example is using imagery to identify the quarrying and transportation routes of pre-Hispanic obsidian stone: “Mapping these routes has helped archaeologists reconstruct production and trade patterns, and hence economic, social and political relations in the region…” And the best part? It’s free.

Scattered Links – 9/6/2008 (Map Edition)

The Washington Post ran an article on the separate efforts of Dan Alexander Hawkins and Dan Bailey to map historical Washington. In the words of the article’s author, “The two men came to their fascination with Washington’s history by very different paths — pencils vs. pixels — yet sometimes their goals appear nearly identical.” The article is well-written, and offers a real glimpse into the rewards and challenges of historical mapping, whether digital or analog. Also, I really need to get a tour of UMBC’s Imaging Research Center.

Meanwhile, the University of Richmond’s Voting America project has been getting some play in the blogosphere. I have not had the chance to fully play around with it, but Sterling Fluharty at PhDinhistory has reported problems when testing the site for accuracy compared to separately compiled statistics. I’d be interested to see if this issue gets resolved.

Finally, on a non-history note, if anyone likes beautiful infographics and maps, take a look at Aaron Koblin’s Flight Patterns. Koblin’s graphics of North American air traffic patterns appeared in Wired magazine and are very well-done. Be sure to not only look at the images, but watch the accompanying animations as well (my favorite is the 3-D blobular).

Scattered Links – 8/21/2008

Lisa Spiro has posted a great recap of her presentation “Doing Digital Scholarship” at the Digital Humanities 2008 conference. The presentation “focuses on a project to practice digital scholarship by relying on electronic resources, experimenting with tools for analyzing and organizing digital information, and representing ideas through multimedia.” All in all, I think the blog post is a wonderful introduction to digital scholarship, both as an overview and a jumping off point for further ideas. Spiro really displays that crucial trait necessary for a digital humanist: a seemingly unlimited willingness to try new approaches.

Errol Morris has yet another interesting post about interpreting photography and identifying fakes, inspired by the faked photographs of Iran’s missile launch several weeks ago. Also, if you haven’t read it, I’d highly recommend his three-part series “Which Came First,” which details his attempt to uncover the truth behind Robert Fenton’s famous “Valley of the Shadow of Death” photograph of the infamous Light Brigade skirmish. What I liked most about it was his microscopic attention to detail and an open willingness to crowdsource, as over a thousand people responded and lent advice, tips, and clues. He even did a recap of these comments, which shows a real embrace of the power of collective intelligence and digital media.

GOOD Magazine has put up an extremely sleek and user-friendly interactive graphic titled “Wanderlust: GOOD traces the most famous trips in history.” Included among these are not only the standard fare of Lewis and Clark, Charles Lindbergh, and Marco Polo, but also fictional accounts such as Pequod from Moby Dick, along with Journey to the Center of the Earth. Although I will point out that it’s phenomenally Euro/American-centric, I do applaud its interface and design. This rivals some of the NYTimes’ recent gold standard information graphics as far as usability, style, and depth.

Finally, if you’d like to get worked up and indignant, read Edward Luttwak’s “A Truman for our times.” His thesis is almost comical: “While anti-terrorist operations have been successful here and there in a patchy way, and the fate of Afghanistan remains in doubt, the far more important ideological war has ended with a spectacular global victory for President Bush.” But what makes me downright irate is his complete hijacking of history. In a characterization that smacks of imperialism, ignorance, and borderline racism, he breezily describes eight hundred years of Chinese history in this pithy statement:

“That describes everything that the Chinese are not, and have never been. The Chinese empire was aggressive and expansionist under the Yuan dynasty and again under the Qing. But one dynasty was established by horseriding Mongols, the other by horseriding Manchus, both the products of foreign warrior cultures. The Han Chinese prefer other pursuits. Perhaps they will change, as cultures sometimes do.

Historical determinism and unprofessionalism at its worst. I wouldn’t have been at all surprised if he then pulled a Lawrence of Arabia and ended with “A little people, a silly people – greedy, barbarous, and cruel.”

Olympic Medal Map

Great interactive graphic on the NYTimes that shows countries’ medal totals for each Olympics. You can use a slider on the top of the graphic to progress through time, and each country is represented by a varying sized circle, depending on how many medals it won that year. Not surprisingly, the United States and the Soviet Union (and later Russia) tend to dominate the one and two positions. What I found interesting was looking at each year and trying to figure out what was going on in the world and how that might have influenced the medal totals. The obvious example is 1980, when the US boycotted the Olympics. Also of note, Seoul in 1988 witnessed East Germany gain the second most medals at the games.

Finally, seeing the low number of medals going to the the southern hempisphere reminded me of the proportional maps at WorldMapper.org, which lends size to different countries based on the information it is displaying. I wish the folks at WorldMapper had some more historical data, so we could compare different factors (infant mortality, food production, etc.) with the NYTimes medal map. Also of note, the NYTimes map reinforces the problem with using proportional area/volume maps – specifically, humans are notoriously poor judges of area and volume, and particularly bad at judging the relative size of circles. A circle that has twice the area of another circle appears, to most people, as much smaller than it is. This leads to the necessity for scaling compensation with maps, as cartographers need to artificially increase the size of circles for humans to accurately estimate its relative size. John Krygier of Making Maps: DIY Cartography has a great post about the entire issue.