A cautionary, historic tale of data-based governance

Robert McNamara at a cabinet meeting
Robert McNamara at a cabinet meeting

Earlier this summer, Washington Monthly ran a piece by Haley Sweetland Edwards entitled “A Short History of Data-Driven Government.” The article focuses on a decidedly dispiriting story about Robert McNamara, who she calls the “father of data-based government.”

Edwards first describes how McNamara, a former army captain, learned about efficiency at Harvard Business School,  used his skills to climb the ladder at Ford Motor Company, and eventually landed in President Kennedy’s cabinet, as Secretary of Defense.

Here’s where the tale turns cautionary. Edwards writes that “McNamara applying those same ideas about efficiency, instituted the now-infamous ‘body count policy’ in the quickly escalating Vietnam War.” This policy, she explains, was later “roundly maligned by liberals and conservatives alike, underscoring the problem of data-driven policies that, by failing to quantify holistic goals, sometimes incentivize appalling behaviors.”

Edwards describes how data-driven governance soon faded out of fashion, until President Clinton revived it in the 1990s, training federal agencies how to measure success and create progress reports. Many of these ideas have carried through into the Obama administration, which requires a similar type of reporting from federal agencies. These efforts have seen success. Edwards mentions that, under Clinton’s initiative, the Coast Guard, “using data to map the problem of crew casualties in the towboat industry, was able to reduce fatalities by two-thirds from 1997 to 2000.” And under Obama, the Treasury’s Bureau of Engraving and Printing converted “to a centralized accounting system—a move that the Financial Management Service estimated could save the government $400 million to $500 million annually.”

Edwards cautions, however, that no “data-driven performance management tool” is “immune to the pitfalls made infamous by McNamara” because metrics are so easy to game.

I wish Edwards had done more to derive lessons from McNamara’s downfalls. The problem was a defective end-goal, not the data analysis itself. Thus, perhaps the biggest takeaway from revisiting this story is the reminder that data analysis in government cannot be an end in itself, the goals must be right-minded. This, of course, is always the biggest challenge in government—deciding which goals are right-minded. In business, the objective is typically easy to define: create sustainable profitability. In government, the objectives are much more complex to formulate, especially as one scales up from local to national governance, where even reaching acceptable goals is often an impossible feat. But as McNamara’s story shows, even with the “big data” available today, this process of defining goals must remain demanding because what we’re working toward makes all the difference.

Legal aspects of big data

big data sheriff's star“Big Data”— the business-world buzzword for the collection and analysis of massive amounts of data—has caught on with local government officials in the past few years as many cities have developed extensive data portals providing citizens access to heaps of public information like data from 311 calls. And its not only local governments getting involved, in 2012 the White House announced the “Big Data Research and Development Initiative,” through which federal agencies would commit funding toward collecting and analyzing “huge volumes of digital data.”

So, what sparked this interest in “Big Data”? In short, innovations in computing, particularly the ability to allow users to remotely access large data sets stored on third-party servers, i.e., “cloud computing.” But as attorney John Pavolotsky wrote last November in Business Law Today, “[w]hile business publications have written widely about Big Data, legal commentators have written sparingly on the subject.”

Pavolotsky goes on to note three areas of legal concern he see with Big Data: privacy, data security, and intellectual-property rights. He does not dwell long on data security and IP rights for long, other than to note that API licenses should be reviewed carefully to determine the permissible scope of data distribution. He focuses instead on privacy, arguing that, because of the “inherent squishness” of the legal standard applied to collection of cellphone or GPS data under the Fourth Amendment—which protects guards people’s “reasonable expectation of privacy”—perhaps legislatures should limit the length of time data can be stored.

I’d like to add one other interesting question surrounding Big Data, though it leans more economic than legal: whether data collection should remain public or be privatized. This issue comes up with vacant-property registration, which I’ve written about before, as many local governments allow registration through MERS, a private company, rather than directly through local data systems. Government officials are then provided access to MERS.

Privatization of data collection and analysis provides many benefits, particularly in that it is cost-effective for local government to take advantage of an already developed platform. The primaru draw back, however, is the risk of industry capture, as with MERS and its association with the mortgage industry.

The best solution, when available, is for local governments to take advantage of open-source programs or nonprofit developers (as available through Code for America). Otherwise, there are companies like Socrata that, as far as I can tell, are not closely associated with any industry other than the cloud-computing and data-collection industry.

Cook County webcast this Friday on new Socrata Data Portal

Here’s an exciting announcement for those of us in Cook County. The County is following the City of Chicago’s lead to create a state-of-the-art data portal at data.cookcountyil.gov. I’m particularly interested in the “courts” data. Here’s a press release for webcast about the portal this Friday (and a couple of other events that are part of Cook County’s “Open Data Week”).

On Friday, April 27th, the County, the State of Illinois, and Socrata, a data archive company, will live stream a webcast on the new regional data portal, MetroDataChicago.org (http://metrochicagodata.org).  During the broadcast, viewers will learn more about how the portal works, how to use data found there, and what are the goals of the County and State going forward.

The County will also release new and updated datasets.

The County is also partnering with global Big Data Week (http://bigdataweek.com) for an international angle to local data.  Big Data is an emerging data science that allows organizations to analyze very large datasets, find patterns, create predictive models, and help understand more about the vast amounts of data generated by the public.  During Big Data Week, Cook County is hosting a webinar showcasing the projects and achievements of local Big Data developers on April 27th, and co-sponsoring a hackathon competition on April 28th.