In 2012, the Bloomberg administration passed Local Law 11, which went on to spawn the NYC Open Data program. Having decided that publishing city agency data increased transparency and fostered interdepartmental communication, the lawmakers’ decision to pursue an open data platform has been framed as a program that “will make the operation of city government more transparent, effective and accountable to the public.” The law intended to accomplish two goals: making city data available online using open access standards to increase transparency and accountability to the public while improving intra-agency and inter-agency communication. This intention of the law has accomplished the former of the two goals by creating an environment of transparency and improved public accountability, but it is still unclear whether the project has improved information sharing within and between different agencies. In light of this potential shortcoming of the implementation of Local Law 11, this project utilized network analysis to answer a different question: can operational relationships between city agencies, in fact, be identified based upon the types of datasets that they are posting, and can those relationships identify groups of agencies that should establish or strengthen inter-operational connections with each other based on data overlap?

As impressive as it is, the NYC Open Data repository highlights some of the problems that we see in big data initiatives. For example, urban planning has to date not incorporated health care data to fully exploit the data’s potential to inform public health policies. As shown in the figure above, health data are directly shared only between the Health and Hospitals Corporation, the Department of City Planning, and the Department of Health and Mental Hygiene. Noticeably missing are direct data-sharing connections between these organizations and the New York Department of Sanitation, Department of Homeless Services, Department for the Aging, and the Office of Emergency Management. The association map illustrates that organizations that would benefit from direct multidisciplinary collaboration appear to be operating in informational siloes, which is a recurring theme among big data projects.