National Center for Access to Justice

The Outcomes Guide

On June 20, 2018, NCAJ posted Tracking Outcomes: A Guide for Civil Legal Aid Providers & Funders. The ABA Journal’s technology and data journalist, Jason Tashea, covered its release in, Focus on Client Outcomes, Says New Report on Legal Aid DataTracking Outcomes was subsequently published (in revised form) in The Cardozo Journal of Equal Rights and Social Justice in 2019. 

Tracking Outcomes presents the leading-edge ideas, issues and strategies that are important to civil legal aid providers and their funders in using data to track and report outcomes achieved by legal aid’s clients and other service recipients. Civil legal aid programs are a lifeline to people contending with civil legal disputes, often making a difference that allows people to keep their children, their homes, their savings, their physical and emotional security, and sometimes even their freedom.

Increasingly in our society, data plays a role in illuminating what legal aid programs do, and how it matters to individuals, families, and our society. Outcomes data can deepen understanding of clients’ circumstances, improve outcomes achieved by clients, and inform the public about the value of civil legal aid. Tracking Outcomes offers three sets of recommendations for using data to track outcomes:

  • Changes achievable today:  i) using existing outcomes data to deepen understanding of clients’  circumstances, improve service, and educate the public; ii) connecting basic outcomes findings to “big goals” that highlight programs’ mission-related achievements that are important to clients, such as client safety, stability, family integrity; iii) combining outcomes data with other existing data sets and with big data; and, iv) improving communication between providers and funders to increase the value of data collected and reduce the burden of collecting it.
  • Harder challenges: i) developing approaches for tracking “systemic advocacy;” ii) securing feedback from clients and from other service recipients about outcomes they achieve; iii) generating better court data on litigants’ outcomes; and iv) developing systems to track outcomes for clients who are referred elsewhere for services for their multiple legal needs.
  • Continuing obligations:  i) preserve the integrity of outcomes data at all times; ii) exercise care in gathering, organizing and analyzing data; iii)  manage the challenge of “causation” – the fact that multiple factors may contribute to winning and losing cases, and that an exclusive reliance on data in any field can mislead audiences or produce skewed incentives; and, iv) pursue multiple research strategies, including randomized controlled trials.

Tracking Outcomes is authored by David Udell & Amy Widman, executive director and deputy director, respectively, of NCAJ, with key contributions from Sara Ilyas, NCAJ’s Research Assistant, and Jamie Gamble, NCAJ’s Senior Counsel. Many people made essential contributions to the project, including the following:

  • dozens of civil legal aid leaders, individuals in philanthropy, and individuals in the courts were interviewed for the project;
  • five national experts on data and civil legal aid constituted an advisory panel, consulted on multiple occasions for the project:
    • Christine M. Fecko, General Counsel, IOLA Fund of the State of New York;
    • Bob Gillett, Executive Director, Michigan Advocacy Program;
    • Rachel Perry, Founder and Principal, Strategic Data Analytics;
    • Rebecca L. Sandefur, Department of Sociology, University of Illinois at Urbana-Champaign; Associate Professor, College of Law, University of Illinois at Urbana-Champaign; Senior Research Social Scientist, American Bar Foundation; and
    • Jessica Steinberg, Associate Professor of Clinical Law, George Washington School of Law.
  • a team of attorneys at Morgan Lewis & Bockius, LLP., guided by law firm partners Joseph Fay & Zane Memeger, interviewed experts, helped analyze issues, and drafted sections; the firm also funded the digital design of the Guide.
  • several additional experts provided feedback on drafts of the Guide:  Sandy Ambrozy of the Kresge Foundation; James Greiner of the Harvard A2J Lab; Alex Gulotta of Komenge LLC; Lynn Kelly of the City Bar Justice Center; and Raun Rasmussen, of Legal Services for New York City.
  • The New York Community Trust financed the overall project.

Tracking Outcomes is downloadable in all its parts: