On June 20, 2018, NCAJ released Tracking Outcomes: A Guide for Civil Legal Aid Providers & Funders. In Tracking Outcomes, NCAJ gathers in a single publication the leading-edge issues and strategies that are important to civil legal aid providers and their funders in using data to track outcomes achieved by legal aid’s clients and by other recipients of legal aid’s services.
In urging the civil legal aid community to track outcomes, the Guide offers observations and findings obtained through dozens of interviews of civil legal aid leaders and individuals in philanthropy. It also reflects insights of members of an advisory panel of national experts and additional support of a pro bono team from Morgan Lewis & Bockius, LLP, guided by Morgan partners Joseph Fay and Zane Memeger. The Guide and its Appendix (containing ideas for implementation of the recommendations, offering examples of outcomes measures already in use in some communities, and collecting sources of information pertinent to tracking outcomes) are downloadable in discrete sections from links, below. The development of the Guide was supported by The New York Community Trust.
In its first set of recommendations, the Guide identifies key options for tracking outcomes that civil legal aid programs can pursue today, including: i) using existing outcomes data to deepen understanding of clients’ circumstances, improve service, and educate the public; ii) connecting basic outcomes findings to “big goals” that highlight programs’ mission-related achievements that are important to clients, such as client safety, stability, family integrity; iii) combining outcomes data with other existing data sets and with big data; and, iv) improving communication between providers and funders to increase the value of data collected and reduce the burden of collecting it.
In a next set of recommendations, the Guide discusses solutions to harder challenges in tracking outcomes, including developing approaches for tracking “systemic advocacy”; securing feedback from clients and from other service recipients about outcomes they achieve; generating better court data on litigants’ outcomes; and developing systems to track outcomes for clients who are referred elsewhere for services for their multiple legal needs.
In a third set of recommendations, the Guide underlines the need to preserve the integrity of outcomes data at all times. The Guide recommends that providers adopt a culture that is careful in gathering, organizing and analyzing data. It also encourages providers to take steps to manage the challenge of “causation” – the fact that multiple factors may contribute to winning and losing cases, and that an exclusive reliance on data in any field can mislead audiences or produce skewed incentives. More fundamentally, the Guide encourages providers and funders to pursue multiple research strategies, including randomized controlled trials, as part of the larger effort to use outcomes data while being mindful of the complexities that surround it.
The Guide and its significance are the focus of a June 20, 2018 news story in the ABA Journal by technology and data journalist, Jason Tashea, titled, Focus on client outcomes, says new report on legal aid data. NCAJ’s Executive Director, David Udell, explains in the Journal, that “Among the main lessons offered is that when outcomes data is considered it should be in transparent consideration alongside multiple factors that together can promote a virtuous cycle of increasing understanding, improving service, explaining the work to others, and refining the use of measurement.” NCAJ’s Deputy Director, Amy Widman, co-author of the Guide, explains, “We are hopeful that the guide will help legal aid providers and funders to increase their use of outcomes data in turn making a difference for the better in the lives of legal aid clients and communities.”
The complete Guide with Appendix is available as a single document, here, with discrete sections viewable and downloadable, as follows:
- The Executive Summary, with the Recommendations, here.
- The Guide, without the Appendix, here.
- The Complete Appendix, without the Guide, here.
- The Recommendations, with Ideas for Implementation (Appendix 1), here.
- The Outcome Measures – Examples (Appendix 2) here.
- The Sources Pertinent to Tracking Outcomes (Appendix 3), here.
- The Guide, on the NCAJ website, here.
The National Center for Access to Justice at Fordham Law School