On June 20, 2018, NCAJ released Tracking Outcomes: A Guide for Civil Legal Aid Providers & Funders, http://ncforaj.org/wp-content/uploads/2018/06/NCAJ-Outcomes-Guide-complete-for-6-20-18.pdf. In Tracking Outcomes, NCAJ gathers in a single document the leading-edge issues and strategies that are important to civil legal aid providers and their funders in using data to track outcomes achieved by legal aid’s clients and by other recipients of legal aid’s services.
In urging the civil legal aid community to track outcomes, the Guide offers observations and findings obtained through dozens of interviews of civil legal aid leaders and individuals in philanthropy. It also reflects insights of the members of an advisory panel of national experts and the additional support of a pro bono team at Morgan Lewis & Bockius, LLP., guided by Morgan partners Joseph Fay & Zane Memeger. The development of the Guide was supported by The New York Community Trust. The Guide and its Appendix (containing steps for implementation of the recommendations, offering examples of outcomes measures already in use in some communities, and collecting sources of information pertinent to tracking outcomes) are downloadable in discrete sections from links, below. The Guide is co-authored by David Udell & Amy Widman, executive director and deputy director of NCAJ.
In its first set of recommendations, the Guide identifies key options for tracking outcomes that civil legal aid programs can pursue today, including: i) using existing outcomes data to deepen understanding of clients’ circumstances, improve service, and educate the public; ii) connecting basic outcomes findings to “big goals” that highlight programs’ mission-related achievements that are important to clients, such as client safety, stability, family integrity; iii) combining outcomes data with other existing data sets and with big data; and, iv) improving communication between providers and funders to increase the value of data collected and reduce the burden of collecting it.
In a next set of recommendations, the Guide discusses solutions to harder challenges in tracking outcomes, including developing approaches for tracking “systemic advocacy”; securing feedback from clients and from other service recipients about outcomes they achieve; generating better court data on litigants’ outcomes; and developing systems to track outcomes for clients who are referred elsewhere for services for their multiple legal needs.
In a third set of recommendations, the Guide underlines the need to preserve the integrity of outcomes data at all times. The Guide recommends that providers adopt a culture that is careful in gathering, organizing and analyzing data. It also encourages providers to take steps to manage the challenge of “causation” – the fact that multiple factors may contribute to winning and losing cases, and that an exclusive reliance on data in any field can mislead audiences or produce skewed incentives. More fundamentally, the Guide encourages providers and funders to pursue multiple research strategies, including randomized controlled trials, as part of the larger effort to use outcomes data while being mindful of the complexities that surround it.
The Guide and its significance are the focus of a June 20, 2018 news story in the ABA Journal by technology and data journalist, Jason Tashea, titled, Focus on client outcomes, says new report on legal aid data. The complete Guide with Appendix is available as a single document, here, with discrete sections viewable and downloadable, as follows:
- The Executive Summary, with the Recommendations, here.
- The Guide, without the Appendix, here.
- The Complete Appendix, without the Guide, here.
- The Recommendations, with Ideas for Implementation (Appendix 1), here.
- The Outcome Measures – Examples (Appendix 2) here.
- The Sources Pertinent to Tracking Outcomes (Appendix 3), here.
- To access all articles and links on this page, visit http://ncforaj.org/research/the-outcomes-project-tracking-outcomes-a-guide-for-civil-legal-aid-providers-funders.
Visit the National Center for Access to Justice at Fordham Law School online.