The Black Lives Matter movement and national voices like Bryan Stevenson of the Equal Justice Initiative have placed a spotlight on the grim shortcomings of our criminal justice system—calling for it to improve the lives entrusted to it.
Downstream organizations like ours, the Center for Employment Opportunities (CEO), are also looking for ways to improve, as we work to help those released from incarceration to rekindle their lives and livelihoods. This requires a deep commitment to translating our good intentions into verifiable results.
Recently, that commitment has entailed expanding our data collection from purely gathering data about our participants to also gathering data from them. The effect has been striking. We now regularly ask participants for feedback, use their feedback to improve our services, and dignify suggestions by letting those surveyed know how their input has influenced our programs. In the process, we’ve learned of ways to enhance our services to better meet the needs of our participants and share decision making. We’ve also discovered a few tools and tactics for listening that we hope other nonprofits—and particularly our peers working in criminal justice—can learn from.
Center for Employment Opportunities participants meet weekly with a job coach. These appointments provide opportunities for participants to both give and receive feedback. (Photo courtesy of Center for Employment Opportunities)
Building a Culture of Data-Informed Decision Making
Since CEO’s inception as a demonstration project of the Vera Institute of Justice in the 1970s to our current configuration as a national nonprofit operating in 21 cities across eight states, we have consistently sought to link mission to organizational performance. Our staff manifest this commitment by relying on data and reporting out facts as evidence that we are enriching the lives of individuals released from prison.
CEO gathers data by providing employees with tools to collect it, such as our participant database built on the Salesforce platform. We nurture our data-gathering mindset by allocating time in staff meetings to analyze and reflect on the meaning of data, and how we can apply insights to achieve our program goals. For example, CEO’s job development staff document feedback on job interviews from both the client and employer perspective and use it help participants improve their presentation skills.
We also periodically vet our long-term outcomes like reduced recidivism and job retention against outcomes of similar groups of people who have not participated in our programs. This allows us to validate whether positive or negative changes in participant lives are attributable to CEO. Based on several such external evaluations, we’ve found that CEO meaningfully reduces recidivism for people recently released from prison.
Incorporating Data From Participants
Historically, our performance monitoring and program evaluation data has been gathered about people, not from them. Since people are the true experts on their lives, it’s important to treat them that way. We can’t fully know if we are making participants lives better unless participants say we are, an idea Fay Twersky discusses in “Time for a Three-Legged Measurement Stool.”
Our ability to expand our data-gathering to include surveying participants came with help from three sources. First, in 2013 we encountered David Bonbright, CEO at Keystone Accountability, who for a couple of decades has been calling attention to “constituent voice,” as he calls input from program participants. Bonbright transformed our thinking about the purpose of gathering constituent feedback to one of seeking to improve customer service while empowering participants by increasing their influence on programs and policies. He also convinced us that the practice of feedback could be simplified by using short surveys, and optimized by letting survey participants know when we implemented suggestions, encouraging them to give more.
Next, Fund for Shared Insight, a coalition of foundations committed to the use of feedback in the social sector, emerged in 2014 to support the efforts of nonprofits to systematically listen and respond to the people they are trying to help. Shared Insight provided CEO a three-year grant to strengthen our culture of feedback, build the technology infrastructure to continuously collect feedback, and hire a staff member to spearhead its implementation.
CEO participants work on transitional work crews led by CEO site supervisors who provide feedback on a participant's performance each day. (Photo courtesy of Center for Employment Opportunities)
Finally, by 2014, most of our participants owned mobile phones. This gave us a quick and efficient channel of direct communication as they progressed through our program.
Thanks to this constellation of insight, financial resources, and mobile technology, we were able to create a platform called Talk to Us, which draws on four feedback channels—SMS, tablets in waiting rooms, one-on-one coaching, and focus groups. We use these means to collect data from participants on their perceptions of program quality, their engagement with the program, and how we are helping them achieve their goals.
Acting on Feedback
Collecting feedback data systematically requires not only tools, but also staff buy-in and persistence. The harder part, however, is consistently acting on the feedback and regularly sharing what we learn with our participants.
Four Channels of “Talk to Us”
Our Talk to Us platform has four channels for listening to participants:
1. CEO sends SMS surveys to participants at various program touch points, such as following a job interview or after reaching a job retention milestone. We ask questions like: “How often do staff at CEO treat you with respect?” and “How likely are you to recommend CEO to a friend?”
2. We pose the same survey questions on a tablet made available in CEO waiting rooms, our second channel of feedback. This anonymous method of collecting feedback provides both quantitative and qualitative results that we compare to feedback received from SMS surveys to indicate levels of courtesy bias, the tendency to tell someone what you think they want to hear.
3. The third channel integrates with our daily job coaching services. When case managers meet with participants, they end their sessions by asking what CEO should change or what the organization can do better? Our staff document this feedback in our database using the signifier #feedback, which helps us track relevant comments. When possible, the case manager responds to the participant in real time. In all cases, we aggregate comments and synthesize them to determine where we should make system-wide changes.
4. Standard focus groups, which we gather routinely in our offices, are our fourth feedback channel. While focus groups are biased by self-selection, we find they are the best forum to authentically listen to participants talk about their experiences. They also allow us to make sense of the feedback we receive from our other channels.
After our initial Talk to Us rollout, CEO made several adjustments to make it easier for staff to respond to feedback.
First, we created a part-time customer advocate position to monitor incoming feedback and to respond quickly when we received a low rating. The new role provided a neutral voice to engage with participants and a team member to help staff problem solve.
Recently, for example, a participant told us via SMS that he thought CEO should do more to help participants get their driver's licenses. Our customer advocate reached out to learn more, and we began offering driving instruction manuals in the waiting room. This improved offerings for our customers; it also empowered the participant who made the suggestion. He now works as a resident counsellor at a nonprofit housing organization and makes asking for and responding to feedback from residents part of the way he works.
Second, we made it easy for staff to access customer feedback via weekly email reports to support “closing feedback loops,” a process that involves collecting, learning, and acting on customer input. Our feedback data resides in our performance monitoring database, which can be cumbersome to access during normal work flows. But our weekly report, emailed to each staff team in our 21 offices, serves as a rallying point for feedback in staff meetings. Often the feedback is positive and encouraging, but when it’s not, we can intervene quickly. For example, in one report a participant spoke in the third person about her concern that some participants were not mentally fit to perform their CEO duties. When a CEO employee reached out privately, the respondent revealed that she was discussing her own anxiety, and our staff connected her to mental health resources.
Third, we frame the practice of feedback as an ongoing conversation, rather than pure data collection, to put the focus on the relationship and mutual learning, which brings joy to the process. To this end, CEO recently adopted “motivational interviewing,” which trains staff to ask open-ended questions, clarify responses, and encourage goal-setting. This approach has improved our feedback conversations, allowing participants to determine for themselves how they want to change and where they want to arrive.
Dignity and Power Through Feedback
The process of gathering and responding to feedback is more than a tool for program improvement; it is a powerful practice for giving dignity to people struggling to make their lives better in a world still beset with racial bias. For the thousands of men and women of color CEO serves each year, the act of using their voices to create change is an important step in rekindling their belief that they can influence their future. Feedback not only improves, it empowers, helping to create a culture of equity and inclusion that ultimately promotes a more just world.
This article is part of a series that was produced for Stanford Social Innovation Review by Milway Media with the support of the William and Flora Hewlett Foundation.
Support SSIR’s coverage of cross-sector solutions to global challenges.
Help us further the reach of innovative ideas. Donate today.
Read more stories by Chris Watler & Brad Dudding.