Image credit: Adobe Stock
Overseeing Fund for Shared Insight’s feedback research portfolio is a privilege and a thrill. A privilege because the work of our research grantees examines one of Shared Insight’s bedrock beliefs: If philanthropy and the social sector integrate high-quality feedback into their work it will lead to better outcomes for the people and communities they seek to serve. And now it is also a thrill because most of the research results are in. As our first blog post of the new year noted, we now have evidence that:
- Gathering and responding to high-quality perceptual feedback correlates with better outcomes for individuals; and
- The act of implementing high-quality perceptual feedback loops in and of itself can lead to better outcomes.
These findings have been rolling in in recent months from six organizations that received research grants from Fund for Shared Insight: Boys & Girls Clubs of the Peninsula (BGCP), Center for Employment Opportunities (CEO), Nurse-Family Partnership (NFP), Pace Center for Girls, REDF, and YouthTruth. In each case, the organizations teamed up with research consultants to explore whether there is a relationship between feedback and outcomes for individuals served by direct-service organizations.
While the grantee teams designed research projects with different methodologies given their varied programs, and despite various setbacks that came with the COVID-19 pandemic, each research project delivered evidence that establishes gathering high-quality perceptual feedback as a best practice. The teams discuss their findings in separate posts at Insights for Change. Here are some highlights:
- Results from BGCP (partnering with Hamai Consulting) show that student perceptions as measured on student surveys are predictive of academic performance, even if not always in expected ways.
- More positive student feedback predicts higher academic performance (as measured by state standardized math and English scores) for middle and high school students. It predicts lower academic performance for elementary school students.
- Student feedback also indirectly contributes to student outcomes through its relationship with program attendance. More positive student feedback predicts higher program attendance, which then predicts academic performance and school attendance, for elementary and middle school students.
- Findings from CEO (partnering with MDRC) suggest that the act of providing feedback in and of itself is associated with positive outcomes.
- Participants in CEO’s employment program who responded to texts sent by CEO asking for their feedback on the program were 5 to 10 percentage points more likely than those who did not respond to be employed 90 or 180 days after initial employment. This result was true regardless of the content of the feedback or whether it was positive or negative in nature.
- Further study of the text-message feedback program demonstrated the importance of engaging participants in different ways beyond texted surveys and at different stages of the feedback process in capturing potentially missing voices.
- At NFP (partnering with OMNI Institute) the feedback-to-outcomes analyses are still in progress. In the meantime, the organization has reported early findings that indicate differences in feedback by the race/ethnicity of the new mothers answering surveys.
- African American clients are more likely to say they experience discrimination.
- Clients who are Asian are least likely to feel they can call on their nurse when they need to talk to someone.
- YouthTruth (partnering with SRI Education) found that student experiences, as measured by how they rated six factors, including academic challenge, culture, and relationships, are variously predictive of ninth-grade retention, reading proficiency, math proficiency, absenteeism, and suspension rates.
- Each of the six student experiences rated on the scale demonstrated evidence of concurrent and predictive validity with at least one of the five outcomes at one or more of the school levels (elementary, middle school, high school).
- Student perceptions of school culture (which center on themes of fairness and respect) were associated with every outcome measured, and student perceptions of belonging and peer collaboration were associated with four out of the five outcomes.
- Pace (partnering with Covian Consulting and Arc of Evidence) concluded that opportunities for girls participating in Pace programs to share their feedback contribute not only to their positive program outcomes, but also to their empowerment.
- Program engagement is predictive of self-efficacy (the ability for girls to believe they can be successful with the skills they possess), and this is stronger for girls who report a positive program experience.
- Providing feedback contributes to program participants’ progression in how they use their voices and speak up in leadership roles.
- The act of providing feedback in and of itself is associated with earning a higher grade point average and more school credits, even after controlling for race, ethnicity, age, days at Pace, baseline reading achievement, and attendance.
- REDF (partnering with RTI International) found that the feedback from participants of four employment social enterprises (ESEs) that provide training and paying jobs was predictive of outcomes.
- Reporting feeling connected to staff and being fearful about not being able to succeed at a job outside of the ESE was predictive of positive program exits.
- Participants who reported feeling that the ESE staff treated them with respect were more likely to remain employed 18 months after intake.
- General satisfaction with the ESE was associated with working more hours per week, and interacting frequently with ESE staff was associated with higher wages.
Taken together, these research results help establish the case for an underlying thesis that animates Shared Insight and so many of our partners and fellow travelers in the feedback field. We believe that gathering high-quality feedback and practicing high-quality feedback loops are each “the right, smart, and feasible thing to do” (a framing perhaps first enunciated by Feedback Labs). And while the case has already been made for feedback as the right thing to do (people and communities have an inherent right to determine their own paths forward) and the feasible thing to do (our own Listen4Good and other models, tools, and resources prove its straight-forward and sustainable), it is indeed a thrill to introduce a new evidence base for feedback as the smart thing to do.
Penny Huang, feedback research portfolio manager, Fund for Shared Insight