Understanding the Credit Union Difference: Measurement and Reporting
Explore the key concepts of impact measurement, including the definition of impact, methods of measurement, and the importance of effective reporting in showcasing a credit union's contribution to members and communities. Discover how to unpack ideas, map stakeholder issues, formulate theories, and refine strategies for demonstrating impact. Learn how to create compelling reports through written formats and visual aids to engage stakeholders and validate the credit union's effectiveness.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
The credit union difference: how to measure and report it .PPT materials for the Toolkit Funded by:
Social Impact Measurement for Credit Unions Workshop 1 on Design Unify 23 July 2018 Hoot 24 July 2018
Agenda 11.00 - Introduction 11.10 - Session A Unpacking the ideas 11.50 - Session B Mapping stakeholders' issues 12.30 - Session C Formulating the theory 1.10 - Lunch 1.30 - Session D Refining the theory 2.10 - Session E Reviewing the theory 2.50 Conclusion 3.00 Ends
11.00 - Introduction Background to the project Agenda Expectations
11.10 - Session A - Unpacking the ideas in SIM Impact... Measurement... Reporting... ToC thinking... Management... Learning Questions A
Impact... Is a long term, positive, durable & sustainable feature of the financial circumstances of members & their communities to which the credit union contributes through working with them; Flows from the members engagement with the credit union; Varies with the financial circumstances of the members, which reflects differences between affluent , coping and vulnerable members;
Measurement... Is the means through which we collect evidence to demonstrate impact; Makes use of either or both qualitative & quantitative methods; Should extract value from existing before collecting new data; May rely on direct or indirect (proxy) sources; Should be proportionate (costs to benefits); E.g. SROI / HACT s Social Value; BSC's Outcomes Matrix;
Reporting... Typically written reports, making extensive use of graphics to enhance visual appeal; Case studies, stories , dashboards , scorecards , personas , etc; The issue is whether the reporting meets the legitimate interests of the stakeholder; Does it convince them that the credit union is having the impact it claims?
ToC thinking 1) Analyse context about needs, barriers, enablers, etc, 2) examine assumptions about how we may intervene, 3) assess the evidence that what we do brings about the desired change; Articulating / surfacing & linking / connecting the imp elements of contexts, assumptions, evidence; Making explicit how what we deliver (memberships, loans, savings) has effects that lead to positive changes in financial circumstances;
Management... Sustains relationships with stakeholders; Evidences the case for resource support from public sector; Enhances governance through facilitating accountability; Creates basis for persuasive / marketing communications; Contributes to more informed decision-making at operational and strategic levels;
Discussion Questions What do you want social impact measurement to deliver for the credit union? What challenges for the credit union arise from social impact measurement? Who wants to know about the impact of the credit union - to whom should the credit union report?
11.50 - Session B - Mapping stakeholders' issues Why map? Clarifying (any) gaps in current reporting to these stakeholders? Consider what is the impact that the interested and influential stakeholders want the credit union to deliver.
Why map? Who is the audience for the reportage? Map / matrix of stakeholders (s), what aspect of your impact (i) do you want to communicate to them?
The key questions are: What is the issue that you and they want addressed? What are the key factors that influence this issue? Whose behaviour needs to change in ways that address the issue? How ill the credit union enable these people to change their behaviour?
Map / matrix of stakeholders (s), showing what aspect of your impact (i) do you want to communicate to them? s) Influence is high s) s) i) i) i) Influence is moderate a) a) a) i) i) i) Influence is low a) a) a) i) i) i) Stakeholders Influence on, by Interest in, CU Interest is low Interest is moderate Interest is high Esp housing, bodies and local government's interest in impacts flowing from saving & loans
Clarifying (any) gaps in current reporting to these stakeholders? Discuss (any) gaps in the reporting through which the credit union communicates impact to these stakeholders (esp. those in blue cells). What evidence would convince the stakeholder(s) about this aspect of the impact? is the credit union providing this evidence? What needs to be done to fill these gaps? Is it feasible that the credit union would do this? Would this be proportionate to the benefits?
Consider what is the impact that the interested and influential stakeholders want the credit union to deliver. Is it sensible to frame this issue as 'too few people borrow from (or save with) the credit union'? EXERCISE 'But why do too few...'
12.30 - Session C - Formulating the ToC What is ToC? Why is ToC important for impact measurement? Assumptions & Propositions Pathways & Hypotheses What should they look like? Understanding the theory / logic underlying the credit union's impact
What is ToC? an approach a method a diagram a definition claims for ToC
What is the ToC approach? Is about understanding the dynamic in which the credit union intervenes (to secure the impact) Financial wellbeing, and how it is fostered, is complex rather than linear; We should have a set of ideas to understand the process, i.e. a theory; With the theory we will be better able to learn about what does or not work and communicate this to interested parties;
ToC, a definition A set of hypotheses and critical assumptions that make up a causal pathway of change which is the basis of the ... design [of the intervention]. Hypotheses are if-then statements between different levels of the change pathway (Drinkwater; 2009)
Claims for ToC Integrate planning, implementation, and evaluation. Prevent mismatches between activities and effects. Enhance accountability by keeping stakeholders focused on outcomes. Help set priorities for allocating resources. Reveal data needs and provide a framework for interpreting results. Enhance learning by integrating research findings and practice wisdom. Define a shared language and shared vision for community change. (Ordonez; 2014)
Why is ToC important for impact measurement? Facilitates analysis of assumptions about context and effect of the intervention Helps show 'reality of the causal pathways Explores propositions about 'if A, then B About the logic / why of pathways Does the intervention make sense?
Hypotheses about change as it occurs in different domains Domain A: changing people's behaviours? COM-B - Capability, Opportunity, Motivation & Behaviour Domain B: changing agency practices? Advocacy in media about change as it occurs in different domains
What they look like? 'left to right' v 'top to bottom (too static?) 'box & arrow (too messy?) mechanistic / linear v holistic / complex A boxy example EXERCISE: What additional elements would align this to good practice (as above, what aspect of context is missing)?
Activities Outputs Outcomes Impacts Stakeholder inputs & questionsStaff members growing? Actions to increase members in key demographic: low to middle incomes, in secure work, with family commitments Increase in level of CU borrowing Decrease in use of high cost credit Better value purchasing across expenditure basket but esp. of e.g. shock buffering products e.g. insurance just about managing managing CU has growing membership of managing borrowers Type 1 impact members are better able to sustain their financial resilience; Management Board City Council Housing Association Increase in number of borrower members in demographic Type 2 impact CU is more resilient and able to serve members in target demographic Are the products reaching target demographic? Does the marketing work? Is the number of managing Loan marketing through social media of targeted packages ( back to school , Christmas, holidays, etc)
Basics of how to do ToC? Forward mapping: inputs activities outputs outcomes impact; Backward mapping from desired impact to inputs is often better as it challenges us; If we have the [specified] impact, then we need... To have delivered these outcomes, we need... To have produced these outputs, we need... To have carried out activities, we need... To have obtained these inputs for resources.
ToC EXERCISE Using the suggested backward mapping & starting from the impact the credit union intends to have, outline / sketch the elements and how they are linked through the causal process Impact, what is it, who benefits from it, at what level does it occur (community v individual) what is the problem to which it is a solution ?;
ToC Exercise Continued What are the outcomes from the work that contribute to the impact esp. moving to low value / high income loans? How do the outputs, (savings, loans, others) feed into these outcomes? In what ways do the credit union s activities produce the outputs? What inputs does the credit union require as resources fuel the process?
ToC QUESTIONS Are the links from inputs to impacts clear? Would the way you present the links help convince a sceptical person? Does the ToC 'add up' as a way of addressing the issues you want the credit union to address? Are there assumptions in the theory that may not be true? What are these? Should the ToC change so that it wont matter? How would you summarise the basic hypothesis(es) about credit / wellbeing that underpins the ToC? (Try to formulate this as a testable idea).
1.30 - Session D - Refining the ToC, Recap... Inputs resource activities that produce outputs that generate results that deliver outcomes that create impact. If you were sceptical what would allay any concerns you might have about the claims made about the process? EXERCISE Taking each of the questions in turn, write brief notes on how you would convince a sceptical observer.
How would we answer the sceptic? Is the process in the ToC consistent with different explanations about how and why the actions influence the effects? Is the impact identified plausibly linked to other explanations (apart from that of the credit union's intervention)? Are the claims made in the ToC open to communicate in plain English what is the theory about the links in the chain of causality, e.g., If we do x action, then y change will follow because...? Are there particular / key / critical points in time (between, e.g., loan made' and 'enhanced financial resilience'), which we would like to know about in order to trace the change in resilience back to the borrowing profile?
2.10 - Session E - Review of ToC Does the theory make sense as a response to the financial circumstances of people's lives in the area (the context)? Would it make sense to a) a senior officer in local government and b) a member of the credit union?
Summary Review Questions Is the theory plausible as a statement of how local people live / stakeholders operate? Is if feasible for the credit union to implement it? Is it sensible to think that we could test the theory?
Review Questions contd Does the impact that the theory identifies offer a reasonable prospect of addressing the 'too few / too many' issue you identified in the stakeholder mapping exercise? Does the diagram make clear the causal process? Are the assumptions made explicit? What is the quality of the evidence for the assumptions? EXERCISE In the light of your answers to these questions, how would you wish to have the theory further refined?
Does the diagram make clear the causal process? Is there enough detail that links the 'if... then' logic? Are there any missing links? Is it clear how the different elements relate to each other (activities >>> results >>> outcomes)? Does it communicate enough of the specific character of the credit union?
Are the assumptions made explicit... about the causal process about the actions of the credit union (to activate this process) about the enablers and barriers that arise from the context?
What is the quality of the evidence? Is there evidence noted for the major hypotheses (about behaviour change?) that underpin the theory? Is there some assessment of the strength of this evidence? Is the assessment plausible?
EXERCISE In the light of your answers to these questions, how would you wish to have the theory further refined?
2.50 - Conclusion Participants to say if, and extent to which, workshop fulfilled their expectations and comment on the nature of any gaps in these expectations left unfilled. Expectations revisited did it deliver? Evaluation what worked, what didn t ?
Next Steps Small Change to write-up report on workshop for the credit union; Small Change to prepare for Workshop 2 on implementation in light of this; Ends [3.00]
Social Impact Measurement for Credit Unions Workshop 2 on Data Unify 31st July 2018 Hoot 1st August 2018 42
Agenda 11.00 - Session A - Overview of workshop 1 & progress 11.30 - Session B - Practical measurement options 12.00 - Session C Quantification & wellbeing 12.30 - Session D Monetisation & partners savings 1.00 - Lunch 1.20 - Session E - Qualitative commentary on impact 1.50 - Session F - Questions for discussion on 3 options 2.20 - Session G - Data system given preferred option 2.50 - Conclusion, evaluation & next steps 43
11.00 - Session A - Overview of workshop 1 & progress to date Unify logic model matrix form of ToC Hoot systems mapping Levels / domains form of ToC 44
Questions for discussion on progress If the CU reports on these impacts to local government / housing & local employers will its credibility rise? What factors will enhance the credibility of the reportage? What factors could detract from its credibility? What else would need to be done? For these stakeholders, what are the hallmarks of a quality report? 45
Outlining the practical measurement options that demonstrates impact assurance of independence / validity / credibility; Option 2) out-source to external consultant or University through securing grant-aid; Discuss advantages & disadvantages of the two options; Option 1) in-house requiring staff time and quality 46
Quantitative measurement of contribution to members' wellbeing (HACT) What is HACT s wellbeing valuation approach? The value of a change in satisfaction with life that correlates with (predicts) an outcome (e.g. reduced stress over unmanageable debt, feeling more in control of own finances ) expressed as the {average} amount of additional money a person would require for s/he to report an equivalent change in satisfaction; 47
How does HACT value wellbeing? Use quality survey data e.g. Understanding Society Identify outcome variables, e.g. health status, fin inclusion Construct statistical model that predicts outcomes Controls for other factors e.g. working, education Model amount of additional money that predicts this Make allowance for what would have happened anyway Do surveys of CU s members before loan & after loan Use comparable questions, note diffs before & after Multiply number changed by HACT s estimates 48
HACTs Financial Inclusion Outcome Values Outcome Debt-free Afford to keep house well-decorated Able to save regularly Relief from heavily burdened with debt 13,376.7 Able to pay for housing Financial comfort Access to internet Able to insure home contents Value 2,548.06 6,272.09 1,293.02 8,974.36 8,878.57 1,663.12 5,843.76 49
HACTs Health Outcome Values Outcome High confidence (adult) Relief from dep n/anxiety (adult) Good overall health Relief from drug/alcohol problems 15,674.32 Smoking cessation Feel in control of life Can rely on family Value 12,549.31 39,301.75 20,186.34 3,948.13 15,733.72 7,036.07 50
