Visualizing a Positive Future: Impact Evaluation of a Pre-Entry Course for Adult Learners
Reflections on the impact evaluation of a non-accredited pre-entry course for mature learners at the Lifelong Learning Centre, University of Leeds. The session discusses the aims of the course, community engagement approaches, Jumpstart program details, and the use of Qualitative Comparative Analysis (QCA) to analyze the success and outcomes of the initiative.
- Impact Evaluation
- Adult Learners
- Lifelong Learning
- Qualitative Comparative Analysis
- University of Leeds
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
UALL 2023 'Visualising a positive future outcome': reflections on impact evaluation of a non-accredited pre- entry course for mature learners Helen Bowman Dr Saad Mufti Fiona Chapel h.bowman@leeds.ac.uk s.mufti@leeds.ac.uk f.chapel@leeds.ac.uk Lifelong Learning Centre, University of Leeds
Aims of session Setting the context the LLC and Jumpstart Why did we apply QCA to Jumpstart Reflections on our process How did we use QCA How can you use QCA
Lifelong Learning Centre (LLC) Multi-disciplinary school & service supporting mature, part time and foundation learners Work with community partners Support student diversity Community events, impartial, advice and guidance, information sessions, subject tasters, adult learner summer school, Jumpstart. Create opportunities for adults from low participation areas to access HE Pre-entry work over years off campus and on campus
TASO CONTEXTImprove Jumpstart confidence analysis SUCCESSFUL BIDSteered May 2022-May 2023 report launch Sept '23 Useful input & support EXPERIENCE sector PROJECT towards QCA evaluation Work with consultant Partners across Labour intensive TASO funding
LLCs community engagement approach Community development approach Start with people s lived experiences Build confidence as learners Focus on skills and experiences (assets approach) Draw on radical adult education tradition and ideas of really useful knowledge Transformative education Importance of critically analyzing the world
JumpStart: pre-entry taster course for adults Aims: develop learners' confidence in themselves as potential HE students + support learners to make informed choices about their next steps in education Who: carefully targeted at adults in LPNs in Leeds & local areas; all applicants interviewed Approach: adult education approach includes critical thinking and transformative learning practice Impact: approx. 50% apply for University of Leeds Foundation Years + other courses
Why Qualitative Comparative Analysis (QCA) Well-suited to small-N studies, can easily handle categorical variables Aim is to analyze complex interplay of factors rather than examining the isolated impact of individual variables Emphasizes which conditions are required or enough to produce observed outcome Differs from methodologies that primarily focus on estimating the strength or magnitude of relationships between variables ! Limited statistical inference cannot make probabilistic claims Case-oriented places importance on studying and comparing individual cases Realist approach considers the specific conditions and factors present in each case and considers how they interact and combine to produce the observed outcome
Activity Jumpstart is a pre-entry course for adults who may be considering their options in Higher Education (HE). The outcome we were exploring was a 'rise in confidence in applying for HE, by the end of the 8 week course. What to do: What 'attributes'/ factors do you think might contribute to a rise in self-reported confidence? You can draw on your own experience, discussions in your group and/or the literature you are aware of. Try to think of at least 3 we had to aim for 6-7 factors.
Attributes List Attendance at Jumpstart Attending outreach/IAG events Experiences of education Time out of education Experience of Trauma Caring responsibilities Support network
QCA in practice the initial steps 1. Define outcome A relevant indicator Increase in confidence following JS? 2. Case selection Small to medium-N Represent a variation in the outcome of interest Participants enrolled in JS Time out of education Participation in outreach activities 3. Identify causal conditions Key variables that influence the outcome IF {time out of education < 10 years}, THEN 1, ELSE 0 4. Calibrate variables Assign each factor a binary value (present or absent)
A prompt for deeper understanding Reported an increase in self- reported confidence following JumpStart Were not observed to have been experiencing ongoing trauma Or that the trauma was known to be mitigated in the event that it was observed Attended at least one outreach or IAG event Attended over half of the available Jumpstart sessions Did not report an increase in self-reported confidence following JumpStart
Reflections Complexity of identifying which attributes most relevant to our outcome Challenges in getting exploring appropriate attributes/factors, building trust and not being intrusive Research combined with delivery resource and time! Exchange of ideas and information with other Higher Education Participants Building in-house expertise in 'novel' evaluation methods and evaluative frameworks (Theory of Change)
QCA in practice the initial steps 1. What is an outcome that you wish to understand? Within the context of complex contributory factors What Next? 2. What are your cases? Is this well-suited to within-case analysis following QCA outputs? 3. What are the contributory factors? Are these obvious? Any roadblocks in gathering data? 4. How would you calibrate your variables? Is binary enough?
QCA Resources For more on QCA i. Schneider, C. Q., & Wagemann, C. (2012). Set-theoretic methods for the social sciences: A guide to qualitative comparative analysis. Cambridge University Press. ii. Thomann, E., and Maggetti, M. (2020). Designing research with qualitative comparative analysis (QCA): Approaches, challenges, and tools. Sociological Methods & Research, 49(2), 356-386. iii. Oana, I.E., Schneider, C.Q., & Thomann, E. (2021). Qualitative comparative analysis using R: A beginner's guide. Cambridge University Press. iv. Global network of QCA practitioners: https://compasss.org/. R packages for QCA i. Oana, I.E. and Schneider, C.Q. (2018) SetMethods: An Add-on R Package for Advanced QCA.The R Journal 10(1): 507-33.URL https://journal.r-project.org/archive/2018/RJ-2018-031/index.html. ii. Dusa, A. (2019) QCA with R. A Comprehensive Resource. Springer International Publishing. URL https://CRAN.R- project.org/package=QCA.
