Insights on Land Evaluations: Lessons and Challenges in Design
Exploring emerging lessons and gaps in land evaluations, this content delves into the impacts of land tenure interventions on agriculture, economic outcomes, and environmental sustainability. The discussion highlights key findings, such as the positive effects of formalization on agricultural land but the tradeoffs between economic gains and environmental concerns. It also sheds light on sampling challenges and the need for a deeper understanding of contextual factors for effective evaluations.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
The Future of Land Evaluations (Part 1): Emerging Lessons in Design Jenny Lisher- World Bank, Global Practice Urban, Resilience, and Land Heather Huntington- DevLab, University Of Pennsylvania November 29, 2022
Land Evidence and Gaps 3IE/CAMPBELL (LAWRY ET AL-2014), MCC (LISHER-2016),IFAD (HIGGINS- 2017), TSENG ET AL (2021) Systematic Review Findings Title affects ag productivity and consumption/income in Asia/Latin America but weaker evidence in Africa; potentially due to strong customary tenure or smaller farms/lower incomes (Lawry) Positive effect on productive investments and soil conservation while mixed evidence on productivity (IFAD/MCC-perhaps due to lack of longer-term research or poor ability to capture productivity) Gaps Early evaluations focused largely on titling and household head survey little use of other data sources or evaluating other key initiatives (institutional/policy reform, NRM) Need further consideration of contextual factors, type of intervention, implementation fidelity, and exposure periods (short exposures) Lack of review of interim outcomes and linkages- does the theory of change hold? Little assessment on evaluation designs and their effectiveness, including methodologies, data sources used, and sampling-much cross sectional and not impact estimates Mixed evidence on access to credit Land Tenure led to largely positive effects but tradeoffs between economic and envt; formalization had relatively positive economic effects on agricultural land but weaker support over environmental outcomes due to shorter-term (Tseng) Rural focus which misses urban/peri-urban dynamics
Behind the Scenes Sampling challenges Incorrect understanding at first of who benefitting->weak sampling frames Lack of overlap btw intervention areas and baseline survey sample or issues matching spatial data to overlap/match Studies underpowered to examine subgroup effects (women) expansion into women s modules Who understands the data? Shift to parcel managers to answer field or parcel specific questions Data quality concerns with measuring - perceptions of tenure, conflict, land characteristics (e.g. land size), land transfers, mortgages, tenure status and agricultural productivity
A Spin on the Systematic Review Our Motivation Update the knowledge base Now evaluations go beyond titling/credit Recent release of RCTs and multiple quasi- experimental designs using multiple data sources and mixed methods Later evaluations have longer exposure periods and have women sampled/data available Understand the effectiveness of both land evaluations and land interventions (not just results but why) Examine impacts across the Global Land Logic Model Analyze potential methodological challenges (exposure period, sampling) Interrogate the contextual factors (location, land governance structure) Assess the programmatic factors (type of interventions; implementation fidelity)
Selection of Studies(2021) Selection criteria: Evaluate the impact of a land- related intervention or reform Employed experimental or quasi-experimental design Strong/credible counterfactual Only impact estimates included in charting the empirical results 25 experimental and quasi-experimental land tenure and governance impact evaluations (interim and endline studies) 8 provide 5+year exposure periods (7 more at 5 years) 5 RCTs
Categorization Research design Length of exposure period Interventions Systematic registration-17 (4 ag) Policy and institutional strengthening to catalyze sporadic registration-4 (1 ag) Mapping only/community/ NRM (1 ag) Noted when combined with agriculture/irrigation support part of larger agricultural project -6 Location (rural-19 or urban/per- urban-5, 1 national) Land governance environment (customary, statutory) Implementation fidelity and sustainability
Global Land Logic Model Lisher 2019
Findings: Shorter-term Outcomes Perception of Tenure: Move to medium-term outcome Relatively positive: 12 studies; 7 pos/5 null More likely to have an effect in riskier areas or settlements with land use change; rural may indeed be more secure/customary tenure (urban where all positive vs rural was mixed) More likely to find positive results over longer exposure period Individual titling or in coordination with ag, positive results 2/3 of time while legal change/support alone has not moved perception MORE LIKELY MEDIUM-TERM OUTCOME Credit: Title is a required but not sole factor to get credit-Move to Medium-term outcome Relatively negative: Only 4/12 Positive all individual systematic regularization/titling, over the long-term Why no effect? Evaluation: Exposure periods short Intervention: paralegal intervention, informal documents not recognized, community level which not recognized for loans External Constraints: high transfer fees, didn t allow transfers, informal settlements/bankability (2 studies) or one case where already high lending volumes TITLE IS REQUIRED BUT NOT SOLE FACTOR; MEDIUM-TERM FOR NEWLY TITLED VS LEGAL/INSTITUTIONAL CONSTRAINT MAY BE SHORTER-TERM
Findings: Medium-term Outcomes Investment: Overall positive Majority positive/most studied variable: 19 studies (11 positive; 2 mixed; 4 null; 1 neg Urban/peri-urban areas largely positive; rural more mixed Similar to perception, higher change in riskier areas/settlements but not always linked to perception -mixed findings Often policy/legal/regulatory change prior Why no or negative results? Evaluation: Poor comparison areas Intervention: Second vs first certification, dispute resolution without land rights provision, informal rights that not allowed to transfer Conflict: Hard to measure-all null All (9) null for actual conflict though a few found changes in perceived concerns over conflict Why no results? Environment/Evaluation: rare/no power
Findings Longer-term Outcomes Productivity-May need complementary investments (Ag/Envt) Mixed results (10 Studies: 4 Positive; 1 negative; 5 null) Positive effects only if complementary irrigation ag/irrigation interventions with long-term exposures (Senegal, Mongolia, Burkina and Malawi). Similar to forest areas, may need land plus complementary investment. Why no results? Logic/Environment: Peri-urban move off farm; increase investments may not lead to productivity Implementation fidelity: insufficient titles issued to show effects Evaluation: Short exposure periods Overall, no regional differences or based around weak/strong tenure security but rather whether complementary agricultural investment involved and implementation/exposure periods Environmental Outcomes: Relatively positive Positive (7/9) Inclusive of a Benin RCT around demarcation& village land planning of agricultural land leading to decreased deforestation Income: Weak and mixed results Relatively little results (3/9) with 2 mixed 2 with farm income and with agriculture (pastoral/irrigation), 1 with off-farm income (Burkina ADP with ag; Ghana- off farm; Mongolia 2 PURLs with herder training/wells); Why no or negative Logic: Off farm income trade offs Evaluation: short exposure periods Intervention: Increased investments but no other effects that might lead to income
Relatively positive effects for perception of tenure, labor and investment while largely null or mixed effects around credit, conflicts, productivity, and income Theory of change not always followed: Overall Results/Theory of Change Sometimes tenure security leads to investment to productivity to income Other times investment increases without earlier changes in tenure security (perhaps at times to obtain security like tree planting, cultivation or construction to prove occupying/improvements) Trade offs with productivity/farm income up but off farm down and land investment decrease and production decreased as move off-farm and get more income (peri- urban)
Capturing land impacts constrained due to: Incomplete understanding during project design of drivers/constraints Implementation fidelity and related sustainability Weaknesses in evaluations design At times perception of tenure not weak or driver of insecurity not addressed Methodologies weak at times Lack of land documentation demanded (especially in customary if not free) /processed (capacity) Over reliance historically on household surveys- insufficient sample to tell effects on mortgages/conflicts/markets Need to better understand drivers/restrictions of land markets and mortgages beyond tenure Lack of office/system/equipment sustainability to process demanded titles after project leaves(Need financial/political buy-in) Lack of analysis on policy/institutional reforms Do farmers want to stay on the farm? Mismatched exposures Would the effect be significant if project carried out to plan/outputs sustained? Evaluators don t always understand differences in beneficiaries and varying timeline of effects or versed in complexities of tenure not scoped to capture differential and complex effects Change in location, timeline or activity so varying results or evaluations no longer matched well Few measure (or not reporting) indicators across TOC Need better way of capturing productivity/ conflict/land use change geospatial/admin data
Implications for Evaluation Design Ensure implementation fidelity/outputs, timeline, and sustainability ---not just interim/longer-term outcomes. Track all factors, including interim outcomes that lead to impacts Consider differences in urban, peri-urban, rural environment, especially when looking at ag productivity /income-capture key potential tradeoffs and across multiple parcels within the hhld Limited number of questions can ask so key to consider what most want to track (trade offs on labor/off-farm, tracking parcels/households especially when promoting privatization/transfers) Move away from cross-sectional waves (panel data better)/endogeneity issues Report out on null findings not just positive/significant findings especially if part of theory of change Sample appropriately to track differences in beneficiaries; women act differently (wives surveys norm rather than exception) Utilize data outside of household surveys to capture results: Research questions require different data sources Certain variables better to track using geospatial and administrative data which can provide higher levels of accuracy at less cost and higher frequency when planned appropriately (land allocation, and sizes, conflict, investments , land use change, crop cover, sporadic demand/markets, transaction time, mortgages, tenure status) or qualitative data to understand context triangulating data best using mixed methods Track perception of tenure security over longer-term as well as access to credit-it may take years for certain populations with deep rooted tenure insecurity (or used to customary practices) to change perception and utilize formal systems-if large tenure insecurity issues, effects more significant/ quickly Questions remain on how best to capture perceptions of tenure, conflict, land characteristics (e.g., landvsize), tenure status and agricultural productivity. I Insufficient power risks especially risky in land as right not always approved by government, or smaller uptake by population when dealing with sporadic titling or payment/pick up of land certificate Project implementers and evaluators should closely coordinate on any changes in control/treatment groups, as well as share geospatial data and workplans to aid with ensuring valid treatment evaluation sample overlaps with project treatment.
