Preliminary Findings & Lessons on Local and Regional Food Aid Procurement
This document presents the preliminary findings and lessons learned from local and regional procurement of food aid programs conducted by various organizations in different countries. The focus is on establishing a framework for monitoring and evaluating the performance of food aid procurement projects to improve effectiveness and impact assessment. The report discusses data collection, analysis tools, and impacts of the programs in Bangladesh, Burkina Faso, Guatemala, Kyrgyzstan, Niger, and Zambia.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Local and Regional Procurement of Food Aid: Preliminary Findings and Lessons Learned from 2010-11 US Programs Christopher B. Barrett and Miguel I. G mez, Cornell University LRP Learning Alliance USDA Local And Regional Food Aid Procurement Pilot Project Lessons Learned Workshop, Nairobi, Kenya, September 19-22, 2011
Introduction The LRP Learning Alliance A group of 4 PVOs working together and with Cornell University to establish a common, rigorous framework for M&E of local and regional procurement (LRP) of food aid under the USDA LRP pilot program and the USAID Emergency Food Security Program. Materials on Learning Alliance web site at https://sites.google.com/site/lrplearningalliance/home. 2
The Global Framework Motivation for Global Framework Motivation for Framework Integrate reporting requirements of USDA and USAID Gather data needed to generate rigorous evaluation of LRP performance along multiple dimensions: timeliness, cost effectiveness, price/price volatility impacts, recipient satisfaction, smallholder supplier impacts Enable direct comparison across LRP project modalities and regions and with other forms of food aid (e.g., traditional transoceanic food aid) to inform policy deliberations Common database to manage data across projects Foster improved PVO market monitoring and analysis
The Global Framework Data Collection and Analysis Tools Constructed eight forms to collect data systematically Data collected to analyze evaluation topics USDA required - Historic supply, demand and price movements; do no harm; reasonable market rate; timeliness; product quality and safety; cost; government interference Additional topics - Producer price stimulus; supplier behavioral change; volumes; and food production shocks Trained PVO personnel on price data collection methods and basic price analysis techniques. Materials available on Learning Alliance web site. Can help establish when/where/whether LRP makes sense and what to monitor and key impacts on which to focus. 4
Impacts: Preliminary Findings Preliminary Findings on LRP Impacts To date, we have sufficient data to do analyses on some dimensions for six different programs: 1) Bangladesh (Land O Lakes USDA LRPPP cereal bars from chickpeas, peanuts, puffed rice, sesame seeds, etc.) 2) Burkina Faso (CRS USDA LRPPP cowpeas,millet,veg oil) 3) Guatemala (CRS USDA LRPPP beans, CSB, white maize) 4) Kyrgyzstan (Mercy Corps USAID EFSP cash transfer) 5) Niger (Mercy Corps USDA LRPPP cowpeas, maize, millet and vouchers for salt and veg oil) 6) Zambia (Land O Lakes USDA LRPPP beans, CSB, maize meal, veg oil) 5 4 more to come: CRS Mali and Niger, WV Kenya and Uganda
Impacts: Preliminary Findings Timeliness of Deliveries Method Compare event histories of LRP and transoceanic (USAID or USDA) deliveries to the same country up to 6 months before or after an LRP purchase. Compare the time it takes from initiating procurement (IFB, tender release, etc.) until delivery to terminal warehouses. 6
Impacts: Preliminary Findings Timeliness of Deliveries Difference between shipments from US and LRP (date of invitation until final warehouse delivery) LRP Bangladesh (n=24/12) 14.8 Transoceanic 20.4 Mean difference = 14.3 weeks (59%) LRP Guatemala (n=9/8) 12.6 Transoceanic 19.6 LRP 14.0 Burkina Faso (n=3/5) Transoceanic 35.6 LRP 6.7 Kyrgyzstan (n=1/7) Zambia Transoceanic 19.9 LRP 6.6 Transoceanic 27.9 (n=4/2) Niger (n=12/19) LRP 5.0 Transoceanic 22.2 0 10 20 30 40 Weeks Huge, statistically significant gains in timeliness. (And gap will increase as more data come in.) 7
Impacts: Preliminary Findings Delivered Commodity Cost Method Same comparison group as timeliness: LRP and transoceanic (USAID or USDA) deliveries to the same country up to 6 months before or after an LRP purchase. Now we match by commodity. Compare the cost of commodity, ocean freight and ITSH of LRP and transoceanic USAID or USDA shipments. 8
Impacts: Preliminary Findings Delivered Commodity Cost Total Costs (Commodity+Ocean Freight+ITSH) Total Savings Country Bangladesh Burkina Faso Commodity Chickpea/Split Pea Veg. Oil Millet/Bulgur Cowpea/Lentil Beans Incaprina/CSB Cash*/Wheat Flour Millet/Bulgur Millet/Sorghum Beans LRP $702 $2,065 $335 $566 $1,085 $1,919 $470 $392 $392 $1,150 Transoceanic LRP $1,066 $1,920 34.1% -7.6% 60.1% 47.5% 0.8% -115.5% 56.5% 53.8% 45.2% 0.8% $838 $1,078 $1,094 $891 $1,081 $849 $716 $1,159 Guatemala Kyrgyzstan Niger Zambia *cash converted to wheat at prevailing market price in distribution zones For processed products and beans, often little or no cost savings from LRP. But for cereals and some pulses very large (and stat. sig. savings). Simple average savings = 20.9% over comparable commodities shipped from US to same (or neighboring) country during same half year. 53.9% for cereals! 9
Impacts: Preliminary Findings Recipient Satisfaction In Burkina Faso, Guatemala and Zambia, in addition to the LRP program, there existed a MYAP region delivering similar products during the same period. We ran household surveys to assess recipients satisfaction with food aid commodities received along various dimensions and costs of meal preparation. Rated preferences on specific attributes of the commodities they received on a scale of 1 (low) to 5 (high) Stated preparation needs from 1 (much more) to 5 (much less) Comparing LRP recipients vs. MYAP recipients gives us insights on preferences and perceptions of recipients, relative to transoceanic commodities (e.g., MYAP). 10
Impacts: Preliminary Findings Recipient Satisfaction Sample multivariate ordered logit regression results Guatemala Maize/Rice Burkina Faso Grain (millet / bulgur wheat) N/A N/A LEAP LEAP LEAP* LEAP** MYAP** LEAP*** MYAP*** Zambia Beans Incaparina/ CSB Legume (cowpeas / lentils) N/A N/A LEAP** LEAP*** LEAP*** LEAP*** LEAP LEAP MYAP*** ZLRP/CFAARM rations (vs. market options) Quality Quantity Ration Size Taste Texture Appearance Cleanliness Storability Nutritional content General Satisfaction ***, **, * Statistically significant at 1%,5% and 10% levels, respectively Note: Zambia method measures relative to equiv. commodity available in local markets. Others use absolute measures. LRP** LRP LRP** MYAP LRP*** LRP* ZLRP** N/A N/A N/A N/A N/A LRP*** LRP** LRP** N/A LRP*** LRP** LRP LRP*** LRP*** LRP** LRP* LRP* ZLRP** ZLRP*** ZLRP*** ZLRP* MYAP ZLRP** N/A N/A LRP*** MYAP* LRP*** LRP LRP** LRP LRP LEAP LEAP*** ZLRP** 11
Impacts: Preliminary Findings Recipient Satisfaction Sample multivariate ordered logit regression results Guatemala Burkina Faso Zambia ZLRP/CFAARM rations (vs. market options) MYAP* MYAP MYAP MYAP* MYAP*** ZLRP*** Beans Maize/ Rice Incaparina /CSB Grain (millet / bulgur wheat) MYAP*** MYAP*** MYAP* MYAP*** MYAP*** LEAP* Legume (cowpeas / lentils) MYAP*** MYAP*** MYAP MYAP*** MYAP MYAP Time Effort Cost Fuel Use Water Use Cooking Oil Use LRP*** LRP** LRP LRP*** LRP*** MYAP LRP* LRP MYAP LRP MYAP N/A MYAP* LRP MYAP MYAP LRP N/A Note: Zambia method measures relative to equiv. commodity available in local markets. Others use absolute measures. 12
Impacts: Preliminary Findings Recipient Satisfaction Estimated multivariate ordered logit models to control for potentially confounding factors. Results very similar to straight bivariate comparisons of LRP vs. MYAP. Core results: - Almost all food aid recipients satisfied with products on all dimensions. - But LRP recipients consistently most satisfied. This holds across countries and commodities. - But recipients preparation costs of LRP commodities often higher ... partly due to commodity differences? 13
Impacts: Preliminary Findings Impacts on Smallholder Suppliers In Burkina Faso, used same matched MYAP/LRP zone technique to survey smallholder cowpea producers, comparing those supplying the LRP with otherwise identical ones in MYAP zone selling just into regular market system. Assess impacts relative to control group (LRP cowpea suppliers vs. cowpea farmers in MYAP region) Behavioral impacts investments, improved storage, management practices (e.g. use of improved seed), use of credit Profitability impacts - self-reported improvements in profitability, farmgate price, transaction costs, time and distance travelled 14
Impacts: Preliminary Findings Impacts on Smallholder Suppliers Relative to previous year (intended and/or actual) participants: had a better understanding of quality standards for cowpeas. decreased travel time and distance traveled selling cowpeas by (stat. sig.) average margins of 52% and 91%, respectively. received 49% higher cowpea prices and 41% higher revenue, on average. enjoyed greater profitability in cowpea sales no more likely to use improved farm management practices direct LEAP participants adopted improved storage practices (such as storing cowpeas in double- or triple-lined bags) due to their involvement in the program. 15
Impacts: Preliminary Findings Impacts on Food Price Levels Does LRP drive up food prices for farmers and/or consumers? Developed a statistical model to estimate the effect of LRP on food prices in local markets, controlling for a range of other factors that influence prices: inflation, climate (temp/precip) shocks, transport costs, seasonality, world market prices, WFP LRP activities in subject and neighboring countries, etc. Spatial effects when the price of one commodity in a location changes as a result of the procurement (or distribution) in another location Intertemporal effects - when the price of a commodity at a location changes as a result of and earlier procurement Not strictly causal estimates due to potential for omitted relevant variables (e.g., government policies). But pretty good. 16
Impacts: Preliminary Findings Impacts on Food Price Levels Regression estimates of LRP s market price impacts Contemporaneous Price Impacts 8.0 5.67 Wheat Estimated % Change Price Level 6.0 Maize Maize 2.23 Veg. oil Veg. oil 4.0 3.90 2.0 0.0 Kyrgyz Kyrgyz Niger Niger Burkina Burkina Zambia Zambia Guatemala Guatemala -2.0 -4.0 -6.0 -8.0 -6.59 B'fast Meal Meal -9.94 Roller Meal Meal B'fast -10.0 Roller -12.0 Red indicates statistically significant coefficient estimates. Larger markers indicate greater statistical significance. Larger markers indicate greater statistical significance. Red indicates statistically significant coefficient estimates. 17
Impacts: Preliminary Findings Impacts on Food Price Levels For most commodities and countries, there is no economically or statistically significant impact on prices. For the few for which there is some impact, it can be either positive or negative, even within the same country (e.g., Zambia) or for the same commodity (e.g., maize and maize meal across Niger and Zambia). The possibility of significant induced price effects underscores the importance of market monitoring. The relative infrequency of such effects suggests that LRP can be undertaken effectively when well designed and monitored. Any price effects typically vanish within two months. 18
Impacts: Preliminary Findings Impacts on Food Price Volatility Does LRP increase food price volatility in recipient country markets? Used the same statistical model to estimate the effect of LRP on food price volatility, measured as the standard deviation of local market prices, again controlling for a range of other factors that influence prices: inflation, climate (temp/precip) shocks, transport costs, seasonality, world market prices, WFP LRP activities in subject and neighboring countries, etc. Again, not strictly causal estimates due to potential for omitted relevant variables (e.g., government policies). But pretty good. 19
Impacts: Preliminary Findings Impacts on Food Price Volatility Regression estimates of LRP s market price volatility impacts Contemporaneous Price Volatility Changes 12.0 Est. % change in std. dev. price 10.0 9.81 Millet 4.71 Wheat 8.0 6.0 4.0 2.83 2.0 Zambia Kyrgyz Niger Guatemala 0.0 -0.42 CSB Burkina -2.0 -4.0 -6.0 Red indicates statistically significant coefficient estimates. Larger markers indicate greater statistical significance. Market price volatility impacts negligible, short-lived, uncommon. 20
Impacts: Preliminary Findings Summary of Preliminary Findings Local purchase offers big gains in timeliness, at least 60% (14 weeks) quicker delivery than shipments from the US. In cereals and pulses, there are considerable cost savings (~50% for cereals). Locally purchased processed products (e.g., incaparina, vegetable oil), however, are often more expensive. Recipients routinely prefer locally purchased commodities along any of multiple dimensions, although preparation time and costs of LP commonly greater. 21
Impacts: Preliminary Findings Summary of Preliminary Findings (2) In Burkina Faso, smallholder suppliers enjoyed high prices and revenues and lower transactions costs. For most commodities/countries, we find no economically or statistically significant impact on prices. But it does happen, which underscores the importance of market monitoring. Any price effects typically vanish within two months. Market price volatility impacts are uncommon and when they occur, small and short-lived. 22
Lessons Learned Lessons Learned General Rigorous M&E of LRP programs necessary to assess benefits and limitations and to communicate w/ policymakers/stakeholders. LRP Learning Alliance critical for credible LRP assessment and potentially helpful for capacity building. But imposes real added efforts (and costs) on all participants. Reporting requirements were somewhat onerous Due to different objectives across LRPs Attempts to streamline CU,USDA and USAID data reporting requirements are challenging Failed effort at establishing a common database 23
Lessons Learned Lessons Learned (2) Data collection and analysis Cornell did not use all primary data generated (esp., prices) Quality of primary data was often poor. Coverage is typically of short duration, making it difficult to establish effects of LRP independent of other factors. Most countries have reasonably long, good quality commodity price series available from secondary sources (Ag ministries, Stat agencies, GIEWS, FEWS, etc.) PVOs can access, monitor and analyze secondary data more effectively than at present. Lower cost, higher quality. General problem of market info as a club good . 24
Lessons Learned Lessons Learned (3) Policy A critical question: What is the purpose of an LRP? Multiple possible multiple objectives: reduce cost or delays, help small farmers, greater recipient satisfaction. The objective(s) drive whether LRP makes sense, what to monitor, and what impacts to monitor. There may be important tradeoffs among objectives. Need to consider such potential tradeoffs explicitly. Ex ante response analysis is needed to establish whether/ where/when LRP makes sense. Yet little or no incentive exists to undertake serious response analysis under present program structures. And PVOs have little capacity to do good response analysis, especially individually. 25
Conclusions Overall, US PVOs LRP programs appear to substantially improve timeliness and reduce costs of food aid distribution, while generating increased recipient satisfaction with rations and some evidence of gains to smallholder suppliers, all without any consistent evidence of causing harm via significant price or price volatility effects. LRP not justified in all cases, but successful on most counts so good reason to push for it to become a broader option. Couple an expanded LRP option with response analysis to choose the right tool for the task and some sort of consortium M&E platform to reduce costs of high quality of M&E. 26
Thank you for your time, attention and comments! 27
