Understanding Nonclassical Measurement Error in Farmers' Responses to Information Treatments

Slide Note

Nonclassical measurement error (NCME) goes beyond mere misreporting or noise, offering insights into respondents' behaviors and beliefs. This study in Malawi embeds a randomized information experiment within a national agricultural survey to investigate the extent, correlates, and persistence of NCME. It focuses on farmers' responses to new information aimed at correcting measurement errors and explores how they learn and adjust to accurate information regarding plot size and non-land agricultural inputs.

knig_9 Follow

Uploaded on Dec 05, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript

Nonclassical Measurement Error and Farmers Response to Information Treatment Kibrom Abay, Chris Barrett, Talip Kilic, Heather Moylan, John Ilukor, Wilbert Drazi Vundru Seminar presentation to CEIDS December 16, 2022

Background Non-classical measurement error (NCME) widespread. Usually treated as a problem to be resolved econometrically or through improved survey design. NCME may represent more than mere misreporting/noise. If NCME reflects respondents mistaken beliefs (Abay et al. AJAE 2021), it offers a window on respondents behaviors, esp. concerning information. Mistaken beliefs could reflect behavioral anomalies: inattention, self- esteem or confirmation bias, etc. Mistaken beliefs about one subject may spillover to beliefs about, and reporting on, other subjects with implications for both policy design and statistical inference Information interventions rely on subjects updating beliefs. Do they?

What we do Embed a randomized information experiment within a nationwide agricultural household survey in Malawi to study: The extent, correlates and persistence of NCME Farmers response to new information meant to correct NCME How farmers learn and adjust to demonstrably accurate information about plot size, both in terms of correcting pre-existing measurement error in plot size as well as their reporting behavior (responses) on non-land agricultural inputs.

Experiment Built into the Malawi National Crop Cutting Study implemented during the 2019/20 IHS5 (LSMS) national household survey. National sample of 72 rural EAs, selected at random from the sample of IHS5 EAs that were visited Dec 2020-Feb 2021 In each EA, 24 maize cultivating households randomly selected 16 into treatment, remaining 8 as control group. (Balance tests indicate that randomization worked properly.)

Experiment (contd) Treatment Group Control Group Post-planting + post-harvest visit Post-planting visit same as post-harvest in control except for information treatment: After self-reports and plot visit, share GPS-based area and self-report measurement error (in levels and as a share of true area) Post-harvest visit exactly same as controls: Ask again about the selected plot area Mirrors the current interview flow in Malawi IHS5/IHPS and LSMS-ISA-supported surveys Post-harvest visit only w/unified agricultural questionnaire self-reported in 1 sitting by plot managers, prior to plot visit Randomly select 1 maize plot and accompany that plot s manager to the plot for demarcation and GPS-based area measurement

Familiar findings on NCME (1) - Regression to mean in NCME: self-reported plot sizes are overestimated for smaller plots, underestimated for larger ones. - NCME is both relatively and absolutely larger for smallest plots. Table 2: Discrepancies between self-reported (SR) and GPS-based area (acres) Plot size quartile 0-25% 25-50% 50-75% 75-100% Total Notes: The analysis uses pooled treatment and control group samples. SR stands for self-reported plot areas, which were reported prior to the treatment during the post-planting interview for the treatment group, and during the post-harvest interview for the control group. GPS stands for plot areas using handheld Global Positioning Systems. Quartile-specific mean and relative biases as a percent of (mean) GPS-measured plot size are reported. *** represents statistically significant differences between GPS and SR means at 1 percent level. Observations Mean SR 0.28 0.46 0.66 1.09 0.62 Mean GPS 0.10*** 0.26*** 0.54*** 1.22*** 0.53 Relative bias (%) without treatment 193.73 74.03 23.93 -10.57 18.27 387 389 390 380 1,546

Familiar findings on NCME (2) and newer ones - Rounding is a major source of measurement error in self-reported plot area. - Newer finding: rounding error persists (~unchanged) even after information treatment

Newer findings on NCME (2) - Heaping around focal plots is asymmetric, esp. for smallest plots

Familiar findings on NCME (3) and newer ones (3) - ME is heavily NC, and overwhelmingly associated w/true plot size and rounding Table 3: Characterizing measurement error in self-reported plot size - as observed prior to treatment (1) Log (SR)-log(GPS) OLS estimates Plot size Log (area: GPS)-centered -0.680*** (0.017) Log (area: GPS)-centered-square -0.014 (0.011) Rounding of values Rounding at 0.5 acre 0.419*** (0.034) Rounding at 1 acre 0.905*** (0.042) Rounding at 1.5 acre 1.186*** (0.070) Rounding at 2 acres 1.430*** (0.096) Household characteristics Plot characteristics Constant 0.006 (0.160) Controls Yes Mean dependent variable 0.313 R2 0.548 No. observations 1535 No. censored observations - Notes: The analysis uses pooled treatment and control group samples. Self-reported plot areas were reported prior to the treatment during the post-planting interview for the treatment group, and during the post-harvest interview for the control group. The first column provides OLS estimates, and the second column reports Shapley decomposition associated with the R2 in the first column. The third and fourth columns are Tobit estimates. Prior to expressing it natural logarithmic terms, GPS-based plot size was demeaned to center the data. Household characteristics include the female identifier, age, literacy, religion, marital status, and non- farm work status of the household head, as well as total farm size and number of plots managed. Plot characteristics include indicator variables for tenure status, rental or owned, pure stand cropping, soil type, and slope. Standard errors, clustered at enumeration area level, are given in parentheses. *p < 0.10, **p < 0.05, ***p < 0.01. Appendix Table A4 reports full regression results. (2) (3) (4) %Overestimation Tobit estimates -210.759*** (6.953) 11.307*** (4.097) 120.048*** (11.903) 270.761*** (15.149) 354.823*** (24.455) 401.143*** (32.847) -94.661* (55.243) Yes 96.248 1535 554 %Underestimation Tobit estimates 37.617*** (1.859) 8.928*** (1.039) -19.058*** (2.781) -58.441*** (4.025) -81.165*** (6.902) -92.946*** (9.508) -11.636 (13.848) Yes 10.423 1535 1035 Shapley 78.72% 18.28% 1.69% 1.32% 100%

Newer findings on NCME (4) NCME is the norm. Only 3.1% (31/982) accurately self-report plot size pre-treatment. Systematic NCME strikingly consistent w/behavioral phenomena: - Inattention to salient information: may be rational (costs>benefits) or irrational (mental gaps, oversight). Variation with true plot size suggests structurally asymmetric inattention. - Self-esteem bias: land a source of status. Upwardly biased mistaken beliefs may bring non-material reward. Asymmetric NCME and focal point bunching consistent w/this.

Familiar findings on NCME (4) NCME at least partly reflects farmer misperceptions (not just misreporting). Building on Abay et al. (AJAE 2021), look for plot size ME reflected in non-land inputs (seed, fertilizer, labor) self-reported post-treatment post-planting: ?? = ?0+ ?1?? + ?2???+ ?3??+ ?? If reject null 2=0 in favor of 2 >0, then NCME reflects farmer misperceptions (not just misreporting) and/or correlated NCME (Abay et al. JDE 2019). Overwhelmingly reject null in any specification. Finding reinforced by asymmetry: self-reported non-land inputs more strongly associated with underreporting of plot size than overreporting.

Familiar findings on NCME (4) Table 4: Measurement error in plot size and non-land input allocation (2) Input allocation per plot log (fertilizer) Panel A: Measurement error and input allocation: full sample 0.685*** 0.655*** (0.050) (0.028) 0.183*** 0.241*** (0.058) (0.039) 0.30 0.35 1034 1529 Panel B: Measurement error and input allocation: control sample 0.786*** 0.686*** (0.055) (0.043) 0.338*** 0.340*** (0.070) (0.053) 395 552 Panel C: Measurement error (over-/under-estimation) and input allocation: full sample 0.747*** 0.649*** 0.628*** (0.038) (0.049) (0.034) 0.008*** 0.002 0.005** (0.002) (0.002) (0.002) -0.094*** -0.044** -0.056*** (0.015) (0.019) (0.015) Panel D: Measurement error (over-/under-estimation) and input allocation: control sample 0.762*** 0.774*** 0.684*** (0.051) (0.056) (0.043) 0.014*** 0.010*** 0.013*** (0.003) (0.003) (0.002) -0.047*** -0.071*** -0.053*** (0.015) (0.019) (0.019) (1) (3) (4) (5) (6) (7) (8) Input allocation per acre log (fertilizer/acre) log (seed) log (total labor) log (household labor) log log (total labor/acre) log (household labor/acre) (seed/acre) 0.781*** (0.035) 0.373*** (0.046) 0.42 1535 0.619*** (0.032) 0.216*** (0.042) 0.30 1527 -0.219*** (0.035) 0.373*** (0.046) 0.28 1535 -0.315*** (0.050) 0.183*** (0.058) 0.16 1034 -0.345*** (0.028) 0.241*** (0.039) 0.30 1529 -0.381*** (0.032) 0.216*** (0.042) 0.27 1527 log(plot size: GPS) Measurement error-pre-treatment R-squared No. observations 0.760*** (0.054) 0.350*** (0.075) 558 0.655*** (0.048) 0.335*** (0.059) 550 -0.240*** (0.054) 0.350*** (0.075) 558 -0.214*** (0.055) 0.338*** (0.070) 395 -0.314*** (0.043) 0.340*** (0.053) 552 -0.345*** (0.048) 0.335*** (0.059) 550 log(plot size: GPS) Measurement error-pre-treatment R-squared 0.596*** (0.036) 0.004* (0.002) -0.055*** (0.015) -0.253*** (0.038) 0.008*** (0.002) -0.094*** (0.015) -0.351*** (0.049) 0.002 (0.002) -0.044** (0.019) -0.372*** (0.034) 0.005** (0.002) -0.056*** (0.015) -0.404*** (0.036) 0.004* (0.002) -0.055*** (0.015) log(plot size: GPS) % SR overestimate (/10) a % SR underestimate (/10) a 0.596*** (0.036) 0.004* (0.002) -0.055*** (0.015) -0.238*** (0.051) 0.014*** (0.003) -0.047*** (0.015) -0.226*** (0.056) 0.010*** (0.003) -0.071*** (0.019) -0.316*** (0.043) 0.013*** (0.002) -0.053*** (0.019) -0.347*** (0.047) 0.012*** (0.002) -0.058*** (0.021) log(plot size: GPS) % SR overestimate (/10) a % SR underestimate (/10) a

Newer findings on NCME and learning Learning is sluggish among treated - 13% accurately self-report post- treatment. - Plurality (37%) don t change incorrect beliefs. - 23% report larger ME post-treatment! - 13% switch from one rounded, wrong value for another wrong, rounded one.

Newer findings on NCME and learning Incomplete and structurally asymmetric learning: Under-estimators with larger plots more likely to update and by a larger amount.

Newer findings on NCME and learning Incomplete and structurally asymmetric learning: Only 13% (up from 3%) correct. Overestimation magnitude unchanged but underestimation falls.

Newer findings on NCME and learning Incomplete and structurally asymmetric learning:

Newer findings on NCME and learning Information treatment has - no effect on fertilizer use reporting at extensive margin - significantly changes intensive margin self-reports (~7%). - treatment effects heterogeneous by plot size and by under/over-estimation. - clear evidence of spillovers on self- reporting of (early season) agricultural input use ... correlated NCME (Abay et al. JDE 2019). Optimal prediction error (Hyslop and Imbens JBES 2001)?

Conclusions and implications Considerable NCME in SR plot areas reflects asymmetric rounding and regression- to-mean consistent with behavioral anomalies such as inattention, self-esteem bias. NCME persists among the treated 3 to 4 months following treatment. Very incomplete and asymmetric updating. Upward corrections more common. Extent of updating varies with true plot area and direction/magnitude of pre-treatment ME. Widespread mistaken beliefs and asymmetric updating suggests behavioral anomalies that matter for: Design and implementation of information-based interventions Survey data collection and statistical inference (esp. correlated NCME)