Sampling Under the RRF - General Principles and Methods
Sampling under the RRF is essential for the Commission to ensure reasonable assurance of fulfillment of numerical milestones or targets. The approach involves statistical risk-based random sampling with specific thresholds and considerations for different types of milestones. Various statistical tables are used to determine compliance rates, with a focus on achieving minimum required rates through random sampling methods. Overachievement of targets allows for flexibility in assessing the sample. Sampling can occur during assessments of payment requests or prior to claims of evidence availability by Member States.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
Presentation Transcript
SAMPLING UNDER THE RRF Alexandru Ciungu, DG ECFIN 24 January 2024
Sampling - General principles Why sampling? As part of ex-ante controls of the RRF, for the Commission to gain reasonable assurance (required by the Financial Regulation) that the MS has evidence available which justifies the claim that a numerical milestone or target with a large number of units (which cannot all be checked) is satisfactorily fulfilled. Article 2(2.2) of the OAs empowers the Commission to sample, irrespective of whether this possibility is mentioned in the verification mechanisms of the specific M/Ts.
Sampling - General principles In which cases does the Commission sample? As a rule, for all numerical M/Ts with values higher than 80. For M/Ts with lower values, all pieces of evidence will be checked. The statistical threshold is 60, but all evidence will be checked when there are under 80 units, to avoid the workload with selecting the sample. For non-numerical M/Ts where the assessment of satisfactory fulfillment necessarily requires the verification of a large number of pieces of evidence (same threshold applies).
Method of sampling Standard approach used: statistical risk-based random sampling 95/95 approach i.e. we are aiming at reasonable assurance of a 95% minimum required compliance rate, with 95% statistical power. Other sampling methods exist. This one was selected because the required sample size (60) is reasonable: it is the most cost-effective to apply.
Method of sampling statistical table Statistical power 90% 230 114 76 45 32 22 15 80% 161 80 53 32 23 16 10 85% 189 94 63 37 26 18 12 95% 299 149 99 59 42 29 19 99% 459 228 152 90 64 44 29 99% 98% 97% 95% 93% 90% 85% Minimum required compliance rate
Method of sampling Random sampling is always used. It may be complemented (but cannot be replaced) by non- random (non-statistical) sampling, depending on the nature of the M/T and of the available evidence. Overachievement is taken into account when assessing the sample. If a M/T is not overachieved, all units in the sample need to be successfully assessed, but the more overachieved a M/T is, the more leeway there is for some units to fail a statistical test is used to determine the exact number.
Sampling step-by-step When does the Commission sample? Sampling can take place during the Commission's assessment of a payment request or before, as soon as the MS claims it has all evidence available for a certain M/T. Running the sampling outside of a payment request is the preferred option, to better manage workload both for Commission services and MS counterparts.It is in everybody s best interest.
Sampling step-by-step The MS provides (along with the report justifying the satisfactory fulfilment of the M/T) a numbered listof units with the key parameter that drives the satisfactory fulfilment, including unique identifiers (to allow linking the evidence to the list). The Commission randomly selects a sample, using an automated tool, and requests the associated evidence from the MS. The evidence corresponding to the sample should be provided by the MS within 5 working days (through FENIX). If longer clock is stopped under a payment request.
Sampling step-by-step Key principle for the Commission in the assessment process: the evidence needs to substantiate the specific requirements for the M/T, as set out in the CID (M/T description or measure description, if directly or indirectly related to that M/T) or the further specifications column of the OAs, if any. All such requirements will be checked for the sample requested, unless there is a way to verify them horizontally (i.e. for all units in the initial list).
Results of the sampling decision tree Request correction and new sample* of 228 (if population>250) or evidence for all units No reasonable assurance, sampling failed No Is the target overachieved? No No All 60 units in sample assessed positively? Use statistical test to determine satisfactory fulfillment. Is result positive? All units in sample assessed positively? Yes No Yes Yes Sampling failed twice, M/T not fulfilled Yes Satisfactory fulfillment *Second sampling step can be skipped in agreement with MS and payment suspension percentage will then be determined based on the results of the first sampling step
How to ensure that sampling succeeds? Plan ahead and prepare for sampling together with your Commission counterparts. Identify early on which M/Ts in a payment request will require sampling. Identify which requirements need to be proven for the units of those M/Ts and jointly test which specific pieces of evidence can be used. Ensure that you have all evidence available. Before sampling starts, submit all evidence for one unit to allow the Commission to check that the types of evidence are appropriate (mock test). Experience shows that sampling can be a time-consuming and challenging process, but is facilitated by advance preparation.
How to ensure that sampling succeeds? As much as possible, start providing evidence for sampling well ahead of the official submission of the payment request and as soon as you have all evidence for a particular M/T ready. Sampling is best performed before the submission of the payment request. As much as possible, overachieve numerical M/Ts, as this provides significant flexibility under sampling (i.e. some units in the sample can fail and the M/T can still be considered fulfilled).
