Evaluating Past Performance for Competitive Procurements Training Overview

Understanding past performance evaluation in federal procurements is crucial for assessing contractors' abilities. This training presentation covers the relevant regulations, evaluation sources, getting started guidance, and considerations like recency. The ultimate goal is to reach a consensus Level of Confidence rating through documented rationale.

• Procurement Training
• Past Performance Evaluation
• Federal Regulations
• Contractor Performance
• Competitive Bidding

gide_1 Follow

Uploaded on Sep 16, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. Evaluating Past Performance GSFC Training Presentation for Competitive Procurements By: Chris Whyte and Steve Kramer March 20, 2014 1

2. Past Performance Overview Federal Acquisition Regulation (FAR), Section 15.3 provides, Past performance information is one indicator of an offeror s ability to perform the contract successfully. The currency and relevance of the information, source of the information, context of the data, and general trends in contractor s performance shall be considered. NASA FAR Supplement (NFS), Section 1815.305(a)(2)(A)) provides, The Past Performance evaluation assesses the contractor's performance under previously awarded contracts. The past performance evaluationshall be in accordance with FAR 15.305(a)(2) and this section. When applying the definitions below to arrive at a confidence rating, the evaluation teamshall consider and clearly document each offeror s relevant past performance (e.g. currency/recency, size, content and complexity) to assess the offeror s past performance data and assign anoverall confidence rating. The past performance evaluationis an assessment of the Government s confidence in the offeror s ability to perform the solicitation requirements. The NFS contains specific Level of Confidence ratings, with definitions (See Slides 27-28). GSFC Past Performance Training March 20, 2014 2

3. Past Performance Overview The Past Performance evaluation may be based on any/all of the following sources of information: The offeror s proposal explanation of relevance/pertinence of experience to the current procurement, performance success, and potential lessons learned and corrective actions. Past Performance Questionnaires (if used) from customers to evaluate the offerors performance and to corroborate claimed experience. Annual Past Performance Evaluations (e.g. CPARS, NASA Form 1680, Award Fee Documentation, or other Agency equivalent) to evaluate performance. Other information available (e.g., evaluator knowledge to supplement other information, customer interviews (if needed), etc.) Clarifications and/or Discussions (if utilized) GSFC Past Performance Training March 20, 2014 3

4. Getting Started The ultimate objective of the SEB* is to arrive at a consensus Level of Confidence rating with a documented supporting rationale. This training provides practical guidance to the SEB on the process of evaluating past performance within NASA Goddard Space Flight Center (GSFC) competitive procurements. These slides address a process that is consistent with GSFC procurement templates for the Past Performance Factor (Provision GSFC 52.215-330 (Services) or 331 (Hardware)). If these templates are not used, or if they are modified, the Contracting Officer needs to make sure the evaluation is consistent with your specific RFP evaluation criteria. *This guidance provided herein is also applicable for Integrated Evaluation Teams (IETs) below the SEB level. GSFC Past Performance Training March 20, 2014 4

5. Evaluation of Past Performance - Recency As provided in the FAR and NFS, the SEB must consider the currency/recency of the offeror s past performance. This includes prime offerors and significant subcontractor references, if applicable. Typically, the solicitation will specify the time period during which past performance is considered recent. Recency is generally a threshold matter: if the past performance is within the timeframe defined in the RFP as recent, it will be considered. If it occurred outside of this timeframe, it is not relevant and should not be evaluated. GSFC Past Performance Training March 20, 2014 5

6. Evaluation of Past Performance The NASA Past Performance Level of Confidence definitions in NFS Part 1815.305 have two components: Relevance and Performance. In the definitions, the term Pertinent is consistent with relevance. Both relevance and performance must be evaluated for each offeror s past performance references (prime offeror and significant subcontractors, if applicable), and ultimately summarized, before the SEB assigns an overall Level of Confidence rating. Relevance considers the similarity between the work performed under previous contracts and the work being procured in terms of size and content. Content is defined in the solicitation (template) as follows: the type and complexity of services, work, or supplies, in comparison to the requirements of this solicitation. Performance considers how well an offeror has performed relevant contractual obligations. GSFC Past Performance Training March 20, 2014 6

7. Evaluation of Relevance Evaluating Relevance: The relevance evaluation includes two components: Size and Content. GSFC evaluations for relevance use the same rating labels as the Level of Confidence definitions (Very High, High, Moderate, Low). The Very Low rating is not included for relevance since the Very Low Confidence definition only includes Performance related language. Size is a comparison of each of the offeror s past performance reference s average annual cost/fee to the estimated annual cost/fee associated with your specific solicitation. This is typically based on the Government Estimate. For IDIQ contracts, the Government Estimate should be used instead of the maximum IDIQ ordering value, which typically includes additional contingency. GSFC Past Performance Training March 20, 2014 7

8. Evaluation of Relevance - Size Size relevance is rated on the scale from very highly pertinent down to not pertinent. GSFC solicitations generally include a minimally relevant dollar threshold that must be achieved in order for a past reference to be considered for the evaluation. If the size of a particular reference contract does not meet the minimum, that contract is not pertinent, and should not be evaluated any further for relevance or performance. For references meeting the minimum threshold, size relevance is determined by making a numerical comparison of each of the offeror s past performance reference s average annual cost/fee to the estimated annual cost/fee associated with your specific solicitation. SEBs are expected to use their judgment in determining how relevant past references are in terms of size, but the evaluation must be fair and consistent for all offerors. Example 1: If the average annual cost/fee for a specific solicitation is $10M, and an offeror s referenced contract has an average annual cost/fee of $9.5M, most SEBs would consider that to be Very High for size relevance, since it is very close to the value of the solicitation. Example 2: If an offeror s referenced contract has an average annual cost/fee of $5M, an SEB might consider that as Moderate for size relevance, since it is only half of the value of the solicitation. The most important thing is to be fair and consistent for each offeror and each contract reference. GSFC Past Performance Training March 20, 2014 8

9. Evaluation of Relevance Content (including complexity) Content evaluation involves comparing the type and complexity of work performed on past contracts to the type of work to be performed on the instant procurement. Potential sources of information include the offeror s proposal, past performance questionnaires (if used), Government past performance databases, customer telephone or e-mail interviews (if needed), or any other sources of valid data. Typically, key SOW areas are compared to questionnaire responses, Government database reports, and/or interview responses as a primary aspect of the relevance evaluation. This SOW comparison is a key portion of the content evaluation. GSFC Past Performance Training March 20, 2014 9

10. Evaluation of Relevance Content Complexity is factored into the Content evaluation and entails comparing the relative complexity of the overall requirement on past contracts to the overall complexity of the requirement to be performed on the instant procurement. For example, if we are soliciting engineering services related to a spaceflight instrument, an offeror s past engineering services contract to build commercial generators may not be nearly as complex as our requirement. Alternatively, an offeror s past engineering services contract for an Air Force weapons system may be of similar complexity to our effort, even though the work was not spaceflight related. GSFC Past Performance Training March 20, 2014 10

11. Evaluation of Relevance Content It is the offeror s responsibility to fully describe its past performance and to demonstrate how the content of the referenced contract is relevant to the content of the instant procurement. Generally, the evaluation is derived from contract references in the offeror s proposal and the burden is on the offeror to show its performance is pertinent to the SOW. Though the offeror is primarily responsible for describing its past performance and demonstrating its relevance, the SEB should also evaluate relevant contracts of which members have personal knowledge or that can reasonably identified, specifically those within NASA. GSFC Past Performance Training March 20, 2014 11

12. Evaluation of Relevance - Content The prime offeror s past references should only be compared to the work that it proposes to perform itself on the subject solicitation, including overall management of contracts similar in content to the SOW. Example 1: The SOW included 4 primary areas of content and the prime offeror has proposed to perform Areas 1-3, while subcontracting area 4. The questionnaire(s) indicate that the prime offeror has performed areas 1-3 on a past effort, but not area 4. In this situation, the prime offeror s lack of performance in area 4 would not necessarily detract from the content evaluation, since they are proposing to have a subcontractor perform that work. Instead, the SEB would also evaluate the subcontractor to determine their content performance in area 4. GSFC Past Performance Training March 20, 2014 12

13. Evaluation of Relevance Content General Guidelines Keep the end objective in mind. Has this offeror performed contracts that are similar in content to the requirements? Do not assign weightings (%) to SOW areas and sub-elements to develop a content relevance. GSFC solicitations typically do not provide for weighting so do not introduce it in the evaluation process. Do not assign pertinence levels below the primary SOW or CLIN areas defined in the solicitation. Whether an offeror does not address a SOW area or is determined to be not pertinent in a SOW area, either scenario should have essentially the same impact on the overall relevance rating. GSFC Past Performance Training March 20, 2014 13

14. Evaluation of Relevance - Subcontractors Evaluating Significant Subcontractors: FAR 15.305 (2)(iii) provides for evaluation of subcontractors. The solicitation may contain additional instructions and criteria relevant to evaluation of the past performance of subcontractors, often termed significant subcontractors . When evaluating relevance for Significant Subcontractors, compare the references to the work that they are proposed to do. For size, a Significant Subcontractor s reference contracts should be compared to the size of their proposed subcontract for the specific solicitation. Example 1: The average annual cost/fee for a specific solicitation is $10M, and a proposed significant subcontractor has a proposed average annual cost/fee of $4M. If the significant subcontractor s referenced contract has an average annual cost/fee of $3.7M, most SEBs would consider that to be Very High for size relevance, since it is very close to the value of the proposed significant subcontract. Example 2: Using the above example, if a significant subcontractor s referenced contract has an average annual cost/fee of $2M, an SEB might consider that as Moderate for size relevance, since it is only half of the proposed value ($4M) of the significant subcontract. The most important thing is to be fair and consistent for each offeror and each contract reference. The same idea holds true for the content evaluation. The Significant Subcontractor should only be evaluated in comparison to the SOW areas they are being proposed to perform, which will often be a subset of the overall SOW. GSFC Past Performance Training March 20, 2014 14

15. Evaluation of Relevance Balancing Size and Content in determining total relevance: It is critical to reference your specific RFP to determine if and how Content and Size are weighted within the evaluation of relevance. Current guidance suggests that content should be weighted more heavily than size in most solicitations. This weighting will guide you in the overall relevance determination for each contract reference, and for the summary relevance rating for the offeror. The following slides provide examples on how an SEB may balance content and size relevance. GSFC Past Performance Training March 20, 2014 15

16. Evaluation of Relevance Example 1: Content is more important than size Size Very High Very High Very High Very High High High High High Moderate Moderate Moderate Moderate Low Low Low Low Content Very High High Moderate Low Very High High Moderate Low Very High High Moderate Low Very High High Moderate Low Total Relevance Very High High Moderate Moderate Very High High Moderate Low Very High High Moderate Low High High Moderate Low These tables are provided for illustration. The SEB s primary objective is to treat all offerors fairly and consistently when balancing size and content to determine overall relevance. GSFC Past Performance Training March 20, 2014 16

17. Examples Continued Example 2: Content and Size are not weighted, and are therefore equal Size Very High Very High Very High Very High High High High High Moderate Moderate Moderate Moderate Low Low Low Low Content Very High High Moderate Low Very High High Moderate Low Very High High Moderate Low Very High High Moderate Low Overall Relevance Very High High High Moderate or High High High Moderate Moderate High Moderate Moderate Low Moderate or High Moderate Low Low These tables are provided for illustration. The SEB s primary objective is to treat all offerors fairly and consistently when balancing size and content to determine overall relevance. GSFC Past Performance Training March 20, 2014 17

18. Evaluation of Relevance Determining Overall Relevance of all references: When considering all of the contract references provided, do not simply take an average of the individual relevance ratings to determine the offeror s overall relevancy rating. Offerors only have to demonstrate relevant experience in terms of size and content on one referenced contract to receive a particular rating. Do not downgrade a relevant contract just because another referenced contract does not meet the relevance requirements. A starting point for the overall relevance rating should be the highest individually rated reference contract. Example: if the solicitation in question is a follow-on services contract, and the incumbent offeror provides two contract references, one of which is the Very Highly relevant incumbent contract and the other of which is a Low relevance contract, the Low relevance contract does not detract from the fact that they have demonstrated Very High relevance on the incumbent contract. Therefore, their overall summary relevance would be Very High. GSFC Past Performance Training March 20, 2014 18

19. Evaluation of Relevance The overall relevance can potentially be rated higher than any individual contract reference from the offeror, particularly when content is the limiting factor for individual references. Example: Assume an offeror provided two past contract references, both of which had Very High size relevance. In this example, our solicitation includes 4 key areas of the SOW for content relevance. In one past reference, the offeror demonstrated High content relevance since the reference included SOW areas 1-3 with similar complexity to the solicitation s requirements. In the other, the offeror demonstrated High Content relevance because the reference included SOW areas 2-4. The SEB could reasonably conclude that the overall relevance of all references is Very High, since the offeror demonstrated Very High Size relevance and Very High Content relevance, when considering the combination of both contract references (SOW areas 1- 4 were covered, though they were not all part of one contract). This is not true for size relevance, which should only be looked at on a contract by contract basis. This would be the typical conclusion for this scenario. Alternatively, the SEB could assign a High rating to overall relevance if they thought it was important to demonstrate all four SOW areas within one contract. The importance of demonstrating all relevance areas in one contract needs to be clearly documented. In either case, the SEB must have a strong rationale to support their conclusion and must be consistent when evaluating all offerors. GSFC Past Performance Training March 20, 2014 19

20. Evaluation of Performance The evaluation of performance tends to be more straight forward than the relevance evaluation. Each Level of Confidence definition includes language related specifically to the performance component of the evaluation. The primary difference between the definitions is the description of any performance problems that might have occurred and their impact on overall contract performance. The SEB needs to determine a performance rating for each contract reference (Very High, High, Moderate, Low, Very Low). All sources of data should be included in this determination, but it is critical to have at least one source that is independent from the offeror s proposal, such as a questionnaire, Government Database, customer interview, etc. The SEB is not limited to looking at only the contracts provided by the offeror in looking for performance issues, however, as a general rule, contracts must be minimally relevant in order for the technical performance issues to be considered. Exceptions to this rule do exist and judgment needs to be applied. For example, systemic problems such as lack of cost control or safety violations across several contracts, regardless of content relevance may be considered in the overall performance evaluation. GSFC Past Performance Training March 20, 2014 20

21. Evaluation of Performance When using questionnaires as a primary source of independent feedback, it is important for the SEB to obtain as many of the questionnaires as possible. This includes following up with customers via e-mail and/or phone calls in an attempt to get responses. A log should be used to document this effort. If this effort is unsuccessful, the CO can contact the offeror to seek alternate points of contact to solicit questionnaire responses. When reviewing the questionnaire responses, if a reference notes adverse performance with little or no explanation or the SEB needs additional information to perform an effective evaluation, it is good practice to contact the reference for further clarification so the board can make an informed judgment when assessing the quality of performance. It is best for multiple members of the SEB to do this as a using a speaker phone (e.g. Contract Specialist and Technical Member) If that is not possible, then the person who calls should report back to the full SEB/IET Phone calls should be documented Consider having the customer provide the additional information in writing (particularly if it is different from the previously provided questionnaire). One method of accomplishing this documentation is for the Contracting Officer to summarize the phone call in an e-mail to the evaluator and request e-mail confirmation of the details. GSFC Past Performance Training March 20, 2014 21

22. Evaluation of Performance The SEB must also consider the content and consistency of the questionnaire (and/or any other sources of data) in determining their independent rating. Different ratings assigned for a similar period of time should be investigated with the customer(s). Example: Two past performance questionnaires are received for a past contract reference, one from the Contracting Officer and one from the Contracting Officer s Representative. Both had overwhelmingly positive statements about the contractor s performance, but the CO s overall rating was High and the COR s overall rating was Very High. No performance problems were identified. The SEB might reasonably assign a Very High performance rating, based on the content of both evaluations. Prior to making this determination, the SEB might contact the CO to inquire why their overall rating was only High, given the overwhelmingly positive comments in their questionnaire. Make sure to read any comments on the CPARS evaluations and questionnaires to fully understand the context of any assigned rating by the customer and to relate it to the instant procurement. GSFC Past Performance Training March 20, 2014 22

23. Evaluation of Performance If the SEB finds adverse past performance that is the determining factor that prevents the Offeror s ability to be selected or included in competitive range, it is imperative to determine whether the offeror (or significant subcontractor) ever had an opportunity to provide a response to the adverse performance information. Evidence of this notification may be in the CPARS system or the Offeror may have addressed their adverse performance in the proposal. Also, the customer for the contract in question can be contacted to determine if the notification occurred. If the adverse performance input was not provided to the offeror, then the CO will generate a written notice, coordinated with the SEB legal counsel, asking for an explanation. The CO is individual responsible for initiating the contact. This may be a clarification or communications within FAR Part 15.306(b)(1). If the company did have the chance to respond via the annual performance evaluation process, whether it took advantage of the opportunity or not, then the company does not have to be contacted to respond to adverse performance information. There is a degree of judgment and risk assessment to be applied in this decision, therefore consult with the CO and SEB Advisor. GSFC Past Performance Training March 20, 2014 23

24. Evaluation of Performance Determining Overall Performance Evaluation of all references: Unlike relevance, the overall performance evaluation can be more of an average of the performance of the past contract references. However, the average should be weighted more heavily toward the more relevant contracts. In other words, the performance of a Very Highly relevant contract should be weighted more heavily than the performance of Low relevance contracts. Example: The offeror included two contract references. One reference had Very High Relevance and Very High Performance and the other reference had Low Relevance and Very Low Performance. The SEB might reasonably assign an overall Performance rating of High, giving more weight to the Very High relevance contract. While this is a simple example, it would be important for the SEB to consider the specific issues that led to the Very Low performance rating on the second contract reference and whether or not those issues would be a concern for our specific solicitation. GSFC Past Performance Training March 20, 2014 24

25. Evaluation of Performance - Subcontractors Balancing the Prime Offeror and Significant Subcontractors: GSFC solicitations typically include the following language (Provisions GSFC 52.215-330 (Services) or 331 (Hardware)): The past performance of the prime contractor will be weighted more heavily than any significant subcontractor or combination of significant subcontractors in the overall past performance evaluation. GSFC Past Performance Training March 20, 2014 25

26. Bringing Performance and Relevance Together Balancing Overall Relevance and Overall Performance to determine the Overall Past Performance Level of Confidence rating: It is NOT an average of the Overall Performance and Overall Relevance ratings. The Level of Confidence rating is based on the lower of the Overall Performance and Overall Relevance ratings. This rating must be based on the language in the Level of Confidence Definitions, each of which includes language related to Relevance and Performance (with the exception of the Very Low , which is based on performance). GSFC Past Performance Training March 20, 2014 26

27. Past Performance Confidence Definitions NFS 1815.305(a)(2) Very High Level of Confidence -- The offeror s relevant past performance is of exceptional merit and is very highly pertinent to this acquisition, indicates exemplary performance in a timely, efficient, and economical manner and very minor (if any) problems with no adverse effect on overall performance. Based on the offeror s performance record, there is a very high level of confidence that the offeror will successfully perform the required effort. (One or more significant strengths exist. No significant weaknesses exist.) High Level of Confidence -- The offeror s relevant past performance is highly pertinent to this acquisition; demonstrating very effectiveperformance that would be fully responsive to contract requirements. Offeror s past performance indicates that contract requirements were accomplished in a timely, efficient, and economical manner for the most part, with only minor problems that had little identifiable effect on overall performance. Based on the offeror s performance record, there is a high level of confidence that the offeror will successfully perform the required effort. (One or more significant strengths exist. Strengths outbalance any weakness.) Moderate Level of Confidence -- The offeror s relevant past performance is pertinent to this acquisition, and it demonstrates effective performance. Performance was fully responsive to contract requirements; there may have been reportable problems, but with little identifiable effect on overall performance. Based on the offeror s performance record, there is a moderate level of confidence that the offeror will successfully perform the required effort. (There may be strengths or weaknesses, or both.) GSFC Past Performance Training March 20, 2014 27

28. Past Performance Confidence Definitions Low Level of Confidence --The offeror s relevant past performance is at least somewhat pertinent to this acquisition, and it meets or slightly exceeds minimum acceptable standards. Offeror achieved adequate results; there may have been reportable problems with identifiable, but not substantial, effects on overall performance. Based on the offeror s performance record, there is a low level of confidence that the offeror will successfully perform the required effort. Changes to the offeror s existing processes may be necessary in order to achieve contract requirements. (One or more weaknesses exist. Weaknesses outbalance strengths.) Very Low Level of Confidence --The offeror s relevant past performance does not meet minimum acceptable standards in one or more areas; remedial action was required in one or more areas. Performance problems occurred in one or more areas which, adversely affected overall performance. Based on the offeror s performance record, there is a very low level of confidence that the offeror will successfully perform the required effort. (One or more deficiencies or significant weaknesses exist.) Neutral In the case of an offeror without a record of relevant past performance or for whom information on past performance is not available, the offeror may not be evaluated favorably or unfavorably on past performance [see FAR 15.305(a) (2) (ii) and (iv)]. GSFC Past Performance Training March 20, 2014 28

29. Bringing Performance and Relevance Together Starting with Very High Level of Confidence and going down, each definition has language that describes lowering Relevance (Pertinence) and Performance. Example: An Offeror has demonstrated Moderate Overall Relevance and Very High Overall Performance. This would result in a Moderate Level of Confidence rating, since the High and Very High Level of Confidence ratings require highly pertinent and very highly pertinent relevance ratings. The Level of Confidence rating assigned depends upon the Performance and Pertinence ratings for each offeror, both of which establish a ceiling on how high the overall Level of Confidence rating can be. In other words, a confidence rating can only be as high as the lowest rating of either pertinence or performance. Exemplary performance + a Somewhat Pertinent relevance = Low Level of Confidence Very effective performance + a Very Highly Pertinent relevance = High Level of Confidence Exemplary performance + a Very Highly Pertinent relevance = Very High Level of Confidence GSFC Past Performance Training March 20, 2014 29

30. Bringing Performance and Relevance Together Guidelines for rating process: Summary data about the Past Performance reference contracts, such as a summary description, period of performance (dates), contract type, customer organization, and contract number may be prepared by one Board member to get the process started. All voting members should review offeror s past performance information individually. The voting members should meet as a group and decide whether any additional information is needed. All voting members should assess the relevance and performance criteria of the offeror s past performance references. The voting members meet and reach consensus on the relevance and performance aspects of an offeror s past performance and then assigns the overall Level of Confidence rating. The voting members document the consensus ratings and underlying rationale. GSFC Past Performance Training March 20, 2014 30