By Kristian Jåtog Trygstad, partner|lawyer, Advokatfirmaet Mageli
In two new posts on Anbud365, the pricing of quality has again been raised as a topic.[1] Both posts point to challenges with such evaluation methods. One of the posts points to relative point calculation methods as a simpler procedure for the clients.
Choosing a method to find the best offer is an important part of a procurement. If one offer has the lowest price and the best quality, the outcome of the competition is self-evident. As a rule, however, the offers have different strengths and weaknesses. One offer may, for example, have a low price while another has a somewhat higher price and better quality. The outcome of the competition then depends on how these strengths and weaknesses are held up against each other.
In such an overall assessment, the client is dependent on a suitable basis for comparison to assess different advantages against each other. Different evaluation methods can be used to structure the evaluation and the trade-off between low price and good quality.
The clients have a relatively wide discretion in this phase, but the discretion is still far from free. At the same time, one can somewhat simplistically say that a choice of offer that is predictable and well commercially justified, which will most often also be in accordance with the requirements of the procurement regulations. Use of suitable evaluation methods facilitates good and legal procurement. However, there is no automaticity in that an evaluation method ensures that the award criteria are judged in a comparable way. A method that is suitable in one procurement may also prove to be unsuitable in another.
As one of those who have gone the furthest in recommending evaluation with price as a basis for comparison, it is natural for me to comment on the challenges highlighted in the two articles. To avoid repeating myself too much, I refer to other sources for a review of the procedure when price is used as a basis for comparison and the many advantages of these methods.[2]
Visibility of willingness to pay for good quality
I agree with the article authors that pricing quality requires that the clients have an active and conscious relationship with how much they think good quality is worth. Clients must make important – and sometimes difficult – choices in competitions where price is used as a basis for comparison. However, such difficult choices are made by the clients in the vast majority of procurements.
The trade-off between price and quality is more visible when price is used as a basis for comparison. But the clients also weigh price and quality against each other – consciously or unconsciously – when the offers are compared using scoring methods.
- It can be difficult to determine the maximum willingness to pay for optimal quality, but this is also the case when determining the weight of the award criteria. If there is a difference between these decisions, it is that the consequences are clearer for the client (and for the suppliers). However, this increased insight should be an advantage as it facilitates more thoughtful and better assessments. The chance of success is greater if the client makes well-thought-out choices related to their own preferences.
- It can also be difficult to calculate the correct price addition or price deduction in the assessment of the award criteria. However, the contractors make the same assessments when points are calculated. The point advantage that the offer with the lowest price gets on the price criterion can be offset by a point advantage the offer with the best quality gets on the quality criterion, and vice versa. Here, too, the possible difference is that the consequences of the client’s assessments come forward more clearly, which facilitates better decisions.
Although the clients have to make several difficult choices if price is used as a basis for comparison, these choices will also be made using point methods. If price is used as a basis for comparison, the clients also avoid a difficult choice, namely finding a method to turn price differences into point differences. When the offers are compared using scoring methods, the method for judging the price criterion must be determined before the offers are opened. If the clients miscalculate the level of the offer prices, it could make the evaluation methods less suitable for identifying the best offer.[3] This is the case with methods that use price and points. If the client uses predefined methods to calculate price additions/deductions, the methods will, however, be more robust to variations in the offers.
It is not a given that it is a disadvantage for the clients that the evaluation becomes more transparent. The comparison of price and quality is more intuitive than the more abstract point assessments. It should be an advantage for the clients since more information facilitates better and more accurate decisions.
Nor is it the case that increased insight into the evaluation increases the Board of Appeal for Public Procurement and the courts’ ability to review the client’s choice of offer. Similar to assessments of points on the award criteria, there may be significant elements of discretion when calculating price adjustments. In case 2022/20, for example, the Board of Appeal accepted the client’s price adjustments even though a supplier had tried to show that the price adjustment was greater than the assumed market value.[4]
The contractors must understand the methods they use
The offer evaluation is not necessarily made simple by using price as a basis for comparison. It can still be easier to do correctly than if the client uses other methods. The challenges the clients will face, they will often face regardless of which method is used. It then seems sensible to choose methods where unnecessary challenges in the offer evaluation can be avoided, so that the focus can be directed towards dealing with the challenges that cannot be avoided.
The central challenge is then how to determine the willingness to pay for quality improvements. In Pricing quality – what are clients reluctant to do? the authors have proposed that this challenge can be avoided by using relative point models, where the best offer receives the maximum payout. It is advice that I hope the clients will not readily follow. Such methods can be easy to use, because the methods are based on the best offer being a frame of reference. More important, however, is that they can be difficult to use correctly, because it is relatively often the case that the best offer is unsuitable as a frame of reference.
Weaknesses of methods that calculate points on the price criterion have been thoroughly reviewed elsewhere.[5] Among the themes that have been highlighted are weaknesses in so-called relative methods, where the lowest bid price in the competition is used as a frame of reference. Coincidences can have an impact on the outcome of the competition, and then the methods will not be consistent. The client’s ranking of two offers may be influenced by which other offers are evaluated. This topic has been discussed extensively and thoroughly previously on anbud365, as part of the discussion about normalization.[6]
I still completely agree that there is a need for more guidance on how to proceed when willingness to pay is to be determined as part of price being used as a basis for comparison. Here, DFØ can contribute with guidance, and it will also be very useful with practical examples and experiences from those clients who have used price methods in their procurement.
. Nor will general templates take nuances into account, for example due to special features of an acquisition or special considerations on the part of the client. The willingness to pay for better consultants will depend, for example, on what work these consultants will do, how difficult the assessments will be, and what the consequences are if one consultant does a slightly worse job than another would have done. Guidance material will, however, be able to contribute to raising general competence and to help clients in the right direction.
It is hardly realistic that you will be able to get general templates that can be used without any processing on the part of the client. There is no definitive answer to what is the right relationship between price and quality. Nor will general templates take nuances into account, for example due to special features of an acquisition or special considerations on the part of the client. The willingness to pay for better consultants will depend, for example, on what work these consultants will do, how difficult the assessments will be, and what the consequences are if one consultant does a slightly worse job than another would have done. Guidance material will, however, be able to contribute to raising general competence and to help clients in the right direction.
[1] Reflections after the webinar on offer evaluation on 20/03/24 and Pricing of quality – what are clients reluctant to do?
[2] See, for example, Trygstad: Awarding of public contracts chapter 29.3, Trygstad: Appreciate good quality (included in Contracts and Procurements – journal for Lasse Simonsen), A better evaluation model? – Anbud365, Good affairs – a strategy for sustainable public procurement (regeringen.se), especially section 3.4.3 and Methods for evaluating price and quality in public procurement (konkurrensverket.se).
[3] See, for example, Does the money disappear in the points? – Anbud365 and Challenges with Difi’s new tool for tender evaluation – Anbud365.
[4] See case 2022/20 section 51 et seq.
[5] See, for example, Methods for evaluation of price and quality in public procurement (konkurrensverket.se), A logical trap – relative scoring of price in tender evaluation in public procurement (konkurrensverket.se) Bergmann and Lundberg: Tender evaluation and supplier selection methods in public procurement and Trygstad Awarding of public contracts 2nd ed. chapters 25 to 28.
[6] See for example Normalization can arbitrarily change the weight of the award criteria – Anbud365, Normalization, a relatively bad idea – Anbud365, Normalization – legal and appropriate – Anbud365, Why normalize when you can not? – Anbud365, Normalization – communicating the weight of award criteria – Anbud365, Let’s continue talking about evaluation methods – Anbud365.
