University of Twente Student Theses
Characteristics of rank-reversal
Nijenhuis, Yoran (2017) Characteristics of rank-reversal.
|
PDF
1MB |
| Abstract: | Research motivation: In recent years, tender procedures for selecting the best supplier in the Netherlands have been using a best value approach more and more often. Under Dutch law it is now almost always compulsory to evaluate suppliers by the Economically Most Advantageous tender method. In practice this implies evaluating suppliers on both price and quality or on quality alone. This can be done by using relative and/or absolute formulas. Relative scoring is used often for the price component; for determining the scores on price suppliers are judged towards each other with the lowest price of all being the reference. Evaluating in this way opens up the possibility of rank-reversal. Rank-reversal is a changed order after removal or entrance of a supplier. If the number of participants changes there is the possibility of a new reference price. Hence, the scores on price could have to be recalculated. Though the possibility of occurrence of rank-reversal has been proven mathematically and possible problems with this method have been acknowledged, not much research has been conducted on the real world occurrence of rank-reversal. This bachelor assignment aims to fill this gap by answering the following main research question: How often does rank-reversal occur in practice and what are the characteristics and situations under which it occurs? Supporting sub questions focused on the motives for using relative scoring methods and its (dis)advantages, a what-if analysis of historic tenders and a simulation to test different situations. Methodology: Data of historic tenders was gathered at UBR|HIS in The Hague, this yielded 252 governmental tenders. Additionally, 51 tenders where available from a previous bachelor report and tender support platform Negometrix. These historic tenders had information about the number of participating suppliers, the weight for both quality and price, the number of sub-criteria, the obtained quality scores and the offered prices. In the what-if analysis the supplier with the lowest price was removed from all tenders in our dataset to see whether rank-reversal could have occurred. For the simulation the historic data on quality and the number of suppliers was used to derive data distributions which could be used as input. The simulation allowed changing the number of to be generated tenders, the weight for price and quality, whether a minimum quality threshold was applied and if the latter is the case how high this should be. Additionally, the number of suppliers in a tender and the standard deviation of the tender could be changed. Besides the two analyses to retrieve rank-reversal rates for different situations a literature study was conducted to find motives for using relative scoring methods. In addition the found advantages and disadvantages of the method where stated. Results: Relative scoring methods are mainly used due to its easiness and the fact that no predefined scoring tables are required. The what-if analysis showed rank-reversal would have happened in one out of fifty real world cases after removal of the supplier with the lowest bid in all analysed tenders. The simulation showed that the rank-reversal rate follows are parabolic pattern with rates converging to zero at low weights for either price or quality as can be seen in Figure A. The highest rate was 4.07%, with comparable values for price weights between forty and sixty percent. In most tenders the weight of the price criterion falls in this range. The appliance of a minimum quality threshold reduced the rank-reversal rate, but the peak still lied at 2.80%. Rank-reversal rates are close to zero towards the edges of graph. In these cases either quality or price is of high importance. At high price weights the tender procedure almost follows the lowest price rule. If the supplier with the lowest price is removed from the tender, the supplier with the second lowest price almost always wins the tender as barely any points can be scored on quality. A same line of reasoning is applicable for high quality weights. Therefore the rank-reversal converges toward zero at both ends of graph. When more suppliers participate in a tender the rank-reversal rate increases. An increase was also found by increasing the standard deviation of the bids leading to more variance between offers. Further research can focus on finding a more appropriate distribution for the submitted offers, the impact of entrance of supplier and whether other relative scoring methods yield comparable results and patterns. As the mathematically proven possibility of rank-reversal occurrence is confirmed by both of the conducted analyses, it is now up to contracting authorities to determine whether they want to change the rules and methods currently in place to evaluate suppliers in a tender. As there are strong options both in favour as against relative scoring, this may become a challenging process. |
| Item Type: | Essay (Bachelor) |
| Faculty: | BMS: Behavioural, Management and Social Sciences |
| Subject: | 50 technical science in general, 85 business administration, organizational science |
| Programme: | Industrial Engineering and Management BSc (56994) |
| Link to this item: | https://purl.utwente.nl/essays/74150 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page