University of Twente Student Theses


Moving forward with Regiotaxi : a benchmark of performance and an evaluation of several DRT services in the Netherlands

Buysse, R.P.C. (2014) Moving forward with Regiotaxi : a benchmark of performance and an evaluation of several DRT services in the Netherlands.

[img] PDF
Abstract:This study is on the performance of Regiotaxi in the Netherlands. Eleven Regiotaxi services were benchmarked on a set of indicators and subsequently evaluated. Also, the transport authorities were asked to fill out a survey on their policy priorities and the relation between their performance and their policy priorities was studied. The study was done for mobility consultancy Goudappel Coffeng. They had customer satisfaction surveys for different Regiotaxi services available and they wanted to benchmark the surveys and get an insight on customer satisfaction across regions. This was expanded to a full evaluation of in total 11 Regiotaxi services. Beside the customer satisfaction surveys, trip databases and management reports were also used for the evaluation. The main objective of the study was: To assess the performance of Regiotaxi in the Netherlands, by (a) benchmarking performance indicators and finding internal relations between them, (b) evaluating different Regiotaxi systems and (c) determining the effectiveness of policy, given the policy objectives of the transport authorities. Regiotaxi is a Dutch demand responsive transport (DRT) system for Wmo-indicated (Wmo is the law on social support) and it can be combined with other functions, such as public, school or work transport. A literature study was done for DRT and the evaluation of DRT. This literature study revealed that research has mostly focused on the financial effectiveness of DRT, for example, the effective scheduling of vehicles and cost structures. Therefore it was chosen to focus on evaluating performance in a broader sense, using the trip databases. An evaluation framework from Andrade (2008) was adapted to do so. In order to get context for the study and to validate the adapted evaluation framework, interviews were conducted with a policy advisor, a policy maker, a director of a transportation company and a representative of public transport travelers. The evaluation framework uses two levels of objectives to generate a single total score. The higher level objectives are perception, performance and economic durability. In the lower level these objectives are split into lower level objectives. These lower level objectives are described by indicators. For example, one of the lower level objectives describing perception is perceived safety. Perceived safety is then scored using the satisfaction about safety and driving style that were taken from the customer satisfaction survey. The framework combines two multi-criteria decision algorithms to generate a final score: the Analytical Hierarchy Process (AHP) and the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS). AHP was used to set weights for the objectives on the different hierarchy levels. In the study policy priorities were used to weight the objectives and AHP made the setting of priorities less complex for the transport authority by reducing it to a series of one-on-one comparisons. The transport authority had to indicate to what extent one or the other objective had a higher priority. The result matrix could then be converted to a set of weights for the compared objectives. Another advantage of AHP is that human errors can be processed and quantified, i.e. the answers do not have to be perfectly consistent and an . This was preferred since for comparing multiple objectives it would be very difficult to be perfectly consistent. So instead of forcing perfect consistency, some small amount of errors was allowed, this was quantified with a consistency ratio. When the consistency ratio would be too high, it would be requested the answers were revisited to make them more consistent. In the end all surveys were sufficiently consistent. In general, on the higher level hierarchy perception was deemed most important. For perception, perceived safety was most important and perceived comfort least important. For performance, reliability was by far considered most important. TOPSIS was chosen because it allowed for a single service to be evaluated, without having to resort to relative ranking of services. This is done by independently setting an ideal and anti-ideal situation and determining how close the service is to both. The ideal and anti-ideal situation would be determined by experts, however, this did not work out in this study as it was advised by experts to use the data available instead of expert opinion as it was expected this would give better values. As a consequence, since the ideal and anti-ideal values were based on the scores of the participating services, the result was still a relative ranking. It was still chosen to continue with TOPSIS since there were no other disadvantages and the ideal and anti-ideal solutions generated in this study can be used to perform future Regiotaxi evaluations in the intended way. In order to benchmark and evaluate the services, monitoring data had to be made comparable; this was one of the issues tackled in this study. The biggest problem was the variation in questions and survey scales used in the customer satisfaction surveys. Different conversion methods were considered to convert all survey scales – different 4- and 5-point scales – to a 10-point scale. In the end two methods were chosen: For converting the 5-point scales to a 10-point scale, scale profiles were matched. For both 5-point and 10-point scales a profile of usage was made, based on the data available from the customer satisfaction surveys. The profiles were then coupled, for example, if 15% of the answers were the first point on the 5-point scale, the 15% of lowest answers on the 10-point scale were averaged to generate an equivalent scale value for 1 on the 5-point scale for the 10-point scale. For converting the 4-point scales this was not possible, because nearly all questions with a 4-point scale were from the same survey, and therefore not enough variation was available to make a robust scale profile. The 4-point scales were therefore converted using anchored extremities, i.e. 1 on the 4-point scale was 1 on the 10-point scale, 4 was converted to 10, intermediate values were rescaled as if the distance between scale points was equal. Besides the customer satisfaction surveys, also the data from the trip databases and management reports had to be made comparable. For several indicators a value per kilometer was used, however, some services work with a zone system and the services that register kilometers use different routing programs to generate calculate the trip distance. Therefore, since the kilometers came up for several indicators and the data were so diverse, reference trip distances and travel times were generated using the origin and destination from the trip database in Google maps. The transport authorities delivered subsidy numbers in very different formats. From the different formats a subsidy per passenger kilometer was distilled. After all the indicator values were benchmarked, they could be used for the evaluation. The evaluation showed that no system scores bad or good across the board, so bad scores in one area are compensated in other areas. This suggests that there is room for the regions to improve by learning from each other. The most important conclusions from correlating the benchmarks and scores were:  Tariff and subsidy were positively correlated, meaning that systems with a higher tariff also had higher subsidies.  Economic durability (the extent to which the system is durable and can go without subsidy) has only a weak correlation with performance and perception. This means that freeing up more money for the system does not necessarily lead to a better performing system.  Travel time performance and travel time perception do not correlate.  Tariff and perception and negatively correlated, suggesting that people are more than willing to pay for the service they’re receiving.  Perceived safety and perceived comfort strongly correlate. This result combined with remarks made in the interviews bring up the question whether travelers are able to correctly estimate their safety and that when surveyed on this subject, they are not actually reporting perceived comfort.  The average trip distance is negatively correlated with trips per inhabitant. There are several possible explanations for this and it is recommended to get a better understanding of this relation, as it may be relevant for the travel budgets Wmo-travelers receive.  Travel time satisfaction is the indicator that is correlated the best with overall satisfaction, so it is an important policy focus. However, since actual travel time performance does not correlate with travel time satisfaction, a better understanding of what makes up travel time satisfaction is needed. The results of the assessment of the effectiveness of policy showed that the priorities for policy objectives were not reflected in the performance level of the policy objectives. Two possible causes for this were hypothesized: a poor translation from policy objectives to requirements for the transport operator or a lack of monitoring of the requirements for the transport operator. The process and outcomes of the study led to several recommendations, both for the practical organization of Regiotaxi as for future research. The main recommendation for practice is to start sharing experiences and data in a regular structural manner. In order to do this, data will have to be made more comparable, as both the data from the customer satisfaction surveys as the monitoring data that is gathered is very diverse at the moment. Uniformity can be achieved by creating a guideline, such as MIPOV, which is a guideline for monitoring data for conventional public transport. When more comparable data are available, sharing data and experiences could be done at a national platform for Regiotaxi systems. This will contribute to improving the relation between policy objectives and actual performance. The evaluation framework used in this study can be used for future evaluations, not only for a comparison, but also to evaluate a single system. Another recommendation to improve the relation between policy objectives and actual performance is for the transport authorities to reconsider how policy objectives lead to performance. Specifically the way the tender is set up and how the system is monitored should be examined. A current topic for DRT in the Netherlands is budget cuts by the government. Two remarks are made about this subject. Firstly, although the results show that in some cases subsidy can potentially be reduced without affecting the performance, it is advised to be careful with cutting budgets. Problems may spread to other systems, for example conventional public transport. Secondly, budget cuts might be realized by reorganizing DRT systems in the Netherlands. At the moment, the division of DRT systems is based on how they are funded. However, there is potentially room to improve efficiency by basing the division on technical requirements. It should be researched whether the efficiency gains are higher than the increased costs of additional bureaucracy for redirecting funding streams. As for future research, it is recommended to look into the causes of some of the relations found in this research. Also, it would be very useful to develop a valid rescaling method for customer satisfaction survey data. Finally, this research the weighting of objectives was done based on policy priorities, which leads to an evaluation score from the perspective of the transport authority. The policy priorities survey which was done to generate the objectives weights could also be set up for travelers or transportation companies to evaluate the system performance from their perspective.
Item Type:Essay (Master)
Faculty:ET: Engineering Technology
Subject:55 traffic technology, transport technology
Programme:Civil Engineering and Management MSc (60026)
Link to this item:
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page