Germany, Austria, Switzerland 2017

THE GREAT 2017 MOBILE NETWORK TEST IN GERMANY, AUSTRIA AND SWITZERLAND

Download mobile network test here

logo-p3-connect

Germany

Germany

Austria

Austria

Switzerland

Switzerland

At the end of 2016, we conducted our annual mobile network benchmark for the 23rd time. Together with our renowned benchmarking partner P3 communications, once again we have investigated, which mobile operators in Germany, Austria and Switzerland are a cut above the rest – with utmost effort and our objective, customer-oriented testing methodology.

routen_dach_englisch

Click to enlarge

The network operators in Germany, Austria and Switzerland wait for the results of our annual mobile network benchmark with the highest tension imaginable. As in the years before, we have conducted this benchmark as part of our well-proven co­operation with the Aachen-based network testing specialist P3 communications.

However, we have never before experienced such harsh disputes with some of the test candidates in the forefront of our test. They argued about questions like which smartphone models were to be used for the measurements, how the test routes should be put together or how we should balance the individual voice and data results. Eventually, all their reasoning could be traced back to one purpose: Some of the candidates hoped to gain advantages in those areas where they believed to be particularly strong and having the edge over their competitors.

We did show good sportsmanship interpreting the pressure built up by some of the candidates in the run-up of our test as proof for the high relevance and acceptance of our benchmarks within the whole industry.

Quality benchmark for many years

Traditionally, fairness and transparency play a very big role in our network test. In any case, with all decisions concerning methodology and scoring, our highest priority is always to ensure the significance of our benchmark for you, our readers. As a matter of principle, connect and P3 value objective and authoritative results about the actual quality and performance of the cellular networks second to none. Everybody interested can find an exact description of our methodology in chapter “Methodology”.

So, on the following pages you can read the answers to many thrilling questions: Has Deutsche Telekom been able to defend its top rank in Germany? How does O2 rank, after its owner Telefónica has started to combine this network‘s former radio cells with those of E-Plus, which they bought in 2014 (also see page 15). Which operators come out on top in Austria and Switzerland, where the contest is traditionally conducted on an especially high performance level?

Not all contestants will like all of the answers and results. This again proves to us that we have done a good job. Because this is the only way for us to guarantee that truly everybody can fully rely on our test results.

 

Germany Germany

7920551

Voice

The quality and reliability of voice connections represent 40 per cent of the final score. Which operator offers the best network in this respect?

In some users‘ communications habits, voice telephony only plays a minor role. But conventional phone calls are far from being outdated. Otherwise, the three German network operators would probably not have gone to the lengths of implementing VoLTE – telephony based on sending data packets over the LTE network. Therefore, quality and performance of voice telephony still played an important part in the drive tests and walk tests conducted by P3: For this purpose, both cars that were driving through 17 large and many smaller German ­­cities carried six Samsung smartphones each. They permanently called their counterparts in the other vehicle. In order to simulate the everyday smartphone use, the phones would constantly transfer data in the background ­during the ­telephony tests. An identical device configu­ra­tion was used in the backpacks carried by the test staff who walked around in city centres and public buildings conducting the walk tests. The devices had been configured to make sure that part of the connections would be established via VoLTE and another part would be transmitted via conventional circuit-switched telephony.

Distinct ranking order

The tests in the city centres ­already showed a clear picture: Both in the drive tests and in the walk tests, Deutsche Telekom is ahead. Vodafone follows at a distance of a few points. O2 is clearly defeated what can be seen in the table below by means of lower success ­ratios, longer call setup times and also a lower average speech quality.

de_tabelle_sprache2016_englisch

As a result of their test drives through smaller cities and on connecting roads, the P3 testing team found basically the same ranking order: Deutsche Telekom leads, Vodafone follows at a comparatively small distance, and O2 comes in last with a considerably larger gap. While Telefónicas network keeps up quite well in smaller towns, its distance to the leading two contenders grows larger on the connecting roads. On the whole, compared to last year‘s test, O2 improved in the voice category. For the voice calls ­examined by the testing staff in trains, even Telekom and ­Vodafone showed some weaknesses. But O2 scores again far behind them. Regarding some measurement values like call setup times and speech quality, Vodafone is ­narrowly ahead. But while the Düsseldorf-based operator could claim a stage win in the voice category last year, in the 2017 test this title goes to Telekom.

de_kategorievergleich2016_englisch

 

Data

Data tests account for 60 per cent of the final score. Who delivers the best performance in this category?

Data communication is the most prestigious category in connect‘s network test. Firstly, the results of these tests represent 60 per cent of the final score. And secondly, the test parcours to be completed by our candidates incorporates a large number of practice-oriented applications. For instance, the smartphones frequently ­access the most popular web sites according to the renowned Alexa ranking, as well as the static „ETSI reference web ­page“ also known as „Kepler page“. Measuring the speed and reliabilty of data transfers is the aim of our upload and download tests. We monitor uploads with test files sized 3 MB and downloads with 1 MB files. Additionally, we verify which amount of data travels over the network within ten seconds.

Another scope of our testing are Youtube videos. The popular ­video platform does not distinguish between standard definition (SD) and high definition (HD) resolutions any more. It rather dynamically adapts the video resolution to the bandwidth that is currently available. In order to respect this new strategy in our tests, we examined the success ratio of video playbacks, the start times, the percentage of playouts that took place without interruptions as well as the videos‘ average resolution or number of lines respectively. Both P3 test cars checked these indicators as part of their drivetest, and also the walk test teams had the same agenda. All data measurements in Germany were executed with the LTE Cat.6 smartphone Samsung Galaxy Note 4.

Strong Deutsche Telekom

In big cities, the results were similar to the voice category: Again, a very strong Telekom takes the lead, Vodafone follows with good results, and O2 is clearly defeated. Separate analyses show that Vodafone could definitely improve over last year‘s results in the data ­category. To some extent, this supports the Düsseldorf operator counterbalancing its shortfall in the data score. Also, Vodafone achieves especially good results at Youtube playback – in this ­category, the Düsseldorfers ­partly draw level with Telekom. Still, this is not sufficient to grant Vodafone a partial victory. The reason is that Vodafone scores a little worse than Telekom in the discipline of web page access and – slightly less distinctive – regarding file uploads and ­downloads.

However, when we look at O2, their deficits are even more ob­vious. While the inner city walk tests show a success ratio of 99.3 per cent for web page access in the Telekom network and 98.5 percent at ­Vodafone, this value drops to 91.3 per cent in Telefónica‘s network. Statistically, this means that ­almost one out of ten web page views will fail when O2 customers are walking through city centres.

When observing indicators like the success ratios or average speeds of file downloads in the big city drive and walk tests, the results show the same trend that looks increasingly familiar this year: Telekom leads, Voda­fone follows at not too big a distance, and O2 clearly comes in last. For example, according to our walk tests, file downloads run at average data rates of more than 13578 kbps in 90 per cent of the cases in the Telekom network. In the Vodafone network, it is still more than 6801 kps, and O2 achieves only a minimum of 1568 kbps. So the latter candi­date accomplishes not much ­more than a tenth of the speed offered by test winner Telekom.

de_tabelle_datencitiesdrivewalk2016_englisch

Drive tests in smaller towns

The drive tests conducted in small towns gave equivalent results: Again, Telekom achieves the best measurement values, Vodafone follows at a distinct but not huge distance – and Telefónica once again comes in last. Similar to the inner city drive tests, the success ratios of web page access via the O2 network are worse than in the networks of the leading two providers, but still better than O2‘s walk test results. Our download and upload tests show comparable results.

Once more, Vodafone turns out to be a Youtube star in smaller towns. Its top performance in this category is at the same level as test winner Telekom. Both operators seem to have a very performant “peering“ to the content delivery network of Google‘s video platform. O2‘s weak overall scores can be explained to some extent with the ongoing integration of the former E-Plus network and the distortions coming along with it. While O2 at least improved in the voice category, the 2017 network test indicates a stagnation for this operator in the data category.

 

Connecting Roads

Modern cars rely heavily on connectivity. How does this actually work out on German roads?

7920555It was approximately 6600 kilometres that P3‘s two test vehicles covered this year on German connecting roads – on top of the 5500 kilometres that each car covered driving through large and smaller cities. The point of this exercise: Gaining closer insights about the quality and reliability of the mobile networks on this particular type of roads.

Distinct ranking order on the roads

While Telekom and Vodafone ­were almost at level in the voice tests conducted on connecting roads, their offset becomes more obvious in the data cate­gory. ­Especially regarding the success ratios of web page access as well as downloads and uploads, Telekom clearly ranks first and keeps its competitor from Düsseldorf at a distance. Still, this match takes place at a very high level when looking at the distance of O2. Similar to the large and small cities before, Telefónica also loses valuable points on the connecting roads and thus falls back further behind the leading two German operators. The test results in this category are quite obvious: Car drivers who need robust data connections on the road – whether for navig­ation, for communication or mobile entertainment purposes – currently cannot pass the ­mobile networks of Deutsche ­Telekom or Vodafone.

de_tabelle_datenroadsdrive2016_englisch

 

Data on Railways

Railways used to be the blind spot of German mobile network operators. Is this still true this year?

The testing staff spent about 33 hours on 15 different ICE trains during their railway tests conducted in 2016. Yet, the measurements did not only take place in these flagship trains of Deutsche Bahn, but also considered regional railway connections. The test results from German trains should not surprise anybody who has read the outcome of the other categories: In the trains, Deutsche Telekom once again leads the pack, Vodafone scores second with viable results, and O2 brings up the rear.

Especially when comparing the partial results obtained on the railways, looking over the borders – specifically at the results of the respective tests in Austria and Switzerland – may make German railway customers quite envious. Both alpine countries are considerably ahead in this respect. And even Telekom, which scores best in the measurements taken in German railways, cannot measure up with the results of the Austrian and Swiss operators – by far. Deutsche Bahn has recently started an initiative to enhance connectivity especially in its ICE trains in close cooperation with the German mobile network operators. But this does not seem to have much ­impact on this year‘s mobile network test.

Much need for improvement

When looking at the details, there are many similarities between the railways and the connecting roads: While Telekom and Vodafone were almost at level regarding voice phone calls in trains, their offset grows in the data category. Here, Telekom offers the best results – but still shows a lot of room for improvement. Vodafone‘s results are mid-level, and O2 once again comes in last. ­Success ratios, like those for web surfing, of around 85 per cent at Telekom, about 77 per cent at Vodafone and approximately 63 per cent at O2 ­convey a clear message: When it comes to connectivity in German railways, there remains a lot of work to be done.

de_tabelle_datenbahnwalktest2016_englisch

 

Single review

de_total-score2016_englischTelekom

logo_telekomTelekom wins this year‘s ­network test at a distinct distance to runner-up Vodafone. Both in the voice and data ­categories, the Bonn-based operator turns out to be the strongest conveyor. Although we have raised our requirements, last year‘s winner Telekom not only managed to defend its position, but actually continued to improve its score. Therefore Telekom absolutely deserves the first place in Germany – and this for the sixth time in a row.

Vodafone

logo_vodafoneVodafone also improved both in the voice and data categories compared to last year‘s results. But still Telekom outperformed their Düsseldorf-based competitor in this year‘s voice tests. Anyway, the 2017 network test emphasizes Vodafone‘s clear ­improvements in the data scores. In this context, Vodafone‘s excellent Youtube results are particularly eye-catching. All in all, these ­results entail a second rank with a good overall score.

O2

logo_o2Compared to the results of last year, both the networks of O2 and E-Plus managed to improve. This is particularly underlined by the results of the voice measurements. A valid explanation for O2 not scoring any better may be the problems caused by the ongoing integration of both networks. So we hope in the best interest of O2‘s customers that this integration will continue to foster noticeable improvements.

 

 

 

Austria Austria

route_at_englisch

In comparison to the previous year, all Austrian operators improved – on an already very high level. Who wins the race in the alpine republic this time?

Since connect‘s network test has been including the alpine countries, Austrian network operators used to have a neck-and-neck contest on the highest level. Compared to the other countries in the DACH region, the Austrian contenders regularly are among the top tiers. No network operator from the alpine republic ever scored worse than the grade “good“.

Austrian customers have every reason to be happy, as they can choose from three very good providers. And this at considerably lower costs than for example in Germany. The Austrian network operators also look pretty good when it comes to the roll-out of LTE. In autumn 2016, A1, T-Mobile Austria and Drei (the Austrian subsidiary of Hutchison Three) already offered 4G to a large part of the Austrian population and could focus on filling the few remaining gaps. So, we highly anticipated the results of the measurements that P3 took in eleven larger Austrian cities, on approximately 2700 kilometres of connecting roads as well as in Austrian railways.

Voice connections

at_tabelle_sprache2016_englischAssessing voice telephony, A1 scores first. Especially in the drive tests – the test calls made from car to car – that P3 took in larger cities, this operator performed a tiny bit better than its competitors.

At the time of testing, A1 was the only Austrian operator who already supported VoLTE. This may have helped particularly in the inner cities with their good LTE coverage. But the high score that A1 achieved in this category, would not have been possible had this operator not also performed ­exceptionally well in its conventional telephony service.

In the walk tests that our testing teams conducted in city centres and public buildings, all three candidates scored almost equally on a very high level. Last year‘s winner is slightly ahead in the discipline of voice connections in rural ­regions, namely in smaller towns and on connecting roads. But even there, the overall difference between the three networks is only minuscule.

at_kategorievergleich2016_englisch

 

 

 

 

 

 

 

 

 

Data communication

When we look at the tests of data connections, the results are very similar. Here again, A1 shows a slight advance in ­larger cities with Drei following closely. For web page access tests, T-Mobile Austria falls a fraction behind – but still offers top results that might be totally sufficient for a test win in some other countries. All in all, the success ratios, session times and data throughputs that can be seen in the adjacent tables are fantastic values.

In the inner city walk tests, Drei becomes the frontrunner by a very thin margin. But here again, the two other providers follow at a distance of only a few points. As we could already observe in the voice tests, Drei scores slightly better than both of its competitors in small towns and on the connecting roads. But, as in the other disciplines before, the contest takes place at a very high level.

 

at_tabelle_datencitiesdrivewalk2016_englischat_tabelle_datenroadsdrive2016_englisch

Mobile communications on Austrian railways

When they talk on the phone or surf the web in trains, Austrian customers have once more good reason to be pleased. The measurement values that P3‘s teams gathered on hundreds of railway kilometres, certify very good results for the three Austrian operators – even if ­indicators like success ratios or data rates somewhat drop in this category when for instance compared to those obtained on the connecting roads.

A1 scores best for conveying voice calls in trains, while Drei turns out to be the data champion. Regarding voice telephony in trains, T-Mobile falls slightly behind, while this is not true for data communications – where this operator offers absolutely no cause for complaint. On the whole, in the railway category Austria scores somewhat behind Switzerland, but is clearly ahead of the results from Germany.

at_tabelle_datenbahnwalktest2016_englisch

 

Single review

at_total-score2016_englischA1

logo_a1It was well worth the effort: In this year‘s connect network test, A1 takes back the gold medal from last year‘s winner Drei. Particularly good voice results make A1 the overall winner in the Alpine Republic. But equally in terms of data communications and connectivity in railways, the A1 network scores very strong. Moreover, with a total score of 918 out of a possible maximum of 1000 A1 is also a top tier when comparing the results from all three involved countries.

Drei

logo_dreiDrei was also ­able to improve on last year‘s results on the whole. But it is by a very close margin of only three points, that the Hutchison-owned provider makes second place. Drei scores better than its competitors particularly in smaller towns and on connecting roads. ­Examining mobile connectivity in trains, the overall winner A1 and Drei are basically on par. And bear in mind that a second place in Austria would be equivalent to a test win in many other countries.

T-Mobile

logo_telekomCompared to last year, T-Mobile Austria took the biggest step forward. The actual reason that this contender still scores razorthin behind its competitors for most of the in­dicators is the extreme strength of all Austrian providers. Still, T-Mobile Austria turns out to be in excellent shape. Its total result would have made this provider a strong number two in Germany and would actually come quite close to the performance of the parent company based there.

 

 

SwitzerlandSwitzerland

route_ch_englisch

In spite of its strong competitors, Swisscom habitually ranked first in Switzerland. However, this year things are a little different.

In the connect network test, the Swiss operators have had sur­prises in store time and again. Traditionally, the bar is set extremely high in Switzerland – quite often all three network providers achieved the grade “very good“. Of course we have raised our requirements once again this year – and still the present test winner succeeded in climbing just a little over the 950 point mark within our 1000 point scoring scheme. Thus, for the first time in the network test for Germany, Austria and Switzerland, we have to award the grade “outstanding“. This is short of a sensation and it goes along with a surprising change at the top. But the other two Helvetian candidates still are graded “very good“ – and their achievements are far from being close calls. But one thing after another.

Voice connections

ch_tabelle_sprache2016_englischVery much the same as in the other countries, the testing staff of P3 examined the quality and stability of voice connections in Switzerland by conducting drive tests and walk tests. During these tests, Sunrise quickly turned out to be ahead in most of the examined scenarios. This is quite clear for phone calls out of cars in larger cities. But in more rural areas (smaller towns as well as on the tested connecting roads), the gap between Sunrise and Swisscom shrinks down to one or two points.

In the results of our walk tests conducted in large Swiss cities, the competitors Sunrise and Swisscom are actually on par. Salt keeps some distance to the two leading contestants, but still achieves very good results. While Sunrise and Swisscom improved over last year‘s results, the third Swiss operator that was formerly known as “Orange“ more or less stays at the same level than last year.

Up to now, Swisscom is the only operator in Switzerland who supports VoLTE. This modern voice standard that was applied for a part of the test calls, contributes to Swisscom‘s excellent results in the voice category. However, in the final scoring Sunrise stays still close ahead.

ch_kategorievergleich2016_englisch

Data communications

The standings and tendencies that we could observe in the voice category repeat themselves almost identically in the data discipline. Sunrise accounts for the biggest improvement over last year‘s results in this category. This is quite obvious in the larger cities where Sunrise‘s lead over the also very strong Swisscom is a little more pronounced in the drive tests than in the walk tests.

For example, Sunrise achieves impressive success rations of 100 per cent for file downloads or Youtube playbacks in the cars. In contrast, in the smaller towns and on connecting roads, the leading duo is almost on par – once again on a very high level.

In the data measurements, Salt has to settle for the third rank once more – but this again only means that Sunrise and Swisscom were able to improve over last year‘s results, while Salt kept its still strong performance stable. In this context, we should bear in mind that Salt attacks its two strong competitors with a particularly aggressive pricing. This makes Salt‘s offerings all the more interesting. Above all, Salt‘s results are definitely top notch when compared to those from other countries.

ch_tabelle_datencitiesdrivewalk2016_englischch_tabelle_datenroadsdrive2016_englisch

 

Mobile connectivity in Swiss railways

The tendencies observed in the voice and data measurements conducted in larger cities and smaller towns as well as on the connecting roads, prevail for the tests of phone calls and data connectivity in Swiss railways too. Again, in this category Swiss customers have every reason to be happy. Their operators achieve the best results within the three countries at a distinct ­distance. Indeed all three Swiss operators achieve a remarkably high performance and reliability in the challenging task of providing connectivity to moving trains in the demanding Helvetic topology.

When looking at the detailed results in railway tests, Sunrise is slightly ahead in the voice category, while Swisscom scores a tad higher in the data measurements. Salt again comes in third regarding both voice and data communications while still showing very good results.

ch_tabelle_datenbahnwalktest

 

Single review

ch_total-score2016_englischSunrise

logo_sunriseBoth in the voice and data measurements, Sunrise achieves excellent results. Compared to the previous year, the operator improved in both categories. Due to a distinct gain in points in data communications, Sunrise brings in the overall win. Interestingly, the gap to last year‘s winner is quite wide – although Swisscom achieved very good results this time as well. For the first time in the connect mobile net­work test for Germany, Austria and Switzerland we have to award the grade “outstanding“.

Swisscom

logo_swisscomThe test results clearly show that last year‘s winner Swisscom has improved as well during the last twelve months. However, in the final scoring Swisscom was beaten by the second largest contender Sunrise. Nevertheless, Swisscom‘s customers can rest assured because with its excellent test results this Swiss operator would still immediately lead the field in Germany or Austria.

Salt

logo_saltThe smallest Helvetic mobile network provider achieves stable – and very good – overall results, both in the voice and the data categories. As they basically ­remain at the same level than in the previous year, Salt comes in third in the overall Swiss ranking. But when we compare the results of all three countries, Salt is still a top tier. Moreover, Salt attacks both of its competitors quite successfully with its aggressive pricing.

 

Methodology

7920529As in previous years, connect’s partner for the network measurements, P3 communications, used two vehicles to test drive the chosen cities, towns and roads. In Germany and Austria each car carried six Samsung Galaxy S5 smartphones to measure voice services and three Samsung Galaxy Note 4 performing the data service tests. In order to reflect the advanced roll-out of LTE with “3 Carrier Aggregation“ (the combination of three carrier frequencies) in Switzerland, we used three Samsung Galaxy S7 for the data measurements there. The same setup of devices was utilized in the walk tests. For this effort, the smartphones were installed in trolleys and backpacks with additional batteries. The devices’ firmware was each operator’s current firmware version. If such software was not available the most current firmware from Samsung was used.

Voice telephony

Voice services were measured with the smartphones performing calls alternating between the two measurement cars (“mobile-to-mobile“). An additional car served as a mobile remote station for the calls of the walk test teams. Background data traffic was transmitted by one of the smartphones simultaneously to each call to reflect a realistic usage scenario. Audio quality was assessed by using POLQA (Perceptual Objective Listening Quality Assessment) wide band scoring. All devices were configured in “LTE preferred” mode. Thus in the three German Networks as well as with A1 in Austria and Swisscom in Switzerland, the modern Voice-over-LTE (VoLTE) service could be used. Within networks not yet supporting ­VoLTE, the smartphones were forced to switch to 3G or 2G technology, the so-called circuit- switched-fall-back (CSFB).

Data connectivity

Each box was housing four smartphones which allowed the simultaneous testing of four mobile operators.

Each box was housing four smartphones which
allowed the simultaneous testing of four mobile
operators.

To assess cellular data performance a sequence of tests were executed. As a dynamic web-browsing test, each country’s top web sites (according to the Alexa ranking) were downloaded in the so-called live web-browsing test. Additionally a static web site was tested, the industry standard ETSI (European Telecommunications Standards Institute) “Kepler“ reference page. HTTP downloads and uploads were performed with 3 MB and 1 MB files, simulating small file transfers. The networks’ peak performance was tested with a ten second download and upload of a single, very large file.

The Youtube measurements considered the new “adaptive resolution“ feature of this video platform. In order to offer a persistent video experience, Youtube adapts the video streams‘ resolution dynamically to the bandwidth that is currently available. Our scoring therefore considers the success ratio, the time until the playback starts, the percen­tage of video playouts that take place without interruptions as well as the videos‘ average resolution or line number count respectively.

Indoor and train measurements

The walk tests consisted of the same tasks as were performed in the cars. For this effort two teams measured in public transport and in public places, like coffee shops, museums, train stations and airport terminals. Travelling from city to city by public transport allowed the assessment of cellular network quality within the long distance trains.

Logistics

The tests were performed in ­Austria, Germany and Switzerland around the same period of time (Germany: October 21 – ­November 12; Austria: October 7 – 27; Switzerland: October 14 – November 1). All measurements were done between 8 AM and 10 PM. Both cars were always in the same cities, but on different routes to avoid any interference of one car’s measurement by the other car’s. Both vehicles followed a given route, including fixed location measurements at “areas of interest” such as well-visited public places. Measurements there lasted one hour. Locations such as train stations, airports, much-frequented public parks or high-density urban areas typically demonstrate how networks respond when a high number of users compete for their share of bandwidth within the network’s available ­radio frequencies.

Three boxes were mounted into the back and side windows of each measuring car in order to support twelve smartphones per car.

Three boxes were mounted into the back and side windows of each measuring car in order to support twelve smartphones per car.

The measurements included 17 larger cities and 26 smaller towns in Germany, while the walk tests frequented six cities. In Austria the drive tests covered 11 big cities and 20 smaller towns, the walk test team visited five cities. In Switzerland, the test route included 13 big cities and 20 smaller towns with the walk tests conducted in four cities. Travel between the cities mainly used highways, but smaller state and county roads were driven as well. For each connect test P3 communications follows a well-defined process to generate four independent and representative city and route plans. The connect editors choose randomly one of these four alternatives.

Test efforts and results

Overall 25,000 km were driven for the connect P3 mobile network test in 2016. In Germany the approximately 12,100 km of driven routes alongside the cities and areas visited represent 13.4 million inhabitants, equaling around 16.7 per cent of Germany’s population. Austria was measured by driving 5,900 km covering about 3 million inhabitants (approx. 36 per cent of the Austrian population). In Switzerland, the test teams drove approx. 7,000 km, covering 1.9 million people representing around 22.5% of the Swiss population. Certainly a huge effort, but necessary to gain the required statistical relevance and confidence in the test results.

Scoring

The results of the voice test contribute 40 per cent of the total score, those of the data tests make up 60 per cent. For the overall result we apply a 1000 point scheme in order to represent sufficiently detailed results. Moreover this scheme allows us to better compare the results of network tests that we have conducted in different countries.

data_voice_donuts2016_englisch

 

O2 and E-plus

Here are the reasons why we tested and evaluated the merging networks of E-Plus and O2 as a single O2 network.

o2_und_e-plusAfter Telefónica/O2 bought out its former competitor E-Plus in October 2014, the merger of both networks goes at full speed. Previous E-Plus customers are being transferred to O2 tariffs, and Telefónica must sell off some base stations that have been occupied by both operators according to the German regulatory authority. The remaining base station sites are already designated “O2“. Cells formerly belonging to E-Plus are no longer visible as a discrete mobile network. Instead, at the moment there are “old“ O2/E-Plus cells along with “new“ ones.

Given this situation, connect and P3 decided to only examine O2 in their network test that we con­ducted in late 2016. As we know from our readers and from our own experiences, difficulties definitely occur in the course of the network merge. These problems that include failing handovers between two differently configured network cells, are clearly recognized in the results of this year‘s P3 connect mobile network test.

 

Fairness and transparency

This year, some of the candidates massively tried to influence the conditions and parameters in the run-up of our test. The connect and P3 staff responsible for the testing project have of course fended off these attempts.

As in previous years, connect and P3 met in early 2016 in order to define the conditions and parameters for this year‘s network test.  In this preceding test design phase, we for example identify new test criteria, discard or confirm old ones and determine their influence on the overall score. We define the timeframe as well as a preselection of smartphone models that we intend to use for the measurements. We then communicate these preliminary ­definitions in advance to the CTOs of the network operators.

Feedback is appreciated

In this process we appreciate feedback about aspects like suitable tariffs that facilitate unobstructed measurements of the best performance possible. After all, our objective is to evaluate the network experience of the most demanding customers. We also agree on the firmware versions used in the measurement smartphones, as each mobile network operator makes ­adjustments to most popular devices to ­ensure a smooth interplay with their network. But this time, some contenders apparently took part in the discussions with the single intent to enforce measurement conditions that would favour their own network. For ­example, there have been attempts to im­pose a smartphone model on us that all in all works less reliably than others – presumably because the involved provider expected an ­advantage for its own network from this. One operator insinuated flaws in the test design more than once – until extensive measurements conducted both by P3 communications as well as by the connect test lab disproved all of them. Permanent changes in the reasonings of some operators led connect to the assumption that one or the other of them would not have minded blowing the rapidly approaching deadline of our test.

The more danger, the more honour

We cannot help but understand such attempts as a compliment for the high relevance that the operators assign to our test. And of course we remain true to ourselves concerning these issues. After all, it is our standard to conduct a test that provides deep insights into the quality and performance of the examined mobile networks.

However, we will draw one conclusion from this year‘s experience: In the future, we will publish obvious attempts to abuse our transparent approach to testing the very same way as we document our test procedures.

 

Historical Development

Looking back at the results of the connect network tests since 2010 provides especially one insight: Despite of the constantly rising requirements, the level of the overall results has steadily improved. We reckon that our demanding and well renowned network test is not entirely blameless.

historie_dach

Click to enlarge

Customers‘ expectations are constantly growing – expanding data volumes and rising transmission speeds are regarded to be absolutely normal. P3 and connect take account of this development by con­stantly raising the requirements and thresholds of our tests.

Network test as a driving force

The adjoining glance at the development of results in Germany, Austria and Switzerland in recent years shows a clear overall tendency: Despite the growing requirements, all tested networks improved steadily. In all modesty, we believe that the high relevance and challenging demands of our annual network tests are an important driving force of this development.

 

Conclusion

Hannes Rügheimer, connect author

Hannes Rügheimer Portrait_v2The operators enthusiastically fight for the top rank in the connect network test. The fact that almost all candidates managed to improve in spite of the rising requirements is clearly supporting our claim that our critical tests contribute to the overall enhancement of the mobile networks’ quality.

Against this background, the ­repeated test victory of Deutsche Telekom in Germany was by no means self-evident. It rather reflects the considerable efforts that Telekom takes in order to maintain and extend its network. Vodafone also worked flat out, but remains on the second rank. O2’s result shows some room for development but can be explained by the ongoing integration with the former E-Plus network.

In the alpine countries, there were rigorous fights as well. This led to a change at the top ranks in both countries. In Austria, A1 managed to gain back the crown from last year’s winner Drei. Particularly its very strong voice results secured the win to A1. Although, also Drei noticeably improved over last year, the Hutchison-owned company fell back behind A1 at a very narrow gap. Compared to the previous year, T-Mobile Austria made the biggest step forward, but still was not able to pass its two extremely strong competitors.

In Switzerland we also see some movement at the top. Especially with its distinct rise in points in the data category, Sunrise manages to outplay last year’s winner Swisscom. Thus, for the first time a candidate of the connect network test in Germany, Austria and Switzerland is awarded the grade “outstanding“. And even though Swisscom only ranks second this time, the company has still improved considerably compared to the previous year. Salt takes the third rank of the Helvetic network providers but still achieved very good results.

The top quality of mobile connectivity on Swiss and also on Austrian railways was especially noteworthy. This is something that German railway customers can only dream about.

dach_total_scores

dach_final_results