Switzerland 2014

THE GREAT MOBILE NETWORK TEST IN SWITZERLANDschweiz_flagge_2

Download mobile network test here

 

Conducted since 1993, the connect mobile network test with its neutral, customer-oriented testing has become an industry standard by which big corporations choose their operator. Who’s leading in Germany, Austria and Switzerland in 2014?

 

As Rudolf Augstein, founder of Germanys biggest news magazine „Der Spiegel“, said as guidance to young journalists: „Just being straight“. This motto should also hold true for a mobile network test, but it is not always so.
One German magazine simply focuses on easy to sell sensations and presents their benchmark as “Germany’s hardest network test” – despite the fact that two of the four German operators ended up third for the second year in a row and with minimal distance to the winner.
Another big publication completely lost track of their benchmark data, rating an operator with less than 3% 4G market share higher than one with more than 30% of the LTE market but slightly less throughput.

The proven Standard – since 1993

Connect refrains from superlatives and sends the proven test team of P3 communications on a trip through Germany, Austria and Switzerland. Two measurement cars per country, equipped with two measurement smartphones per operator, analyzing voice and data services in big cities, small cities and on the connecting highways.
In addition, two walk test systems per country, one for voice and one for data, are used for the same tests as in the measurement cars, with focus on indoor locations and railway connections. A third measurement car is used as counterpart station for the walk test voice system. The detailed methodology description can be found at the end of this page. The results of the German tests can be found on the following pages. How the Austrian and Swiss operators perform this year, and whether they outperform the Germans again, can be found in the menue point “Network Tests”.

 

 

SWITZERLAND

No Mobile Network Operator managed to win more times in a row than market-leading Swisscom. Are they also successful in 2014?
The online-portion of wellknown “Neue Züricher Zeitung” recently called Swisscom the “Sun King”. This not only related to their biggest market share in post-paid contracts of 63.8% in the first half of 2014, but also to the biggest share of new contracts (more than 50% in the same period). The other two operators Sunrise and Orange have very similar shares with 19.1 and 17.1% postpaid contracts respectively.
It is interesting to note that while data tariffs in Germany apply Fair Use Policies with price-dependent monthly limits of included data volume and subsequent throttling of data speeds, Swisscom takes a different approach. In their Infinity tariffs, the maximum high-speed data volume is not limited and not dependent on the monthly fee paid, but only the maximum speed offered is. Hence, no customer has to worry about data consumption, hence further increasing the traffic in the network, which is already highly with the large number of customers. Having access to unlimited data volume for a fixed price seems to appeal to many Swiss customers, since Orange has also offered an unlimited data plan for a while now.

Telephony

Before testing the mobile Internet, telephony comes under scrutiny. All three Network Operators showed success ratios around 99%, however Sunrise is slightly out in front. Swisscom and Sunrise tie on speech quality while Orange trails them by only 0,15 points on the MOS scale (range 1-5) still showing a very high quality level. Looking at the Call Setup Time, Sunrise scores the best value with 4,4 to 4,5 seconds. No other network with a similarly high share of LTE has ever connected calls faster than that.
On the transfer roads between cities, the reliability naturally goes down, but stays at a remarkably high level for all. One reason for this is that Orange has managed to improve by 6.5% compared to last year. The quality of calls made on trains is even more impressive. It may not come as a surprise that even in a country with extremely high railway usage such as Switzerland there is a visible performance gap between highways and railways. But only the best non-Swiss operator A1 Telekom Austria comes anywhere close to thirdplaced Sunrise. Orange takes the lead in the train telephony scenario while Swisscom finishes somewhat behind, still way ahead of the other competitors in the international comparison.

Data in cities

In the Swiss cities, usage of mobile Internet is a dream. Success Ratios usually range between 99 and 100 percent for all Network Operators. Accessing a popular web page typically takes no longer than 3 seconds irrespective of the Operator. Swisscom still manages to raise the bar over the very good competition, impressing with average download speeds of 35Mbit/s and uploads of 27Mbit/s. Average speeds have improved by more than 50% since last year, showing how hard the operators work on their networks. With such high data speeds, it does not come as a surprise that YouTube videos play almost perfectly in Switzerland. Only Sunrise still shows some problems with HD videos in small cities.

Data on transfer roads

The high level of reliability of Swisscom and Sunrise only drops marginally on the transfer roads, while only the critical YouTube HD suffers slightly more. Orange falls behind but stays at a very good level. Swisscom once again scores the highest average data rates. On the trains, performance is affected more, but again the Swiss number 3 ends up level pegging with the Austrian number 1, let alone the German networks. Switzerland confirms its reputation as a railroad country.

Conclusion

The big “Netztest” leads to the conclusion that Swiss customers deserve our envy as they only have the choice between “very good” networks in both the voice and the data domain. Roughly quoting Oscar Wilde: “One who is always satisfied with the best” when it comes to mobile service in Switzerland will end up with Swisscom. Owing to the very good coverage of the rural areas Swisscom achieves the advantage that earns test victory in Switzerland. In addition, they score more points than the best networks in Germany and Austria and – for the first time in the 7-year existence of the connectDACH-Netztest – finish ahead of A1 Telekom Austria.

 

TELEPHONY

OPERATOR Swisscom Sunrise Orange
TELEPHONY(BIG CITIES OUTDOOR/INDOOR)
Call Success Ratio (% / %) 98,6 / 99,4 98,9 / 99,4 98,4 / 99,6
Call Setup Time (s / s) 7,5 / 7,2 4,5 / 4,4 5,9 / 5,8
Speech Quality (MOS-LQO / MOS-LQO) 3,4 / 3,5 3,4 / 3,5 3,2 / 3,4
Call Sustainability (% / %) 99,7 / 99,5 99,4 / 99,8 99,5 / 99,9
TELEPHONY(HIGHWAYS/TRAINS)
Call Success Ratio (% / %) 97,8 / 93,7 97,7 / 91,6 97,2 / 96,5
Call Setup Time (s / s) 7,5 / 7,9 4,6 / 4,9 5,8 / 5,9
Speech Quality (MOS-LQO / MOS-LQO) 3,4 / 3,3 3,3 / 3,2 3,1 / 3,1
Call Sustainability (% / %) 99,7 / 98,8 98,8 / 97,2 99,6 / 99,5

 

In addition to metropolitan areas and big cities, the route in Switzerland also included 22 small cities, such as the famous Mon – treux.
Continuous supervision of the measurements by a controller in the passenger seat ensures the quality of the measurements.


DATA

OPERATOR Swisscom Sunrise Orange
DATA (BIG CITIES OUTDOOR)
WEB-PAGE DOWNLOAD (LIVE/STATIC)
Success Ratio (%/%) 99,4 / 99,6 99,5 / 99,6 99,4 / 99,2
Avg. Session Time (s/s) 2,7 / 1,1 2,6 / 1,1 3,0 / 1,1
FILE-DOWN-/UPLOAD (3 MB/1 MB)
Success Ratio (%/%) 99,8 / 99,9 99,8 / 99,8 99,5 / 99,9
90 % faster than (kbps / kbps) 11619 / 4613 4908 / 1871 7221 / 2497
FILE-DOWN-/UPLOAD (10 S/10 S)
Avg. Throughput (kbps / kbps) 31245 / 26906 25357 / 19331 22155 / 16118
90 % faster than (kbps / kbps) 13573 / 5847 5683 / 2211 9812 / 3053
YOUTUBE SD/HD
Success Ratio / Start Time (%/s) 99,9 / 99,3 99,7 / 98,7 99,8 / 99,0
Video playouts without interruptions (%/%) 99,8 / 98,9 99,5 / 97,3 99,7 / 98,4
DATA (BIG CITIES INDOOR)
WEB-PAGE DOWNLOAD (LIVE/STATIC)
Success Ratio (%/%) 99,5 / 100 99,5 / 99,8 99,6 / 100
Avg. Session Time (s/s) 2,8 / 0,9 2,6 / 1,0 3,0 / 0,9
FILE-DOWN-/UPLOAD (3 MB/1 MB)
Success Ratio (%/%) 99,8 / 99,7 99,8 / 99,5 99,8 / 100
90 % faster than (kbps / kbps) 10774 / 3023 8742/ 2700 10150 / 3963
FILE-DOWN-/UPLOAD (10 S/10 S)
Avg. Throughput (kbps / kbps) 34639 / 27285 29782 / 21128 23945 / 16701
90 % faster than (kbps / kbps) 13411 / 3083 8590 / 2338 11614 / 4946
YOUTUBE SD/HD
Success Ratio / Start Time (%/s) 100 / 99,8 100 / 99,3 100 / 99,8
Video playouts without interruptions (%/%) 100 / 99,8 100 / 99,3 100 / 99,6
DATA (SMALL CITIES OUTDOOR)
WEB-PAGE DOWNLOAD (LIVE/STATIC)
Success Ratio (%/%) 99,9 / 100 99,8 / 99,7 99,9 / 99,7
Avg. Session Time (s/s) 2,7 / 1,1 2,7 / 1,4 2,9 / 1,1
FILE-DOWN-/UPLOAD (3 MB/1 MB)
Success Ratio (%/%) 100 / 100 100 / 100 100 / 100
90 % faster than (kbps / kbps) 13296 / 6182 4286 / 1399 6289 / 1254
FILE-DOWN-/UPLOAD (10 S/10 S)
Avg. Throughput (kbps / kbps) 31029 / 26828 17733 / 12744 20904 / 15524
90 % faster than (kbps / kbps) 14045 / 7954 4355 / 1551 6414 / 1613
YOUTUBE SD/HD
Success Ratio / Start Time (%/s) 100 / 100 100 / 96,1 100 / 99,3
Video playouts without interruptions (%/%) 100 / 98,7 100 / 94,8 100 / 98,0
DATEN (TRANSFERROUTEN)
WEB-PAGE DOWNLOAD (LIVE/STATIC)
Success Ratio (%/%) 99,1 / 98,9 98,7 / 99,3 97,9 / 96,3
Avg. Session Time (s/s) 3,1 / 1,6 3,2 / 1,9 3,6 / 1,8
FILE-DOWN-/UPLOAD (3 MB/1 MB)
Success Ratio (%/%) 99,3 / 99,5 98,0 / 99,0 96,8 / 97,4
90 % faster than (kbps / kbps) 6379 / 2059 2535 / 1094 2537 / 1376
FILE-DOWN-/UPLOAD (10 S/10 S)
Avg. Throughput (kbps / kbps) 28271 / 22778 17270 / 10976 20672 / 14364
90 % faster than (kbps / kbps) 7301 / 1849 2706 / 949 3848 / 693
YOUTUBE SD/HD
Success Ratio / Start Time (%/s) 99,7 / 98,2 98,8 / 96,1 99,1 / 96,2
Video playouts without interruptions (%/%) 99,7 / 97,2 98,2 / 94,2 98,8 / 95,4
DATEN (BAHN)
WEB-PAGE DOWNLOAD (LIVE/STATIC )
Success Ratio (%/%) 98,2 / 98,1 97,9 / 98,0 97,6 / 98,0
Avg. Session Time (s/s) 3,8 / 1,3 4,4 / 1,9 4,0 / 1,5
FILE-DOWN-/UPLOAD (3 MB/1 MB)
Success Ratio (%/%) 98,6 / 97,6 95,6 / 96,6 97,0 / 99,0
90 % faster than (kbps / kbps) 2685 / 692 1622 / 928 2352 / 1165
FILE-DOWN-/UPLOAD (10 S/10 S)
Avg. Throughput (kbps / kbps) 20919/ 17797 10983 / 7178 18297 / 14797
90 % faster than (kbps / kbps) 3883 / 530 1226 / 205 3290 / 1005
YOUTUBE SD/HD
Success Ratio / Start Time (%/s) 98,5 / 96,4 97,0 / 89,3 98,0 / 96,4
Video playouts without interruptions (%/%) 98,5 / 93,8 95,5 / 82,7 97,5 / 93,8

 

SWISSCOM

In this year’s test Swisscom sets the benchmark. No need to add anything more.
Swisscom almost made a clean sweep. The performance of the telephony service is at the highest level with respect to reliability and audio quality. Only the call setup duration blurs the results and allows the competition to catch up. For its market share of 60 percent Swisscom passes the tests of its data services with flying colours. Even the coverage of rural areas is at the highest level. For these reasons Swisscom has the best network in Switzerland and in the Germany-Austria-Switzerland benchmark.
connect-Scoring
very good (456 points)


SUNRISE

For telephony Sunrise is a step ahead due to fast call setup. For data services, Sunrise scores “very good”.
For telephony Sunrise runs a close race against Swisscom and Orange in terms of reliability and audio quality at an overall very high level. In the big cities Sunrise wins this race but loses on the rural routes. However, due to the fastest call setup Sunrise finally wins the telephony competition even in the Germany-Austria-Switzerland contest. On top of that, Sunrise shows consistently high reliability and speed levels leading to a final rating of “very good”.
connect-Scoring
very good (442 points)


ORANGE

Orange shows very good performance for both voice and data services and shines on railway tracks.
Like the competition, Orange also provides telephony services with literal Swiss clockwork precision in all aspects, including the audio quality. The smallest network operator in Switzerland even reached the top ranking in trains, not only for telephony but also – with a slight advance over Swisscom – for data services. Despite minor weaknesses for data upload and high definition YouTube on the transfer routes, Orange provides data services at a very high level. >>
connect-Scoring
very good (441 points)


TEST METHODOLOGY CONNECT

As in previous years P3 communications, Connect’s partner for executing network benchmarks, sent out two measurement cars to perform the tests. The cars were equipped with ten Samsung Galaxy S4 LTE+ smartphones each – six to eight of them used for testing depending on the country. As far as possible, the smartphone’s firmware matched the network operator’s genuine ones. In cases when no operator specific firmware was available the most recent Samsung firmware was used. In addition to the cars a separate test team executed walk tests indoors or in trains using a portable battery powered measurement system.

Smartphone-Telephony

Data and voice services were executed with separate smartphones. Telephony was tested mobile-to-mobile from one car to the other. In order to ensure realistic testing conditions background data traffic was generated on the smartphones during a voice call.
The audio quality of the transmitted speech samples was evaluated using the HD-voice capable and ITU standardized so-called POLQA wideband algorithm.
This year, all network operators offer LTE capable subscriptions. Therefore all smartphones were set to “LTE preferred” mode. As a consequence, the phones needed to switch (“fall back”) to 2G or 3G when they camped on the LTE service and a phone call was initiated. This procedure is known as circuit-switched fall back (CSFB).

Smartphone-Data

The testing of data services was done with three to four separate Samsung smartphones in the cars. As a first measurement task several popular live web pages were browsed using the built-in smartphone browser. The web pages had been selected previously according to the Alexa ranking. In addition, the artificial (static) “Kepler” test web page as specified by ETSI (European Telecommunications Standards Institute) for such testing purposes was used.
In order to test the data service performance, files of 3MB and 1MB for download and upload respectively were transferred from or to a test server located in the Internet. In addition, the peak data performance was tested in up- and downlink direction by assessing the amount of data that could be transferred within a 10 seconds time period.
While 10 seconds peak throughput test focuses on network capability the YouTube tests add a more end user centric view to the test curriculum. Here, videos with standard definition (SD, 360p, 2.7MB, 30s) and high definition (HD, 720p, 11.9MB, 30s) were played using the YouTube player on the smartphones.
In order to simulate in-car or indoor coverage a 12dB attenuation was inserted in the path between the smartphones in the car and the antennae mounted on the car’s roof top. Due to the usage of MIMO technology (Multiple Input Multiple Output) in LTE two antennas were used per each smartphone.

Walk tests indoors and in trains

The same test program executed for voice and data services in the measurement cars was also executed by the walk test teams.
Four smartphones on the left-hand side are used for testing telephony while the phones on the right serve for testing the data Services

Logistics

During the measurements, the two cars were in the same city, but not at the same location. This was to avoid disturbances from one car to the other. Both cars visited defined destinations. For stationary measurements they stopped at selected “areas of interest”, typically crowded locations such as airports, train stations, recreation areas or densely populated residential areas.
Especially at those locations, the measurement systems have to share the available networks capacities with other normal users. This is according to the usual competition between users for the more or less scarce bandwidth.
The selected cities should be representative for a large proportion of the population. Therefore the measurements were executed in 21 of the bigger cities in Germany, from Osnabrück with 156,000 inhabitants up to Berlin with 3,4 million inhabitants. Smaller cities were visited by one measurement car during the transfers from one large city to the next one, along the routes. 27 smaller cities were visited, covering sizes from 20,000 inhabitants, like Witten or Nauen, up to almost 90,000 inhabitants in cities like Esslingen. In Switzerland and Austria 16 of the biggest 25 cities were covered, including approximately 20 smaller cities in each country. Between the cities highways were mainly used by the measurement cars.

Effort and output

During 45,000 measurement kilometers – a circumnavigation of the world – 150,000 measurement samples were collected, almost half of them in Germany. This effort is necessary to provide statistically relevant network quality Statements.


FINAL CONCLUSION Bernd Theiss, connect editor

The 2014 mobile network test reveals that building and engineering an LTE network for data combined with a reliable 2G/3G network for voice is not a trivial exercise. In Germany, Telekom was successful in doing so. The network operator provides telephony services of the highest performance in both small and big cities. But Telekon is also ahead of the competition on rural routes despite some weaknesses in trains. However Vodafone has made a great leap ahead and is now – in contrast to previous years – on Telekom’s heels and showed the best indoor coverage in the city centres during the test. Furthermore, Vodafone provides reliable and fast data services though Telekom is still ahead in terms of speed. Therefore, Telekom remains the test winner in Germany. O2 having introduced LTE late shows signs of weakness in voice service quality while E-Plus falls behind in data service performance due to a lack of LTE coverage. It will be exciting to see whether there will be a performance boost in the future after last year’s merger of the two operators seemed to have slowed down the development of their networks.
The Swiss networks Sunrise and Orange show how to exploit room for improvement and outpoint Germany’s number one Telekom this year. A1 Telekom Austria scored even slightly better but had to make way for the winner of the three country test: Swisscom.

The measurements took place on public transport and in particular in publicly accessible buildings like cafés, museums, train stations or airport terminal buildings. As far as possible the testers took regional trains to travel between cities and for taking measurements during the journey.

Mobile network test final results of Germany, Austria and Switzerland:

COUNTRY GERMANY AUSTRIA SWITZERLAND
Netzbetreiber Deutsche
Telekom
Vodafone O2 E-Plus A1 Telekom
Austria
T-Mobile Drei Swisscom Sunrise Orange  
Telephony max.210 175 171 128 145 188 175 183 181 189 182
Big Cities (outdoor) 95 85% 85% 58% 69% 90% 86% 87% 84% 91% 84%
Big Cities (indoor) 35 82% 84% 62% 78% 91% 89% 94% 89% 95% 93%
Highways 70 85% 79% 66% 68% 91% 80% 87% 88% 89% 87%
Trains 10 61% 57% 54% 43% 78% 66% 59% 82% 77% 89%
Data max. 290 261 229 207 182 265 242 230 275 253 259
Big Cities (Outdoor) 135 91% 83% 80% 68% 96% 87% 83% 94% 87% 88%
Big Cities (Indoor) 50 90% 78% 67% 65% 94% 91% 74% 94% 90% 92%
Small Cities 35 86% 65% 59% 53% 77% 72% 71% 96% 81% 86%
Highways 50 95% 88% 70% 58% 90% 77% 85% 97% 91% 90%
Trains 20 76% 54% 46% 55% 86% 77% 68% 95% 85% 96%
connect Rating max. 500 436
very good
400
good
335
satisfactory
327
satisfactory
453
very good
417
good
413
good
456
very good
442
very good
 441
very good