Austria 2014

THE GREAT MOBILE NETWORK TESTFlagge_oesterreich

Download test here

 

Conducted since 1993, the connect mobile network test with its neutral, customer-oriented testing has become an industry standard by which big corporations choose their operator. Who’s leading in Germany, Austria and Switzerland in 2014?

As Rudolf Augstein, founder of Germanys biggest news magazine „Der Spiegel“, said as guidance to young journalists: „Just being straight“. This motto should also hold true for a mobile network test, but it is not always so.
One German magazine simply focuses on easy to sell sensations and presents their benchmark as “Germany’s hardest network test” – despite the fact that two of the four German operators ended up third for the second year in a row and with minimal distance to the winner.
Another big publication completely lost track of their benchmark data, rating an operator with less than 3% 4G market share higher than one with more than 30% of the LTE market but slightly less throughput.

The proven Standard – since 1993

Connect refrains from superlatives and sends the proven test team of P3 communications on a trip through Germany, Austria and Switzerland. Two measurement cars per country, equipped with two measurement smartphones per operator, analyzing voice and data services in big cities, small cities and on the connecting highways.
In addition, two walk test systems per country, one for voice and one for data, are used for the same tests as in the measurement cars, with focus on indoor locations and railway connections. A third measurement car is used as counterpart station for the walk test voice system. The detailed methodology description can be found at the end of this page. The results of the German tests can be found on the following pages. How the Austrian and Swiss operators perform this year, and whether they outperform the Germans again, can be found in the menue point “Network Tests”.

 

AUSTRIA

In recent years, Mobile Network Operators in Austria set the standards others tried to achieve. Can they repeat it this year (2014)?

The mobile market in Austria is known among experts as particularly interesting. 8 million inhabitants use only 2.6 million landline connections but over 13 million SIM cards (as of 2013). Thus, communication is strongly focused on the mobile networks. This is proven by the steep increase in transferred data volume, which rose by 57% (equivalent to 112,000 Terabytes) between 2012 and 2013. Operators need to respond with continuous capacity expansion even to maintain the current standard.
The strong position of mobile radio in Austria must be credited to the Network Operators who have continuously offered high quality and performance levels. Ever since connect started testing all three countries in 2009, an Austrian Operator finished first in the international comparison, although the leading position changed twice between local competitors. And no Austrian operator ever scored lower than “good”.

Telephony

This year, the performance level of mobile telephony in Austria is high again. Where German operators are top, the performance range of Austrian operators only starts. In cities, T-Mobile offers very good telephony with success ratios between 98,4% and 98,7% and a Mean Opinion Score (MOS) of 3.6. Drei and A1 Telekom Austria are even better. This means Drei and TMobile managed the introduction of LTE well, overcoming the difficulties this brings about for the voice service. A1 and T-Mobile also showed remarkably short Call Setup Times in LTE.
Stability suffers slightly on the transfer roads between cities, leading to a success ratio of 95% for T-Mobile. Drei and A1 Telekom Austria end up with 97-98% success ratios and thus show better performance than other operators in big cities. A1 also improved compared to the previous year.
So is all now perfect in Austria? Not quite, as success ratios substantially drop during measurements on trains. Drei only achieves 82.6%, trailing T-Mobile with 85.9%. A1 manages to score best and loses slightly less then every tenth call (90.8%).

Data in cities

Voice telephony is mandatory for network operators while data service gives the possibility to differentiate. The network operators in Austria also show an impressive reliability in this discipline in big cities. Success ratios beyond 99.5% for internet browsing and file up-/download are visible and only for YouTube high-definition two out of three cross the 99% threshold due to slightly increased number of interruptions. The indoor measurements are even slightly more critical than the outdoor tests. A1 Telekom Austria also manages this challenge impressively and improves average download speeds from 13 to 52 Mbit/s. This is also related to the high throughput required by YouTube HD service and provided by the A1 network. Web browsing benefits less from this; there the difference between a loading time of 2.2 seconds on average for a popular web page with dynamic content for A1 and around 3 seconds for T-Mobile and Drei is not really significant. The differentiation is perceived for up- and download of larger files where A1 is up to 3 times faster than Drei, who increased average data rates compared to last year from 7- 30 Mbit/s. Also TMobile made a big step ahead from 13 to 28 Mbit/s. In smaller cities the network operators are getting closer to each other, however A1 still shows highest performance despite partly higher reliability for Drei and T-Mobile.

Data on transfer routes

The picture for transfer routes looks similar to that for cities. However Drei can outperform T-Mobile and catch up with A1. Both improved heavily compared to the previous year. For railways, the known ranking A1, T-Mobile und Drei is visible again, where only the number 1 provides a more or less acceptable performance.

Conclusion

The overall performance level offered in Austria is very high, with some exceptions in the train and small city scenarios. Consequently, Drei and T-Mobile finish the test well inside the “good” range, although conditions and requirements got tougher this year. Particularly in the big cities, they provide plenty of power

for everything but the most demanding tasks. Demanding customers who also expect to use telephony and mobile internet on the trains make the best choice with A1 Telekom Austria, deserving the mark “very good” for the 6th time in a row. >>

TELEPHONY

AUSTRIA
OPERATOR A1 Telekom Austria T-Mobile Drei
TELEPHONY(BIG CITIES OUTDOOR/INDOOR)
Call Success Ratio (% / %) 98,7 / 99,1 98,4 / 98,7 98,7 / 99,8
Call Setup Time (s / s) 5,7 / 6 5,7 / 5,6 6,8 / 6,9
Speech Quality (MOS-LQO / MOS-LQO) 3,6 / 3,7 3,5 / 3,6 3,7 / 3,8
Call Sustainability (% / %) 99,9 / 99,7 99,6 / 99,7 99,8 / 99,7
TELEPHONY(HIGHWAYS/TRAINS)
Call Success Ratio (% / %) 97,9 / 90,8 95,0 / 85,9 97,1 / 82,6
Call Setup Time (s / s) 5,7 / 5,5 6,5 / 6,3 7,4 / 7,7
Speech Quality (MOS-LQO / MOS-LQO) 3,6 / 3,4 3,3 / 3,2 3,5 / 3,4
Call Sustainability (% / %) 99,5 / 98,7 99,4 / 98,7  99,6 / 98,5

Kufstein and Wörgl are among the 20 small cities that the test team visited in addition to the important big cities.
The large number of antennae are distributed across the roof box metal plate for homogeneous RF conditions.

DATA

AUSTRIA
OPERATOR A1 Telekom
Austria
T-Mobile Drei
DATA (BIG CITIES OUTDOOR)
WEB-PAGE DOWNLOAD (LIVE/STATIC)
Success Ratio (%/%) 99,7 / 99,6 99,8 / 99,7 99,8 / 99,5
Avg. Session Time (s/s) 2,2 / 0,6 2,8 / 1,2 3,0 / 1,6
FILE-DOWN-/UPLOAD (3 MB/1 MB)
Success Ratio (%/%) 99,5 / 99,7 99,9 / 99,8 99,5 / 99,5
90 % faster than (kbps / kbps) 2355 / 4379 7397 / 1136 4419 / 1207
FILE-DOWN-/UPLOAD (10 S/10 S)
Avg. Throughput (kbps / kbps) 52033 / 30923 28124 / 16952 30433 / 15497
90 % faster than (kbps / kbps) 23617 / 4274 7436 / 873 4435 / 729
YOUTUBE SD/HD
Success Ratio / Start Time (%/s) 99,9 / 100 99,8 / 98,6 99,8 / 99
Video playouts without interruptions (%/%) 99,9 / 99,8 99,6 / 97,8 99,7 / 97,1
DATA (BIG CITIES INDOOR)
WEB-PAGE DOWNLOAD (LIVE/STATIC)
Success Ratio (%/%) 99,7 / 99,8 99,7 / 99,4 99,8 / 100
Avg. Session Time (s/s) 2,5 / 0,6 2,8 / 0,7 3,8 / 1,5
FILE-DOWN-/UPLOAD (3 MB/1 MB)
Success Ratio (%/%) 99,8 / 99,8 99,8 / 99,6 99,2 / 100
90 % faster than (kbps / kbps) 2132 / 2782 11452 / 1159 3266 / 634
FILE-DOWN-/UPLOAD (10 S/10 S)
Avg. Throughput (kbps / kbps) 54518 / 28694 39943 / 19911 23252 / 9998
90 % faster than (kbps / kbps) 19834 / 2784 11187 / 1171 3350 / 351
YOUTUBE SD/HD
Success Ratio / Start Time (%/s) 100 / 99,8 99,8 / 98,8 99,6 / 96,0
Video playouts without interruptions (%/%) 100 / 99,6 99,8 / 98,6 99,2 / 91,8
DATA (SMALL CITIES OUTDOOR)
WEB-PAGE DOWNLOAD (LIVE/STATIC)
Success Ratio (%/%) 97,9 / 96,9 99,0 / 97,7 99,8 / 99,6
Avg. Session Time (s/s) 3,0 / 1,8 3,7 / 2,0 3,7 / 2,2
FILE-DOWN-/UPLOAD (3 MB/1 MB)
Success Ratio (%/%) 99,2 / 98,4 100 / 99,2 98,5 / 99,3
90 % faster than (kbps / kbps) 1085 / 890 5063 / 667 2421 / 1239
FILE-DOWN-/UPLOAD (10 S/10 S)
Avg. Throughput (kbps / kbps) 26233 / 11151 14558 / 5468 18576 / 8263
90 % faster than (kbps / kbps) 8243 / 649 5471 / 516 3428 / 473
YOUTUBE SD/HD
Success Ratio / Start Time (%/s) 100 / 98,4 98,4 / 98,4 100 / 94,6
Video playouts without interruptions (%/%) 100 / 97,6 98,4 / 97,6 100 / 90,0
DATEN (TRANSFERROUTEN)
WEB-PAGE DOWNLOAD (LIVE/STATIC)
Success Ratio (%/%) 97,7 / 97,3 98,0 / 96,5 98,9 / 98,4
Avg. Session Time (s/s) 3,4 / 2,2 4,8 / 3,2 4,2 / 2,8
FILE-DOWN-/UPLOAD (3 MB/1 MB)
Erfolgsrate/Ladezeit (%) 97,5 / 98,8 97,8 / 99,1 98,0 / 99,1
90 % schneller als (kbps) 841 / 530 1207 / 318 2482 / 863
FILE-DOWN-/UPLOAD (10 S/10 S)
Avg. Throughput (kbps / kbps) 24485 / 11290 12500 / 4164 16400 / 7109
90 % faster than (kbps / kbps) 4458 / 354 1800 / 188 2974 / 241
YOUTUBE SD/HD
Success Ratio / Start Time (%/s) 99,1 / 98,3 97,1 / 94,5 98,4 / 96,1
Video playouts without interruptions (%/%) 98,9 / 98,1 96,4 / 93,3 97,3 / 93,9
DATEN (BAHN)
INTERNET-SEITENAUFRUF (LIVE/STATISCH)
Success Ratio (%/%) 96,1 / 96,4 96,5 / 96,9 96,8 / 97,8
Avg. Session Time (s/s) 4,1 / 1,8 5,5 / 2,5 6,2 / 3,0
FILE-DOWN-/UPLOAD (3 MB/1 MB)
Success Ratio (%/%) 91,4 / 95,8 95,3 / 98,4 91,6 / 97,2
90 % faster than (kbps / kbps) 392 / 301 763 / 317 827 / 195
FILE-DOWN-/UPLOAD (10 S/10 S)
Avg. Throughput (kbps / kbps) 14578 / 4311 11329 / 2818 5946 / 1710
90 % faster than (kbps / kbps) 2702 / 168 1376 / 143 769 / 74
YOUTUBE SD/HD
Success Ratio / Start Time (%/s) 98,3 / 94,6 93,9 / 84,1 97,1 / 80,7
Video playouts without interruptions (%/%) 97,8 / 90,4 91,7 / 81,8 91,9 / 69,3

 

A1 TELEKOM AUSTRIA

With the best telephony service as well as fast and solid data transfers A1 forges ahead in Austria.
A1 Telekom Austria wins eight out of nine categories owing to the stable and high quality speech transmission as well as the almost area-wide and extremely fast data network. In big cities, the average throughput was about 55 Mbit/s in downlink and about 29 Mbit/s in uplink direction. Even on the hard to cover railway tracks A1 is one step ahead of the competition and wins the undisputed sole lead in Austria.
connect-Scoring
very good (453 points)

 

T-MOBILE

T-Mobile Austria provides voice and data services with an out-and-out good performance.
Despite of the even more ambitious test conditions this year, T-Mobile shows a boost in its network performance. Within the last 12 months the average download throughput rose from 13 Mbit/s to 28 Mbit/s. At the most, T-Mobile lost some points with telephony on railway tracks, data upload and high definition videos in small cities. However, this might go unnoticed by the end user as these deficiencies are marginal. Instead, T-Mobile can cope with the more relevant disciplines like data transfers in the metropolitan areas and thus serves very well those customers who attach importance to a sound data performance in the city centres.
connect-Scoring
good (417 points)

 

DREI

Drei satisfies with excellent telephony. And Austria’s smallest network operator provides even more.
With respect to telephony Drei outperforms T-Mobile by far despite weaknesses on railway tracks. However, in cities and on the interconnecting routes the success ratio is high and the audio quality is strong. Drei suffers the loss of some points with its data services. Here, the upload speed and the stability of high definition YouTube videos take their toll again. This happened in particular for indoor coverage in big as well as small cities. As a matter of fact, the provisioning of voice and data services in trains is also problematic. The bottom line is that the good performance in the most relevant areas attests Drei as giving a sound data service.
connect-Scoring
good (413 points)

 

 

TEST METHODOLOGY CONNECT

As in previous years P3 communications, Connect’s partner for executing network benchmarks, sent out two measurement cars to perform the tests. The cars were equipped with ten Samsung Galaxy S4 LTE+ smartphones each – six to eight of them used for testing depending on the country. As far as possible, the smartphone’s firmware matched the network operator’s genuine ones. In cases when no operator specific firmware was available the most recent Samsung firmware was used. In addition to the cars a separate test team executed walk tests indoors or in trains using a portable battery powered measurement system.

Smartphone-Telephony

Data and voice services were executed with separate smartphones. Telephony was tested mobile-to-mobile from one car to the other. In order to ensure realistic testing conditions background data traffic was generated on the smartphones during a voice call.
The audio quality of the transmitted speech samples was evaluated using the HD-voice capable and ITU standardized so-called POLQA wideband algorithm.
This year, all network operators offer LTE capable subscriptions. Therefore all smartphones were set to “LTE preferred” mode. As a consequence, the phones needed to switch (“fall back”) to 2G or 3G when they camped on the LTE service and a phone call was initiated. This procedure is known as circuit-switched fall back (CSFB).

Smartphone-Data

The testing of data services was done with three to four separate Samsung smartphones in the cars. As a first measurement task several popular live web pages were browsed using the built-in smartphone browser. The web pages had been selected previously according to the Alexa ranking. In addition, the artificial (static) “Kepler” test web page as specified by ETSI (European Telecommunications Standards Institute) for such testing purposes was used.
In order to test the data service performance, files of 3MB and 1MB for download and upload respectively were transferred from or to a test server located in the Internet. In addition, the peak data performance was tested in up- and downlink direction by assessing the amount of data that could be transferred within a 10 seconds time period.
While 10 seconds peak throughput test focuses on network capability the YouTube tests add a more end user centric view to the test curriculum. Here, videos with standard definition (SD, 360p, 2.7MB, 30s) and high definition (HD, 720p, 11.9MB, 30s) were played using the YouTube player on the smartphones.
In order to simulate in-car or indoor coverage a 12dB attenuation was inserted in the path between the smartphones in the car and the antennae mounted on the car’s roof top. Due to the usage of MIMO technology (Multiple Input Multiple Output) in LTE two antennas were used per each smartphone.

Walk tests indoors and in trains

The same test program executed for voice and data services in the measurement cars was also executed by the walk test teams.
Four smartphones on the left-hand side are used for testing telephony while the phones on the right serve for testing the data Services

Logistics

During the measurements, the two cars were in the same city, but not at the same location. This was to avoid disturbances from one car to the other. Both cars visited defined destinations. For stationary measurements they stopped at selected “areas of interest”, typically crowded locations such as airports, train stations, recreation areas or densely populated residential areas.
Especially at those locations, the measurement systems have to share the available networks capacities with other normal users. This is according to the usual competition between users for the more or less scarce bandwidth.
The selected cities should be representative for a large proportion of the population. Therefore the measurements were executed in 21 of the bigger cities in Germany, from Osnabrück with 156,000 inhabitants up to Berlin with 3,4 million inhabitants. Smaller cities were visited by one measurement car during the transfers from one large city to the next one, along the routes. 27 smaller cities were visited, covering sizes from 20,000 inhabitants, like Witten or Nauen, up to almost 90,000 inhabitants in cities like Esslingen. In Switzerland and Austria 16 of the biggest 25 cities were covered, including approximately 20 smaller cities in each country. Between the cities highways were mainly used by the measurement cars.

Effort and output

During 45,000 measurement kilometers – a circumnavigation of the world – 150,000 measurement samples were collected, almost half of them in Germany. This effort is necessary to provide statistically relevant network quality Statements.


FINAL CONCLUSION Bernd Theiss, connect editor

The 2014 network benchmark reveals that building and engineering an LTEnetwork for data combined with a reliable 2G/3G network for voice is not a trivial exercise. In Germany, Telekom was successful in doing so. The network operator provides telephony services of the highest performance in both small and big cities. But Telekon is also ahead of the competition on rural routes despite some weaknesses in trains. However Vodafone has made a great leap ahead and is now – in contrast to previous years – on Telekom’s heels and showed the best indoor coverage in the city centres during the test. Furthermore, Vodafone provides reliable and fast data services though Telekom is still ahead in terms of speed. Therefore, Telekom remains the test winner in Germany. O2 having introduced LTE late shows signs of weakness in voice service quality while E-Plus falls behind in data service performance due to a lack of LTE coverage. It will be exciting to see whether there will be a performance boost in the future after last year’s merger of the two operators seemed to have slowed down the development of their networks.
The Swiss networks Sunrise and Orange show how to exploit room for improvement and outpoint Germany’s number one Telekom this year. A1 Telekom Austria scored even slightly better but had to make way for the winner of the three country test: Swisscom.

The measurements took place on public transport and in particular in publicly accessible buildings like cafés, museums, train stations or airport terminal buildings. As far as possible the testers took regional trains to travel between cities and for taking measurements during the journey.

COUNTRY GERMANY AUSTRIA SWITZERLAND
Netzbetreiber Deutsche
Telekom
Vodafone O2 E-Plus A1 Telekom
Austria
T-Mobile Drei Swisscom Sunrise Orange  
Telephony max.210 175 171 128 145 188 175 183 181 189 182
Big Cities (outdoor) 95 85% 85% 58% 69% 90% 86% 87% 84% 91% 84%
Big Cities (indoor) 35 82% 84% 62% 78% 91% 89% 94% 89% 95% 93%
Highways 70 85% 79% 66% 68% 91% 80% 87% 88% 89% 87%
Trains 10 61% 57% 54% 43% 78% 66% 59% 82% 77% 89%
Data max. 290 261 229 207 182 265 242 230 275 253 259
Big Cities (Outdoor) 135 91% 83% 80% 68% 96% 87% 83% 94% 87% 88%
Big Cities (Indoor) 50 90% 78% 67% 65% 94% 91% 74% 94% 90% 92%
Small Cities 35 86% 65% 59% 53% 77% 72% 71% 96% 81% 86%
Highways 50 95% 88% 70% 58% 90% 77% 85% 97% 91% 90%
Trains 20 76% 54% 46% 55% 86% 77% 68% 95% 85% 96%
connect Rating max. 500 436
very good
400
good
335
satisfactory
327
satisfactory
453
very good
417
good
413
good
456
very good
442
very good
 441
very good