Spain 2016

The 2016 Mobile Network Test in Spain

Download English version of the test here
Download Spanish version of the test here

click to enlarge

Druck

A total of 56 million mobile subscribers makes Spain one of the largest mobile network markets in Europe. Telefónica’s Movistar has the most subscribers, while Vodafone and ­Orange are constantly competing for the second rank. Yoigo has started to grow under its new ownership.

So, the 2016 P3 connect Mobile Benchmark Spain promises interesting results. Which network is the best choice for voice and for data communi­cation? Which operator scores best in quality and performance?


Results in a nutshell

The 2016 P3 connect Mobile Benchmark in Spain shows clear improvements, both in the scores and KPIs of all four contenders. The overall ranking of Spain’s operators, however, does not present many surprises.
7915041P3‘s network benchmarks are widely accepted for setting industry standards as well as being highly objective. The carefully designed methodology scheduled four drivetest cars to visit 14 large cities and 26 smaller towns in Spain as well as the roads and motorways connecting these municipalities. The test routes also included the islands Tenerife and Gran Canaria.

The areas in which we tested ­accounted for more than 11 million people, or approximately 23.8 per cent of the Spanish population. One aspect of P3‘s great attention to detail is the use of up-to-date LTE “Cat 9” smartphones for the data tests, which accounts for the latest technical develop­ments such as LTE carrier aggregation. Another factor is the use of the most comprehensive mobile plans avail­able from each operator.

2016‘s results see an overall improvement in performance levels – each operator was ­able to gather more points than in last year‘s mobile network benchmark, although we have ­increased thresholds and requirements to ­reflect the general technical development.

es_totalscore2016_englisch

Like in 2015‘s P3 connect Mobile Benchmark in Spain, the overall winner is Vodafone with strong results both in the voice and data categories. Vodafone‘s improvements in the data category are even more impressive than in the already strong voice discipline.

Movistar comes in as a strong second and also shows respectable results both in the voice and data disciplines. Like Vodafone, Movistar has improved more distinctly in the data category than in voice. Orange ranks third, but achieves the same voice score as its ­constant rival Movistar. Both networks fully ­deserve their overall grade “good“. Yoigo ranks last with a clear distance to the other three. In spite of its only “sufficient“ result, the smallest operator has improved in the voice cate­gory, but stagnated in the data section.

es_overall_results2016_englisch

 

Spain’s operators

Movistar

logo_movistarMovistar is the brand name that the Spanish telecommunications company Telefónica uses for the mobile network in its home market. Telefónica itself is one of the largest telco companies in the world. The operator is present in 21 countries with a total of 130,000 em­ployees and achieved worldwide revenues of over €47 ­billion in 2015.

While the company intro­duced the ­Movistar brand in Latin American countries in 2005, it has been active in Spain since the launch of GSM services back in 1995. Today, Movi­star is the largest mobile operator in Spain with about 17.1 million subscribers, which equals a market share of roughly 30 per cent. It offers GSM service at 900 and 1800 MHz, UMTS/3G at 900 and 2100 MHz and LTE at 800, 1800 and 2600 MHz.

Since the end of 2014, ­Movistar has supported 4G carrier ­aggregation – which means that suitable smartphones can combine LTE frequencies for higher bandwidth.

 

Vodafone

logo_vodafoneVodafone España has been present on the Spanish mobile communications market since the ­year 2000. Then, the British ­Vodafone Group acquired ­Airtel Móviles which had operated in Spain since 1994.

In Spain, ­Vodafone now has 14.1 million mo­bile customers, which adds up to a market share of about 25 per cent. In the fiscal year 2015/ 2016, Vodafone Spain achieved revenues of €4.5 billion which contributes about nine per cent to the whole Vodafone Group‘s financial result.

Vodafone‘s mobile network in Spain offers GSM service at 900 and 1800 MHz, UMTS/3G at 900 and 2100 MHz and LTE at 800, 1800 and 2600 MHz. The Vodafone mobile network in Spain supports LTE carrier aggregation of its 1800 and 2600 MHz frequency bands. The company offers LTE down­link speed with up to 300 Mbps and calls this service “4G+“. This service is available in all large cities including all provincial capitals as well as in many rural areas. Vodafone ­España continues to upgrade its mobile network sites to this “LTE Advanced“ standard.

 

Orange

logo_orangeOrange España is the brand ­name of France Telecom‘s mobile network in Spain. It has been operating under this name since 2006. Previously, the network was known as “Amena“ – this brand name lives on in Orange Spain‘s portfolio as a low-cost offer that is only available on the internet. Also, its network serves a number of mobile virtual network operators such as MasMovil, Carrefour Móvil and others. With 12.8 million custo­mers, Orange is the third largest Spanish mobile operator – its market share includes about 23 per cent of the population. In the fiscal year 2015, Orange Spain achieved a revenue of €4.2 ­billion which contributed 10.5 per cent to the whole Orange Group‘s 2015 results.

Orange Spain has deployed 2G networks at 900 and 1800 Mhz, 3G networks at 900 and 2100 MHz and 4G at 800, 1800 and 2600 MHz. Orange offers “4G+“ or “LTE Advanced“ with data rates of up to 200 Mbps in most larger cities including all provincial capitals.

 

Yoigo

logo_yoigoYoigo was the latest mobile operator to enter the Spanish market. Founded in the year 2000 under the name Xfera, the company started its actual operation in 2006, offering only a UMTS/3G network at 2100 MHz. At this time, the Swedish telecommunications company TeliaSonera ­acquired the majority of shares and rebranded the network as “Yoigo“. This name was supposed to reflect the simplicity and ease in rates as well as in the use of the service. In June 2016, the former MVNO (mobile virtual network operator) Másmóvil bought the company.

Yoigo has a national roaming agreement with Movistar until the end of 2016, which is sche­duled to switch to Orange in January 2017. Following the merger with Másmóvil, the  customer base has increased from 3.7 million former Yoigo subscribers to approxima­tely 4.2 million, which equals a market share of 7.5 per cent.

In July 2013, Yoigo started the deployment of a 4G network at 1800 MHz. Its LTE ­coverage now includes all ­major Spanish cities and most of the provincial capitals.

 

A close look at Spain’s networks

In 2015’s mobile network benchmark, Vodafone was the clear winner, followed at a distance by Movistar and Orange with Yoigo in last place. How do these competitors score one year later?

7915043

P3 communications GmbH, based in Aachen, Germany, is a world leader in mobile network testing. It is part of the P3 group, with over 3000 employees world­wide and a turnover of more than €300 million. P3 is partnering with the German telecommunications magazine connect, which has more than 20 years of editorial experience and is one of the leading test authorities in Europe for telecommunications products and services. Together, P3 and connect have been conducting the most important mobile network benchmark test in Germany for nearly 15 years, extending it to Austria and Switzerland in 2009. Starting in 2014, P3 has also been conducting network benchmarks in Australia and the UK, and expanded to cover the Netherlands and Spain last year.

data_voice_donuts2016_englischIn 2015 alone, P3 compiled more than 60,000 measurement hours in 47 countries, with its test vehicles covering more than 1.2 million kilometres. As the de-facto industry standard, the P3 benchmarking methodology focuses on customer-perceived network quality – examining both voice telephony that makes up 40% of the total result as well as data connectivity that accounts for 60% of the score. P3‘s network benchmarks are widely accepted as an objective authority.

Interesting results in 2016

After our first mobile network benchmark in Spain conducted in 2015, it was interesting to see whether the overall performance level would increase within one year – and whether there would be any surprising changes in the overall ranking. See for yourself how the Spanish operators performed in 2016.

 

Hakan Ekmen, Managing Director P3 communications: “All Spanish operators have improved compared to 2015‘s exercise – further developments can be expected in the coming year with Voice over LTE and extended 4G coverage.”

Hakan Ekmen, Managing Director P3 communications: “All Spanish operators have improved compared to 2015‘s exercise – further developments can be expected in the coming year with Voice over LTE and extended 4G coverage.”

 

 

 

 

 

 

 

 

 

Voice

Mobile customers expect reliable voice services. How do ­Spanish networks fulfill these expectations?

es_categorycomparison_voice_2016_englischIn order to check out the voice quality of Spain‘s mobile networks, P3 visited 14 ­cities with more than 100,000 inhabitants each and additionally covered 26 towns in the Spanish provinces as well as the roads connecting them. The good news upfront: Compared to last year‘s results, all four Spanish operators improved their performance in almost all tested categories.

Especially in the cities, Vodafone has achieved the best results over its competitors, with Orange ranking second and Movistar third. Vodafone leads with a clear distance to the midfield, whereas Orange and Movistar are only six points or two per cent apart. Yoigo comes last with a considerable gap. Interestingly, in the smaller towns the performance of the strong three competitors is closer together, with Movistar taking a narrow lead. Yoigo‘s results stay more or less at the same level as in the larger cities.

Yoigo particularly weak on the roads

On the connecting roads, Vodafone reclaims the top position, with a definitely strong ­Movistar and a still good Orange following. At last, Yoigo‘s performance on the roads leaves considerable room for improvement. Out of a hundred calls, almost 17 would fail in the Yoigo network – and those connected achieve an average speech quality at best.

es_tabelle_voice2016_englisch

VOICE results at a glance

Vodafone España takes a clear lead in the voice category. On the second rank, Movistar and Orange produce a tie. In smaller towns, Movistar is particularly strong and Orange achieves the same score as the voice champion Vodafone. ­Yoigo comes last in each of the tested ­scenarios with a clear distance to the top three.

 

Data

With the transmitted volume of data growing exponentially, all operators face challenges in providing a satisfying user-experience. Who manages to best meet the growing demand?

7915038

All four Spanish mobile network providers offer 4G/LTE to their customers. With its high data rates and low latency times, this cellular technology is clearly the best choice for mobile data communications. Building on this foundation, Movistar, Vodafone and Orange have engaged in a race for top speeds, having implemented “LTE Advanced“ or “4G+“ in their networks – an expansion stage that combines two or more carrier frequencies in order to achieve higher data rates. Yoigo does not join this race and concentrates more on enlarging its 4G footprint.

P3‘s testing takes both aspects into account. Fast troughput is rewarded by the scores for web page downloads, file downloads and uploads. By examining success ratios at the same time, the networks‘ availability and ­stability is also evaluated. P3‘s approach for testing Youtube playback recognises that this popular video service has introduced adaptive bit rates recently. This decision by Youtube to provide a better user experience, surrenders pixel resolution in favour of stable playback. As a consequence, besides success ratios, start-times and the absence of interruptions, the average value of the obtained video resolution became another important performance indicator.

es_categorycomparison_data_2016_englisch

Vodafone is data champion in the cities

es_tabelle_datacities2016_englischIn the large cities, all four Spanish networks have reached the most mature level of LTE coverage. This is distinctly recognisable as all four candidates reach their best performance values in this environment. Vodafone takes the lead in the data category in the cities. This time it is Movistar that follows at a close distance of only four points. Orange achieves the third rank with a gap of eight points or three per cent. The lag of Yoigo is distinctive with a gap of 48 points or 13 per cent. However, Yoigo‘s success ratios in the cities are still okay – for example, video playouts once started successfully (which worked in 97.3 per cent of the examined cases) did not show any interruptions at all.

So in larger cities, Yoigo may not quite reach the high performance level of its competitors, but it is still a viable alterna­tive when it comes to data communications. In smaller towns, Vodafone again takes the lead, but Movistar and Orange once more follow at close distance and both achieve iden­tical scores. However, the deficits of Yoigo become more obvious in the smaller towns than in the cities.

Yoigo considerably falls behind on roads

The four candidates give basically the same impression on the connecting roads: Vodafone is the strongest again, Movi­star ranks second and Orange third. However, contrary to the smaller towns, Orange does not quite manage to measure up to Movistar‘s level of performance and reliability.
Yoigo trails behind the competition once more. But while the smallest Spanish operator showed acceptable results in the cities and towns, its data performance on connecting roads is distinctly behind with a gap of 34 points or 27 per cent. And while the other three providers were able to improve on their results from the P3 connect Mobile Benchmark Spain 2015, Yoigo remained static at best.

 

es_tabelle_datatowns2016_englisch

es_tabelle_dataroads2016_englisch

 

DATA results at a glance

The data category shows a distinct ranking: Vodafone leads, Movistar comes in second, and Orange third. All three improved over 2015, with the exception of Yoigo. The smallest Spanish operator’s data performance stagnates and is particularly weak on roads.

 

Testing Methodology

The methodology of the P3 connect Mobile Benchmark is the result of P3’s many years of experience. It was carefully designed to evaluate and objectively compare the performance and service quality of Spain’s mobile networks from the users’ perspective.

7920572

The P3 connect Mobile Benchmark in Spain took place from 7th to 28th October 2016. All samples were collected between 8am and 10pm. The network tests covered larger cities, smaller towns and connecting roads. The combina­tion of test areas had been selected to provide a significant series of test ­results covering the Spanish population. The areas chosen for the 2016 test account for more than 11 million people, or 23.8 per cent of the total population of Spain.

P3 conducted the tests with four drivetest cars, equipped with arrays of Samsung Galaxy S5 Cat 4 smartphones (Voice) and Samsung Galaxy S7 Cat 9 smartphones (Data) for the simultaneous measurement of voice and data services.

Voice testing

Three boxes were mounted into the back and side windows of each measuring car in order to support twelve smartphones per car.

Three boxes were mounted into the back and side windows of each measuring car in order to support twelve smartphones per car.

Two smartphones per operator in each car were used for the voice tests, setting up test calls from one car to another. The audio quality of the transmitted speech samples was evalua­ted using the HD-voice capable and ITU standardized so-called POLQA wideband algorithm. All Spanish network ope­rators offer 4G capable subscriptions. To take the high share of LTE into account, speech samples were ­acquired ­partly in 4G preferred to 3G preferred mode and partly in 4G preferred to 4G preferred mode. As a consequence, the phones in most cases needed to switch (“fall back”) to 2G or 3G when they were logged in to the LTE (so called “circuit-switched fall back“ or CSFB).

In order to account for typical smartphone use scenarios during the voice tests, background data traffic was generated in a controlled way through random injections of small amounts of HTTP traffic. The voice test scores account for 40 per cent of the total benchmark results.

Data testing

Each box was housing four smartphones which allowed the simultaneous testing of four mobile operators.

Each box was housing four smartphones which
allowed the simultaneous testing of four mobile
operators.

Data performance was measured using one smartphone per operator in each car. The radio access technology was set to LTE preferred mode in order to reflect the customer experience. The web tests accessed web pages according to the widely recognized Alexa ranking. In addition, the artificial (static) “Kepler” test web page as specified by ETSI (European Telecommunications Standards Institute) for testing purposes was used.

In order to test the data service performance, files of 3MB and 1MB for download and upload respectively were transferred from or to a test server located on the internet. In addition, the peak data performance was tested in uplink and downlink directions by assessing the amount of data that could be transferred within a 10 seconds time period. Another discipline was the playback of Youtube videos. It took into account that Youtube dynamically adapts the video resolution to the available bandwidth. So, in addition to the success ratios, start times and playouts without interruptions, the Youtube measurements also determined the average video resolution.

All the tests were conducted with the best-performing mobile plan available from each operator. Data scores account for 60 per cent of the total results.

Routes and samples

The test routes are shown at the top of this website. In the 14 big cities and 26 smaller towns indicated, the cars had to follow predefined routes. Altogether, the four test cars covered more than 11,300 kilometres, of which approximately 4800 km led through the big cities, while 6500 km were covered in smaller towns and on connecting roads. The test routes also included the islands Tenerife and Gran Canaria.

Performance indicators and rating

data_voice_donuts2016_englischThe score weighting reflects both the geographical distribution of Spain‘s population and the ranking of usage scenarios. Therefore, 600 of the total of 1000 maximum points were assigned to the cities – 240 maximum points refer to the voice results and 360 maximum points reflect the data results. For the towns and the roads, a maximum of 200 points each is available. In both categories, the possible maximum is 80 points in the voice, and 120 points in the data category. The adjacent table shows the percentage of maximum points that each operator has achieved in each discipline.

es-score-breakdown-englisch

Conclusion

All four Spanish operators improved over last year’s results – although at different ­levels and ­areas of advancement.

Hannes Rügheimer Portrait_v2

Hannes Rügheimer, connect author

The total score development compared to 2015 shows the clear improvements of the Spanish networks.

The total score development compared to 2015 shows the clear improvements of the Spanish networks.

To those readers who are familiar with the outcome of 2015‘s P3 connect Mobile Benchmark in Spain, this year‘s results do not present many surprises: 2016‘s overall ranking looks exactly like the one 12 months ago. Vodafone is the clear winner, both in the voice and data categories. And, like all candidates, the winner was able to improve over last year‘s points. The biggest improvement could be observed in the data category, however Vodafone‘s score also grew in the voice category.

Movistar also manages to keep last year‘s position and ranks as a strong second candidate. Like the winner Vodafone, Movistar improves both in the voice and data categories. Its constant competitor Orange finishes only 13 points behind and even drew level with ­Movistar in the voice category. The third rank results from an overall weaker performance in the data category. Spain‘s smallest operator, Yoigo, brings up the rear. However, it improved visibly in the voice discipline but basically remained static in the data category. In the big cities, Yoigo achieved fair results. In smaller towns and on connecting roads though, it showed much room for improvement.

In an international comparison, the Spanish networks still score below the best operators in Switzerland, Austria and Germany. However, Vodafone España does better than its cousins in some of the other European countries. At the other end of the range, Yoigo is on a similar level as Telefónica‘s German O2 network.

es_totalscore2016_englischresults-spain_english