THE MOBILE NETWORK TEST IN SPAIN
Download test here
With a total of 50.7 million mobile subscribers, Spain is among the largest mobile network markets in Europe – and among those that are fiercely battled over. Telefónica’s Movistar dominates its home market, but Vodafone and Orange are constantly competing for the second rank. Yoigo comes in as number four, with the remaining percentage of market share split up among a number of mobile virtual network operators.
This highly competitive situation leads to increased efforts of all involved network operators in order to increase their market share and win customers from their rivals. This again leads to great efforts to improve network coverage, voice quality and data performance.
First independent network test in Spain
This situation makes the Spanish market particularly interesting for a closer look at the actual performance of its mobile networks. Therefore, in October and November 2015, Aachen-based P3 communications conducted its first mobile network test in Spain. The company, which is also being active in Germany, Austria, Switzerland, the UK, the USA, Australia and many other countries, is a world leader in mobile network testing.
As for many other European countries, connect, the biggest European magazine on telecommunication, has teamed up with P3 communications in order to analyze and publish the results of this benchmark. As a result of this joint effort, Spanish customers gather valuable information about the performance of the competing Spanish operators – and thus can make an informed decision, which network best suits their needs.
In order to benchmark the voice performance of Spanish networks, two P3 teams drove through a large number of big and small cities as well as the roads connecting them. Each car was equipped with eight Samsung Galaxy S5 smartphones that would permanently call each other, thus generating more than 55,000 speech samples per operator. In order to ensure realistic testing conditions, background data traffic was generated on the smartphones during a voice call. About half of the test calls took place between two phones that were set to “4G preferred“ mode, while the other half took place between one “4G preferred“ handset and one preferring 3G.
As the LTE standard originally was not designed to handle voice calls, in most cases the smartphones had to switch their connection back to 3G/UMTS or even 2G/GSM, whenever a call was received or initiated. This workaround is called circuit switched fall back (CSFB) and it takes additional time and can reduce reliabilty.
In July 2015, Vodafone introduced the more modern Voice over LTE („VoLTE“) in its Spanish network – as the first and so far only mobile operator in Spain. But as this was announced too short before the network testing started, all telephony tests including those of the Vodafone network were still based on the more widespread circuit switched fall back.
Voice in Big Cities
But Vodafone seems to master the older technology as well. In big cities, this operator achieves the shortest call setup times and at a close margin over Movistar and Orange also the best speech quality. In big cities this lead is very clear, however Movistar and Orange are within hitting distance. Yoigo‘s results, on the other hand, are clearly behind the top three. This is most noticeable regarding call success ratios, where Yoigo has a failure rate of 5.6 per cent (compared to the other candidates‘ values that range between 1.6 and 2.6 per cent) as well as lower speech quality.
Voice in small cities and on connecting roads
In smaller cities and on the connecting roads between them, the performance indicators drop – not suprisingly. In the rural areas with their lower network coverage, the differences between the strong three participants level out to some extent. Call setup times and speech quality came closer together in small cities and on connecting roads. Again, in the countryside Yoigo ranks considerably behind their competitors. Outside the big cities, out of one hundred attempted calls, 15 would fail in the Yoigo network. And those that could actually be connected show a comparably low speech quality. In the telephony discipline overall, Vodafone achieves a clear lead, Movistar and Orange rank in the middle field, and Yoigo definitely at the very end.
Movistar is the brand name that the Spanish telecommunications company Telefónica uses for the mobile network in its home market. Telefónica itself is one of the largest telco companies in the world and the second largest corporation in Spain behind the Santander Group. The operator is present in 21 countries with a total of 123,600 employees and achieved worldwide revenues of over 50 billion euros from January to December 2014.
While the company introduced the Movistar brand in Latin American countries only in 2005, it has been in use in Spain since the launch of GSM services back in 1995. Today, Movistar is the largest mobile operator in Spain with about 22 million customers, which equals a market share of roughly 42 per cent. It offers GSM service at 900 and 1800 MHz, UMTS/3G at 900 and 2100 MHz and LTE at 800, 1800 and 2600 MHz. Since the end of 2014, Movistar supports 4G carrier aggregation – which means that suitable smartphones can combine the two supported LTE frequencies for higher bandwidth.
Vodafone España has been present on the Spanish mobile communications market since the year 2000. Then, the British Vodafone Group acquired Airtel Móviles which has operated in Spain since 1994. In Spain, Vodafone nowadays has 14.2 million mobile customers, which adds up to a market share of about 27 per cent. In the fiscal year 2014/ 2015, Vodafone Spain achieved a turnover of 4.7 billion euros which contributes about seven per cent to the whole Vodafone Group‘s financial result. In 2014, Vodafone Spain expanded its fixed-line business by acquiring the cable operator Ono.
Vodafone‘s mobile network in Spain offers GSM service at 900 and 1800 MHz, UMTS/3G at 900 and 2100 MHz and LTE at 800, 1800 and 2600 MHz. The Vodafone mobile network in Spain supports LTE carrier aggregation of its 1800 and 2600 MHz frequency bands. The company offers LTE downlink speeds up to 300 Mbps and calls this service „4G+“. The so-called 4G+ coverage is available in 41 provinces and in all provincial capitals.
Orange España is the brand name of France Telecom‘s mobile network in Spain. It has been operating under this name since 2006. Previously, the network was known as „Amena“ – this brand name lives on in Orange Spain‘s portfolio as a low-cost offer, that is only available on the internet. Also, its network serves a number of Mobile Virtual network operators such as Carrefour Móvil, MasMovil and others. With 11 million customers, Orange is the third largest Spanish mobile operator – its market share sums up to about 21 per cent. In the fiscal year 2014, Orange Spain achieved a turnover of 3.8 billion Euros which contributes ten per cent to the whole Orange Group‘s 2014 financial results.
Orange Spain has deployed 2G networks at 900 and 1800 Mhz and 3G networks at 900 and 2100 MHz. Since mid 2013, the company also offers LTE at 1800 and 2600 MHz, supporting data rates up to 200 Mbps. By the end of 2015, Orange is planning to cover 85 per cent of the Spanish population with 4G, and 95 per cent by the end of 2017.
Yoigo was the latest mobile operator to enter the Spanish market. Founded in the year 2000 under the name Xfera, the company started its actual operation in 2006, offering only a UMTS/3G network at 2100 MHz. At this time, the Swedish telecommunications company TeliaSonera acquired the majority of shares and rebranded the network as „Yoigo“. This name was supposed to reflect the simplicity and ease in rates as well as in the use of the service. Yoigo has a national roaming agreement with Movistar which is valid at least until 2016. With currently about 3.4 million customers, Yoigo has a market share in Spain of about seven per cent and has shrinked in recent years. Current owner TeliaSonera is rumoured to be looking for a buyer.
In July 2013, Yoigo started the deployment of a 4G network at 1800 MHz. Since then, the company has expanded its LTE coverage to all major Spanish cities and most of the provincial capitals and now covers about 70 per cent of the Spanish population with its 4G network.
Can Spanish network operators keep the promise of high speed data access in metropolitan areas? At least, in the big cities their LTE coverage is best.
During their test drives, the two teams of P3 communications visited a total of 13 Spanish cities with more than 100,000 inhabitants. In these big cities, all four network operators have reached a mature level of LTE coverage. With its clear focus on high data rates and comparably short latency times, LTE is the most desirable standard when it comes to mobile data use. The real-life test scenarios (also see page 8) concentrated on accessing both live and static web pages as well as downloading and uploading files and watching YouTube videos in both standard and high definition.
Web page access
The important web site access was tested with a mixture of nine popular live web pages and one static web page. In big cities, each of the four operators shows a reasonable performance, with a close lead by Vodafone. However, Yoigo turns out to be a little less mature than the other three networks. While Yoigo‘s LTE coverage in big cities is still far better than in rural areas, even in the metropolises 4G service was not consistently available.
File uploads and downloads
This gap gets even larger when it comes to file uploads and downloads. The top three operators show success ratios well above 99 per cent, while Yoigo can only obtain lower ratios.
This is also true for the speed of the networks, although a little more differentiation can be seen. In order to emphasize this, the benchmark measures transfer rates in two different ways. With files of fixed size the overall speed of the network is tested including the time it takes to initiate a speedy data stream. This is important in cases where many small files will be transferred. On the other hand, the “10 seconds measurements“ show the peak performance once a stable transmission has been achieved. It gives insight on how the network behaves when large files like high-res photos or videos are uploaded or downloaded. The value given for „90% faster“ is what a customer can expect most of the time.
Especially when looking at the download performance, Vodafone and Movistar are in a league of their own, closely followed by Orange. Yoigo clearly scores worst in terms of data rates and reliability – although their customers should bear in mind that data in big cities is definitely Yoigos best discipline.
YouTube in standard and high definition
Today, YouTube amongst other video services is the major source of traffic in mobile networks. As customers expect to be able to access videos at any time and any location via their smartphones, operators should be able to deliver a reliable stream at least in standard definition (SD). Again, the top three manage to achieve this with success ratios well over 99 per cent. With 97 per cent, Yoigo still shows a good performance in this category. With the more demanding high definition (HD) videos, success ratios fall – and looking at Yoigo, the decrease down to 92.9 per cent is even more dramatic than within the other three networks. Interestingly however, when playing out HD videos, there was an even slightly higher quota of interruptions in the Orange network than in Yoigo‘s.
Close finish with one more distant contender
For data in big cities, Vodafone shows a strong lead, followed by Movistar and Orange. As the smallest operator, Yoigo follows at a distance – however still offers acceptable data performance to its customers.
While providers usually deliver good performance in the metropolises, smaller cities often lag behind. How noticeable is this effect in the Spanish networks?
Besides the 13 cities with more than 100,000 inhabitants, the test routes of the two measurement cars also led the P3 teams through a considerable number of smaller cities. The map on the first page of this report shows which of them have been taken into account for this benchmark. In total, the test drives covered about 17 per cent of the Spanish population which gives the P3 connect Mobile Benchmark
a high degree of statistic validity.
Web page access
Obviously, in smaller cities and rural cities, the broadband network coverage is less dense than in the metropolitan areas. So when looking at the success rates for accessing the nine live web pages and the one static test website, these results drop for each operator in comparison to the big cities. While this drop stays in the range of one to two per cent on the Movistar, Vodafone and Orange networks, it amounts to more than six per cent on Yoigo. This means for Yoigo subscribers that on avarage nine per cent of attempted web site accesses would fail. The decreasing success ratios go along with increasing session times. However, if websites actually succeed to download at all, the differences mostly stay within fractions of a second and thus are still bearable.
File uploads and downloads
Falling success ratios and decreasing data rates can also be observed when it comes to file transfers outside of the big cities. Regarding downloads, the three top networks Movistar, Vodafone and Orange obtain almost the same success ratios. But on average Vodafone delivers faster downloads than the two runner-ups. Yoigo is defeated with download success ratios of only 95 per cent. When uploading large files, the Vodafone network achieved a success ratio of 97.7 per cent, which is a clear lead over the 94 to 95 per cent results of Movistar and Orange.
Once more, Yoigo is defeated with download success ratios of only 95 per cent that drop dramatically to a mere 84.1 per cent in the 1 MB file upload tests. Along with this, the average data rates are lower than half of the values achieved by Orange and Movistar – and in most cases lower than a third of the average data rates on the Vodafone network. So customers interested in reliable data connectivity in smaller Spanish cities are served best by Vodafone, with Movistar ranking second and Orange ranking third. The same is true for the average data rates of uploads and downloads. Yoigo‘s network ranks considerably worse in terms of reliability and speed. The TeliaSonera subsidiary loses a large number of points of the total rating especially in this discipline.
YouTube in standard and high definition
The observations from the web page access and upload/download categories also apply when receiving YouTube videos. Again, in the SD video benchmarks, Vodafone turns out to be the most reliable network, with Movistar and Orange following closely. Yoigo shows a lower SD success ratio but is at level concerning playouts without interruptions. This highlights that Yoigos network coverage is not as good as its competitors‘ – but if a Yoigo SD video can be played back, it will very probably run all the way through without hickups. This is however not entirely true for the more demanding high definition videos. Here, the success ratios drop decidedly, with Movistar performing somewhat better than Vodafone and definitely better than Orange and Yoigo. In the Yoigo network, almost 20 per cent of attempted HD playouts failed, and almost nine per cent of them were interrupted.
Distinct ranking order
All in all, the data performance of the top trio suffers noticeably in smaller cities with a distinct ranking order of Vodafone, Movistar and Orange – and a considerable distance of Yoigo‘s performance.
DATA ON CONNECTING ROADS
Connected navigation and data services used in cars emphasize the importance of mobile network coverage on the road. How do Spanish networks perform there?
On their test tours, the two measurement cars passed a total of 5,600 kilometres on the connecting roads between Spanish cities – about 2,800 kilometres per car. While driving on these roads, the two cars permanently examined the coverage, reliability and data performance of all four Spanish mobile networks.
Web page access
On the roads outside of big and small cities, the Spanish market leader Movistar delivers the best performance. Movistar clearly leads with a ratio of 94.2 per cent of successful web page downloads, and in this discipline also Orange scores a little better than Vodafone. Yoigo comes in last, but its performance in this discipline is not considerably worse than that of Vodafone.
File uploads and downloads
Similar results can also be observed in the file transfer disciplines. Again, Movistar and Orange provide better success ratios here. If a download can be performed, Vodafone delivers slightly better average throughput rates. For file downloads and especially for uploads, Orange proves to be a strong competitor that ranks close to category winner Movistar and frequently outruns Vodafone which only achieves a third rank in this discipline. Once again, Yoigo scores fourth, but the lag is less obvious than in the telephony discipline or the data disciplines in big and small cities.
This general picture is confirmed by the video tests, where Movistar (fittingly – considering the double meaning of its name) scores best for both standard and high definition video playbacks, with Orange and Vodafone turning out as runner-ups and Yoigo falling not too far behind.
Movistar wins in the road category
The results in this discipline strongly nurture suspicions that Vodafone has been concentrating its network deployment to the cities. Spanish customers who need the best performance on connecting roads should definitely take a closer look at the offerings of Movistar.
The network tests in Spain were conducted in October and November 2015. As in Germany, Austria, Switzerland, the UK and the Netherlands, P3 communications, connect’s partner for executing network benchmarks, sent out two measurement cars to perform the tests. The cars covered a total distance of 17,000 kilometres (about 8,500 kilometres per car) through big and small Spanish cities and the connecting routes between them. The route is shown on the first page of this report. It covered about 10.2 million inhabitants or roughly 17 per cent of the Spanish population.
Each car was equipped with eight Samsung Galaxy S5 and four Samsung Note 4 smartphones. Eight of them where Cat3 devices and used for the voice tests, the other four belonged in the Cat6 category and were used for data measurements. Whenever possible the smartphone’s firmware matched the network operator’s genuine ones. In cases where no operator specific firmware was available the most recent Samsung firmware was used.
Data and voice services were executed with separate smartphones. Telephony was tested mobile-to-mobile from one car to the other. In order to ensure realistic testing conditions, background data traffic was generated on the smartphones during a voice call. The audio quality of the transmitted speech samples was evaluated using the HD-voice capable and ITU standardized
so-called POLQA wideband algorithm. All network operators offer 4G capable subscriptions for everybody interested in a high performance data network. To taket he high share of LTE into account, speech samples were acquired partly in 4G preferred to 3G preferred mode and partly in 4G preferred to 4G preferred mode. As a consequence, the phones in most cases needed
to switch (“fall back”) to 2G or 3G when they were logged in to the LTE service before the call (circuitswitched fall back, CSFB).
The testing of data services was done with four separate Samsung smartphones in the cars. As a first measurement task several popular live web pages were browsed using the built-in smartphone browser. The web pages had been selected previously according to the Alexa ranking. In addition, the artificial (static) “Kepler” test web page as specified by ETSI (European Telecommunications Standards Institute) for such testing purposes was used.
In order to test the data service performance, files of 3MB and 1MB for download and upload respectively were transferred from or to a test server located on the Internet.
In addition, the peak data performance was tested in up- and downlink direction by assessing the amount of data that could be transferred within a 10 seconds time period. While the 10 seconds peak throughput test focuses on network capability, the YouTube tests add a more end user centric view to the test curriculum. Here, videos with standard definition (SD, 360p, 2.7MB, 30s) and high definition (HD, 720p, 11.9MB, 30s) were played using the YouTube player on the smartphones.
Route and Samples
In big and small cities the cars had to follow pre-defined routes. Altogether more than 55,000 speech samples were logged per operator, about half of them in 4G preferred to 4G preferred mode, while the other half in 4G preferred to 3G preferred. For the data benchmark about 50,000 samples were obtained per operator. About 60 per cent of the samples were obtained in big cities while the remaining 40 per cent were collected in small cities and on connecting routes.
A new approach to measure mobile networks
Previously, it was the usual approach to conduct mobile network drive tests with the measurement devices mounted in the car and connected to antennas in a roof box mounted on the vehicle. In accordance with ETSI TR 102 581 an attenuation of the roof antenna signals of -12 decibel was used to compensate for the antenna gain (+ 5 dB) and to simulate a typical attenuation (- 7dB) of a mobile phone used indoor. With modern smartphones having their orthogonal MiMo antenna arrangements used for HSPA and LTE and with the use of more intelligent antennas changing their directivity and tuning in relation to the signals received, a new approach is better suited to reflect real life conditions. This new approach is a highly optimized mounting case for the smartphones – the so called P3 Antenuatr. It is designed with unique materials selected for their specific absorption rate at high frequencies between 800 MHz and 2,6 GHz. By choosing the right selection of materials in the right places, an attenuation rate typical for indoor measurements can be achieved without altering the antennas of the smartphones. The validity of this approach, for which a patent is pending, has been confirmed by comparing measurements between unaltered phones. The legacy approach with antennas in a roof box and attenuators has been further verified by comparing TIS (total isotropic sensitivity) and TRP (total radiated power) according to the CTIA test plan for over the air measurements in the Testlab owned by WEKA Media Publishing, the parent company of connect. These measurements confirmed an attenuation rate very close to the wanted – 7 dB from LTE band 20 (800 MHz) to LTE Band 7 (2,6 GHz). The variation is even smaller than with external antennas connected over wideband attenuaters.
Three of these antenna boxes were mounted in each of the two cars in the rear side and in the rear windows. One Antenuatr was responsible for 4G preferred to 3G preferred voice measurements, while the second was used for 4G preferred to 4G preferred voice measurements. The last Antenuatr was responsible for the data measurements. Real live testing confirmed the advantage of using the differently polarized original MiMo antennas, as this increases the likeliness of achieving maximum data rates due to the higher linear independence of the antenna signals.
Hannes Rügheimer, connect author
And the winner of the first P3 connect Mobile Benchmark Spain is … Vodafone. This operator achieves a clear lead in voice and a slight advantage in the data scoring. Regarding data, we actually observed a close photo-finish between overall winner Vodafone and the incumbent operator Movistar who is almost equally strong in connectivity. On the connecting roads outside of the cities, Movistar turned out to be the strongest contender, and Orange generally performed stronger than Vodafone. Yoigo ranks clearly behind the top three, most noticeably in speech quality and call success as well as in the data categories. Generally, Yoigo achieves its best results in big cities and distinctly falls back in rural areas. Interestingly however, on connection roads Yoigo’s lag behind the other competitors is less obvious. In comparison to the drivetest results of the other recent European benchmarks, the Spanish networks all in all rank in the same league as most of their counterparts. Spanish operators do worse than those in the strong alpine countries Austria and Switzerland, are about level with Germany and on an average perform better than UK networks. In this international comparison, Yoigo’s performance is on a similar level as the Telefónica-owned networks (O2 and E-Plus) in Germany and the Telefónica mobile network in the UK.