Thomas Cook beat online consumer expectations with thinkTRIBE load testing
Thomas Cook is one of the best known names in travel, with over a hundred years of trading history, more than 800 high street stores, a leading website and apps, ownership of some of the world’s favourite travel brands and various services related to travel such as insurance and foreign exchange.
Challenge
Confidence that a site will be be “stable under load”, that it will continue to perform as it should when there is heavy user traffic, is crucial. For travel companies that experience incredibly high numbers of visitors at different seasonal peaks, and that rely on fast moving realtime data, this is of particular concern.
In the past year Thomas Cook ran a number of different tests across different sites, brands, and countries with thinkTRIBE, as well as tests that focus on particular functions. There was close co-operation between the Thomas Cook and thinkTRIBE teams at all stages of the process, from first identification of the test objectives and user journey mix, through live conference call monitoring of the test, to in-depth post testing discussion.
In the testing the previous year, before thinkTRIBE was involved, the selections were not weighted and Thomas Cook found this caused availability issues and negatively impacted the results.
Solution
The targeting / simulated load in tested by thinkTRIBE in 2011 was not random but used specific and historic data to ensure popular destinations (which involves more complex search results) were accounted for. The thinkTRIBE approach to realistic load testing involves concurrent virtual users taking a number of different dynamic user journeys through the site, based on an algorithmic weighting of proportional site usage calculated from real historic data and future projections. This kind of planning is extremely important if test results are to reflect how the site really performs outside of controlled test conditions. Traditional load testing that throws huge numbers at the URL, or at one path through the site, will only tell you how that one specific aspect works in isolation. In reality there are many different users doing many different things concurrently on any site.
Another thing the team was conscious of was ensuring that tests took place over an adequate length of time. The danger of a short test is that is easy to draw false conclusions from what are extreme behaviours if they are seen in isolation and deemed to be “normal” site performance. With all this in place the thinkTRIBE Load Test Engineer was able to highlight lots of detail on how the site performs under load: which Journeys and which steps slowed down the most, which threw the most errors, what the errors
were, and, by comparing and contrasting the various errors, provide an indication of network problems, server load issues and so on.
The Result
“This combined approach revealed a number of system weaknesses and threshold levels, a database bottleneck and an unexpected memory leak that were not visible during non peak usage. Discovering these things in testing meant that pre-emptive action could be taken to adjust configurations on key web and app layers, as well as the code adjustments in the core base code and amendments to the platform. This meant that by the time the seasonal peak hit the site was able to easily cope with the demand.”