When traditional load testing just isn’t enough (updated 19 Nov 2014)
Today’s online car tax web failure left people waiting up to 90 minutes to complete payment for their car tax online. Evidence that you need realistic, thorough load testing and good scalable website architecture, in advance of peak usage.
Clearly the DVLA were caught out when an extra 30,000 people visited the site, compared to the same day last year. Without doubt they carried out some load testing prior to the new system coming in to force, but clearly the testing wasn’t thorough or realistic enough to prevent this situation.
Update 19 November 2014
Judging from the DVLA website’s own stats, the level of online licensing in October, was pretty much the same as September and every month in the past: unless I’m mis-reading their graph. This graph says that the percentage ratio of digital to total renewals rose ever so slightly to 61% in September and 65% in October
And this 2nd graph shows that the absolute number of digital renewals did spike in September at 2.8M, but overall was only about 10% higher than the previous biggest month of February at 2.5M – so no big jump really: as far as total traffic across the month: did DVLA’s planning really not imagine that the extra 10% may all bunch to the last days of the month? Why was 30,000 visitors a problem when the site had already achieved in February 2.5M over the month – which meant the site could handle as a minimum 2.5M/28 per day; that’s nearly 100,000 per day. And in reality February too would have had peaks and troughs so that 100,000/day figure would have been exceeded for sure some days.
Was it so hard to plan for a mere 30.000 extra visitors per day on a site proven already to do over 100,000 day?
https://www.gov.uk/performance/tax-disc/volumetrics
Original blog article.
It’s true to say that nobody could have predicted the volume of traffic on the website, there’s always an element of guesswork involved with something totally new like this (car licencing going paper-free).
The Daily Telegraph quote Deri at thinkTRIBE:
Deri of website performance monitoring company thinkTRIBE, said with modern technology there is “no excuse” for sites to crash in these circumstances. “It’s true to say that nobody could have predicted the volume of traffic on the website, there’s always an element of guesswork involved with something new like this,” he said. “But with today’s elastic cloud technology there’s really no excuse. With a well-designed site it’s possible to scale up a website as and when you need it, even for short periods of time.”
With a well designed site it’s possible to scale up a website as and when you need it, even for short periods of time. At thinkTRIBE we are seeing more of our clients moving toward clever scaling of their websites – and carrying out load testing that is specifically testing that cloud auto-scaling elasticity is working as intended, as traffic shoots up.
( it’s not always entirely easy : Auto scaling cloud apps is not easy )
With today’s complex and dynamic websites you need load testing which can truly replicate what your users will be doing; in this case filling in forms, making selections, entering payment details. Traditional load testing looking at how many visitors a page can handle is no longer enough. You need to test what real users will be doing on the site, to find ‘the size of your shop’ ie. how many users can complete a transaction at one time. The DVLA apologised for the disruption, and advised people who need to renew their tax urgently to do so at a Post Office instead – rather embarrassing in this massively online, mobile 21st century! A wake up call to all website owner’s that for today’s websites, load testing can’t just be ‘good enough’.
Don’t think it’s just government sites that are vulnerable to this type of problem. At thinkTRIBE we test from the biggest to the smallest websites across numerous sectors; including companies such as Clarks, Cineworld, Sage, Channel 4, the Royal Albert Hall, Joules, Debenhams and RSPB and we see that every industry sector and type of site or technology is potentially vulnerable.
For a government site it’s only embarrassing, they won’t lose revenue but for commercial organisations the repercussions of a failure at peak time are much greater, potentially resulting in lost sales and damage to the brand.
So next time you need to load test just be sure you know how thorough and realistic your load testing and that your site architecture is clever enough to be scalable.
For help managing load testing, that overcomes the complexities of modern websites download our eBook: How to Plan a Successful Load Testing Programme.
Related Stories