What to do when your website becomes too successful

"Build it and they will come", as the saying goes, and this mantra is repeated to many companies that have built websites. However, for some, a website can become too successful and groan under the weight of traffic.

This sounds like a nice problem to have, but if it means your customers are having a poor experience when the servers can't cope, action is necessary.

"While large traffic spikes are something most website owners don't need to worry about, it can and does happen – such as after a successful newsletter, popular article or viral video," says Karl Mendez, managing director at CWCS Managed Hosting.

"However, if a web server is overloaded and causes temporary disruption, it can do more harm than good for business. After all, online users expect websites to load quickly and, if they don't, they'll (more likely than not) go elsewhere."

This strain can happen around holiday time or other special events. The run-up to Christmas is an obvious example but other times of year can catch out website owners.

"Popular consumer events such as Black Friday are a prime example of when retail websites that see too much success in one go struggle to manage," says John Beechen, head of managed services at Salmon.

"As a result, websites can experience outages, slow browsing due to degradation in site loading times, and on retail sites queues that are implemented to cope with the high volumes can cause frustration for customers."

Monitoring the situation

It is all too easy to blame lots of users, but often problems are caused by not addressing issues before they manifest as the traffic increases.

Paul Griffiths, technical director at Riverbed Technology, says that website failures are often a result of a number of network performance issues, such as poor change management or simply lack of capacity on the hosting platform.

"If a company doesn't have the monitoring and diagnostics systems in place to detect where the issues lie, it can take much longer to resolve and have the website back online as normal," he says.

In today's competitive environment, it's vital for a digital business to ensure that its website can scale in-line with the company's growth, or the firm can quickly become a victim of its own success. There are a number of key steps that businesses can take in order to do so, according to Michael Allen, Solutions vice president at Dynatrace.

"Load testing is key amongst these measures, enabling businesses to see how the website will perform under heavy traffic before it hits," he says.

This is becoming critical, as research has shown that application delays and failures during high demand periods directly affect revenue and brand reputation – 75% of all smartphone/tablet users will abandon a mobile site or app that is buggy, slow or prone to crashes, whilst 42% will complain on social media about a poor online experience.

"However, it's also important to continuously monitor website performance and maintain real-time visibility into the end-user experience. This will enable businesses to intervene in the event that users begin to experience difficulties, reducing abandonment rates and potentially lost revenues," adds Allen.

Capacity planning

Another way for firms to ensure their websites are up to the job is by the use of capacity planning tools that help in preparation for peaks and troughs in demand on designated national shopping days such as Black Friday and Cyber Monday.

"Add in the layers of complexity driven by mobile consumers, and many organisations are finding that applying advanced operational analytics to their capacity planning – using real data as the basis for building contingency plans – gives them the confidence needed when making fundamental decisions to support peaks in demand," says Peter Duffy, CTO at Sumerian.

Duffy says there is no reason why companies and customers alike should continue to suffer due to outages caused by poor IT capacity planning.

"Companies need to be ready to support growth, and in the age of online retailing, that all begins with knowing exactly what capacity headroom they have available, and having the ability to model different scenarios of increased demand on existing and hypothetical infrastructure. This enables them to know exactly how much traffic they will be able to cope with, and plan accordingly," he says.

Clouding the issue

Mendez says that some firms should take advantage of cloud load balancing. He says this enables traffic to be distributed across multiple virtual cloud servers that are within the same data center, or across multiple data centers for additional redundancy. "A load balanced cloud means web hosting can be extremely robust and flexible, increasing the reliability – and therefore the user experience – even as a website grows."

Another way to avoid putting pressure on the system is to let someone else take the heat and use a content delivery network (CDN), with queuing systems being used like a safety valve.

"These redirect visitors to a third-party website until the main site has the capacity to deal with them," says Alex Painter, website performance consultant for NCC Group. "Instead of a blank screen or error message, visitors get meaningful feedback while they wait. This is good for delivering a relatively seamless experience and also reduces the extent to which traffic problems are exacerbated by frustrated users hitting the refresh button."

Some outages are inevitable, but if a business takes steps to test and optimize its site, the firm can at least take some comfort in the knowledge that it wasn't the architect of its own demise.

Use specialized databases

One of the biggest factors that brings a website down to its knees is its database. Traditional databases were not designed to handle the volume of data that popular websites experience these days.

To remove performance bottlenecks, one of the first things you should look at is to switch to a NoSQL database, which are designed specifically for handling big data. Their biggest advantage is that NoSQL databases will help you store large volumes of data that might have little to no structure. They’ll also enable you to add new data types as per your changing needs.

Using a NoSQL database will also help you make the most of your cloud computing architecture. There are several types of NoSQL databases and you should spend some time evaluating them before zeroing on one that meets your requirements.

If you use MySQL, you don’t necessarily need to migrate to a NoSQL database. Instead you can use Vitess, which is an open source database clustering system. With Vitess you can scale your existing relational database to handle the increased inflow of traffic.

Some outages are inevitable, but if a business takes steps to test and optimize its site, the firm can at least take some comfort in the knowledge that it wasn't the architect of its own demise.