Service Level Agreement (SLA) weaknesses fall into two categories: weak guarantees and weak enforcement. An example of a weak guarantee is “less than one per cent packet loss.” In some situations, one per cent seems quite small (for example, “One per cent of students got an A in the course”). In others, one per cent is huge (“One percent of the tires were defective”). On the Web, one per cent packet loss is quite large; two lost packets for the 200 packets that a typical home page requires. Unfortunately, each lost packet tends to have a noticeable performance penalty as a result of factors such as network timeouts and slow paths through protocol code. So if your Internet Service Provider (ISP) delivers anything close to one per cent average packet loss, look for a new ISP.
I found other examples of weak guarantees in ISP availability metrics. One vendor promised 100 per cent availability but defined an availability incident as an hour or more of 100 per cent packet loss. So 59-minute outages and periods of 90 per cent packet loss aren’t availability problems for this SLA.
Scope limitations are a third example of a weak guarantee. Most ISPs will guarantee round-trip performance only within their networks. However, most Web user requests must traverse multiple major ISPs to reach your servers, as the largest providers host less than 50 per cent of end users.
In fairness, intranetwork performance is the only thing ISPs directly control. Beware of promises about performance across the entire Internet - they are an example of weak enforcement. One SLA had a decent performance guarantee for domestic US traffic (55-msec roundtrip time) but weakened it with enforcement based on traffic from automated test clients located within the provider’s network. The result is no more of an end-to-end guarantee than the intranetwork promise.
The most common enforcement problems arise from statistical treatments applied to SLA metrics. Performance and packet-loss guarantees commonly are based on monthly averages. Note, however, that the Internet is exceptionally fast during off-peak hours. A few hours of poor performance during the day disappear when statistically blended with hours of excellent performance at night. So a service provider with bad performance could satisfy what seemed like a tough SLA. Conclude that SLAs based on monthly averages are weak.
In defense of the service providers, SLAs are weak for a good reason: the Internet is a federation. Perhaps a weak promise is better than no promise at all. My advice: demand an SLA from your ISP, then read and understand it. Your service provider should know you understand the SLA and will be looking for a new provider long before the SLA terms are invoked.