Visitor Abandonment
Page Load Time	Percent of Users Continuing to Wait
10 seconds	84%
15 seconds	51%
20 seconds	26%
30 seconds	5%

What You Need To UNDERSTAND: You will lose nearly half your visitors if they have to wait longer than 15 seconds for a page to load. Only 5% of visitors will wait longer than 30 seconds.

In 2008, where most of the population is on broadband, I expect that visitors are less patient than ever.

Update:  2006 data, quoted from http://www.avactis.com/forums/index.php?showtopic=238 : “The research shows that four seconds is the maximum length of time an average online shopper will wait for a Web page to load before abandoning one retail site and moving on to another.”

• Block All Cookies
  Blocks all cookies from all web sites from being accepted, and won’t send any existing cookies.Should be renamed “Unusable”.
• High
  Blocks all cookies from websites that do not carry a compact privacy policy (P3P) and cookies that contain personally identifiable (contact) information. A November 2007 study shows that only about 4% of sites use P3P, so this security setting is almost as unusable as “Block All Cookies”.
• Medium High
  Same as “High” for 3rd party cookies. Also blocks first party cookies that contain personally identifiable information.
• Medium
  As above, but rather than “blocking” first party cookies that contain personally identifiable information, it only “restricts” them. Efforts to find the difference between “block” and “restrict” have so far been fruitless. It may mean that cookies are accepted but not sent (how useful would that be?), or that such cookies can only be used in the same web page that created them (i.e. a restriction on the domain/path components of the cookie), of that the cookies are not kept beyond the current session.
• Low
  Same as “High” for 3rd party cookies. No restrictions on first party cookies.
• Accept All Cookies
  No restrictions.

In contrast, Safari (MacIntosh) allows the simple options: Accept Cookies Always/Never/Only sites you navigate to. (i.e. Always/Never/Only First Party). Firefox by default allows all cookies except where specific exceptions have been defined. There do not seem to be any Firefox extensions which emulate the IE or Safari behaviour – which perhaps places into perspective the real threat that third party cookies are(n’t) in general.

References

P3P Usage Survey

This has been my long-standing mantra. Any bit of software – be it a website or other program – if it hasn’t been tested against certain inputs, then chances are there’s a bug in there that has never been uncovered. Browser nuances and incompatibilities are notorious for making sites look ugly, behave badly or simply not work at all. So unless you have tested your website against all of the major browsers, you can’t be sure that there isn’t a problem with the site causing some of your users to just go away in disgust.

It is impossible to test your website against every web browser out there…

In a recent test, more than 30,000 different “user agent” (browser) identifiers were found in a sample of one month of data from one site – and that excludes search engine crawlers and other Internet robots. Most, in fact, are Internet Explorer (IE) variants of some sort, with different combinations of plugins. Here is a typical identifier:

  Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; InfoPath.1; .NET CLR 2.0.50727)

‘Mozilla/4.0′ identifies the browser’s basic capabilities. MSIE 6.0 is the browser version. Windows NT 5.1 is the operating system (though it is probably really Windows XP). SV1 is “Security Version 1″ as introduced in Windows XP Service Pack 2. InfoPath is a Microsoft Office plugin. .NET CLR … is the version of the .NET Common Language Runtime that the browser supports.

This is just one example; other browsers identify themselves as Mozilla/5.0, or MSIE 7.0, or Windows NT 5.0, or with other plugins/toolbars, or with multiple versions of .NET CLR, and so on. An then there are the other browsers – Firefox, Safari, Opera, Konqueror, and many more. There is a combinatorial explosion – and it is impossible to test every single possibility.

How much testing do you need to do?

Here is the mix of browser types from three different types of sites

• Site A: a B2B site in the electronics area, likely to be dominated by technically savvy male users, looking for products for immediate purchase
• Site B: a general homewares and gadgets type site, representing a broad cross-section of home users
• Site C: a women’s fashion site

Clearly, testing against IE 6.0, IE 7.0 and Firefox 2.0 is mandatory, these being the dominant browsers in the market. But what about Safari? It could be up to 5% of your market that you are losing if your website does not work well with Safari. And what about all of those variants – could your site test satisfactorily for IE 6.0 but be broken if the InfoPath plugin is added to the browser? Even considering the most common browser/platform/option combinations results in a substantial number of variants, and every extra platform required for testing is an additional expense – both to maintain the test suite and to run the tests on each update. Is there another way?

A Statistical Approach

Perhaps, by looking at the distribution of user agents for all the visitors on a website, and then looking at the same for those visitors that convert, it would be possible to identify any browser types that are not converting at the expected rate. This might indicate a problem with the site for that particular browser type.

This of course assumes that users are equally likely to convert regardless of which type of browser they are using. As it turns out, this assumption is completely wrong.
The following table compares browser strings at the website point of entry with browser strings at the point of transaction for site B and shows that the difference is significant.

On Site B, users are generally spending their own money; one might speculate that Firefox users represent a younger (poorer) group of users and therefore convert at a lower rate, and perhaps IE 7.0 users are either cashed up and have just bought Vista, or are people who feel more comfortable transacting on the Internet because they know their computer is running the latest up-to-date software. The statistics for Site C show Firefox users converting at half the rate of other users; in contrast, for Site A (the B2B site where users are probably spending their employer’s money, and where browsers are governed by company IT policy) the conversion rates are fairly constant across all browser types.

However, the basic idea of detecting problems using statistics is sound, providing two points of comparison can be found where the assumption holds that the distribution of browsers is the same. For example, any two points within the https conversion funnel, or any two product pages, or a product page and a category page.  Or, you could take one page and compare it against the statistics for all pages in a similar section of the site.

The analysis proceeds as follows:

1. Take your two data sets – the HTTP request logs comprising at least the user agent strings – one data set for each URL to be compared, and calculate the frequency statistics for the user agents in each data set. Perl is an ideal tool for this analysis. You will probably need about a month of data to get reliable results.
2. Where the frequency in both data sets for the same user agent is less than about 10, add this to a user agent called “Other” and throw away the original data. Where the frequency falls below about 10, the assumptions of the chi-square test start to fail, and since this is a distribution with a very long tail, the tail errors can end up dominating the result.
3. Perform a chi-square test to see if the distributions are different. If they are, keep going to find out where they are different.
4. Place the resulting data in a spreadsheet – frequencies of each sample, expected values, maximum errors between expected and actual frequencies (in absolute and/or relative terms). Discard any samples where the expected values are too low to be worth considering.
5. Sort the result by error descending. The browsers appearing at the top are causing the most trouble.

In Action

This approach was applied a tracking code implementation. A random sample of users received a cookie. The presence of the cookie was then tested on a particular page. The expectation was that the distribution of browsers should be the same with or without the cookie. The results, however, showed that the distributions were different, and the final analysis pointed to the Safari browser running on Mac to be the problem per the picture below. Tests against other sites using the same tracking code confirmed the result. Note, though, that the site wasn’t completely broken on Safari, as some users had managed to acquire the cookie. This is the type of problem that testing directly with a browser may or may not have picked up – depending on what settings the tester’s browser happened to be using.

Summary

Test, validate, test, validate, then validate some more. It is absolutely necessary to test your website with the more popular browsers, but not always sufficient. User agent statistics provide an additional test, potentially a stronger test and definitely a cheaper test compared to manual testing, for browser-specific problems.

References:

http://blogs.msdn.com/ie/archive/2004/09/02/224902.aspx