Cross-browser testing

With the exponential pace of technological advancement, businesses strive to get the best out of it and to ensure their web app runs smoothly on all browsers and platforms. As an end user’s attention span continues to shorten, so will he or she not hesitate to press the back button on their browser if a site seems faulty. So, what’s the solution? Make web apps and sites work flawlessly on every browser, device, and platform. It sounds like a simple and straightforward goal, right? If a business doesn’t wish to lose customers due to less than optimal UX, cross-browser compatibility and cross-browser testing must be considered. 

So, what is cross-browser testing and what should we know about it to assure the quality of our websites?

Cross-browser testing is the practice of making sure that the websites and web apps created work across an acceptable number of web browsers.

Why do cross-browser issues occur?

As a matter of fact, there are many cross-browser issues that have nothing to do with bugs in a website’s code. One of them is the different interpretation of Open Web Standards by browser vendors, meaning that HTML, CSS, and JavaScript are rendered in unique ways owing to different levels of support for technological features and the differing implementation of those features.  Even though browsers are much better at following standards these days (for example using the same Chromium rendering engine by Chrome, Safari and Edge), differences will still creep through. But yes, bugs in browsers can of course be another reason, and these may occur in new versions of browsers implementing the new features and standards. To check whether the problem is with the website or the browser, website validators may be used (e.g. W3C for CSS and HTML validation) or browser Devtools for source code validation.

Examples of common bugs in code

• Lack of a Valid HTML/CSS:

No <div> code, for example, in which case it can be difficult even figuring out what’s wrong in the browsers that do not automatically add it.

• No CSS Reset:

Resetting a website rendered is needed to unify the way it is presented to the user and to make sure all customised CSS styles won’t be overridden by the browser's own set. 

• Vendor-Specific CSS Styles (-moz, -ms, -webkit, -o):

To ensure a hassle-free working of code in different browsers, the unprefixed version should be added along with the prefixed version.

• No Doctype or Incorrect Doctype in HTML Files:

The first line of the HTML code has the DOCTYPE tag which helps create a smoothly rendered site. A mistake there can cause many problems, such as hindering a page’s proper loading.

• Layout Compatibility:

Layout compatibility issues generally happen when the default browser design is removed using CSS resets. This could be due either to non-responsive web applications or to a lack of support of the design on a specific browser and browser versions.

Workflows for cross-browser testing

The workflow for cross-browser testing on a project phase can involve the following steps:

1. Identify features to test and specify scenarios.

2. Establish the baseline: design and functionality tests should be run on the primary browser before beginning cross-browser testing (usually on Chrome due to its huge market share). This way, you’ll know how the website is supposed to look and behave.

3. Identify the browsers and platforms you’ll be testing on. Usually, the full list of supported browsers is defined at the beginning of a project.

4. Pick how you’ll execute the test scenarios, manually or automatically. You can use manual tests for exploratory testing to discover UX pain points a user might encounter when engaging a touchpoint. The rest of the tests can be automated for quick, repeatable execution and near-instant feedback. A single test script, written using automation tools, can execute a test scenario on multiple browsers as many times as is required, provided your tool/framework can support the browsers in the scope.

5. Set up devices/browsers you’ll test on, or connect with a cloud-based provider: To account for website behavior on different operating systems, you’ll need either to set up a testing infrastructure using emulators/simulators/virtual machines (VMs) and install browsers on them for testing or set up a device lab of your own on real devices to maintain their integrity over time. Alternatively, a cloud-based testing infrastructure can be used to run tests on a remote lab of secure devices and browsers.

6. Execute test scenarios on browsers with the highest share of traffic, then move on to outliers.

7. Document and share the test results with a team.

8. Continually run cross-browser compatibility tests.

Depending on your role and workflow, you could be running cross-browser tests:

• During development: Developers in Continuous Integration pipelines test new features to make sure they’re cross-browser compatible before passing the changes on to production.

• In staging/pre-release: QA teams do this for every Release Candidate to make sure that no browser compatibility issues crop up in the latest version of the website.

What features are analysed during a browser test

To do cross-browser testing effectively, product teams constrain their testing with a test specification document that outlines a list of features to test, browsers/versions/platforms to test on, test scenarios, timelines, and budget.

It is a good idea to categorize features that will undergo testing. For example:

• Base functionality such as dialog boxes and menus, form fields validation, handling first-party cookies, as well as features that depend on them like personalisation etc.

• Design: the website’s appearance — fonts, images, and layout — must match the design.

• Accessibility: Accounting for compliance with Web Content Accessibility Guidelines (WCAG) to enable differently-abled users to access the website.

• Responsiveness: Verifies that design is fluid and fits different screen sizes/orientations - mobile consistency, in particular.

• Browser performance: Variances in some browsers with regards to the user interface or processing performance. 

• Code: The CSS and JavaScript validation across all target browsers.

How to choose a browser

Once the required feature-set, initial design and stack of technologies are determined, the question about the target audience should be raised — what browsers, devices, etc. will the target audience be using? The sheer number of browsers, devices, and OS make it impossible to test all combinations. A more realistic goal is to focus on maximizing the website’s reach within the target market. The client might already have data about this from previous research they've done. If not, it’s a good practice to showcase the more critical browsers and versions based on:

• Popularity: Choose the most popular browsers and platforms. Their number depends on the project's budget, time and goals. This is to maximize your reach in any target market. Information on popularity can be found in different searches via analytics tools like Google Analytics or Kissmetrics. However, do keep in mind that this only provides a sample of website visitors as well as potential visitors.

• Analysis: Look at your website’s traffic stats as captured by analytics tools and break them down by device/browser. This way, you can find out which browser-OS combinations are most commonly used by your target audience and what devices the website is generally viewed on.

• The field of the project: Some projects may require a different approach according to the project field. For example, a company wants to provide an application for a closed set of supported devices within the company. This makes things much easier in terms of testing, whereas a project in the field of education may prove more of a challenge when requiring support for deprecated versions of browsers owing to a school’s technical limitations.

In such cases, choose browser-OS combinations that are most popular with your end-users. (Commonly, these will be ones getting over a 5% share of traffic.)

Once a range of browsers and devices is agreed upon with the project owner,  the website should be coded defensively to give other browsers an optimum capacity to use the website content.

Browser-testing tools

There are commercial tools available such as Browserling, Sauce Labs, BrowserStack, Endtest, LambdaTest, TestingBot, and CrossBrowserTesting which allow QA to test websites and web apps on different combinations of browser-OS without having to worry about the setup. It is also possible to set up an environment that automatically runs tests, which allow you to check in your changes to the central code repository only after the tests are passed.

Our approach and recommendation is to use the BrowserStack tool for cross-browser testing. As a general method we test on the latest versions of the four most popular browsers, however, this is still dependent upon a project’s needs. At Startup House, we’re always keen to share the wisdom, so if you’d like to know more, don’t hesitate to contact us: hello@start-up.house