Graphical validation of web elements
WebDriver is dedicated to validate web pages from the HTML point of view, so we are checking if desired HTML tag is correctly located in the DOM tree and if it has a correct attributes. From time to time we need to validate a little bit more. One of examples is to validate if e.g. uploaded image is correctly scaled and cut - in short if it is displayed as we expect. In this article I will describe how we can do that with use of java and Selenium WebDriver.
Most complicated part of this task is to compare the expected image with this just created screenshot. The problem is that images are compressed and I can't promise that always image will be compressed on the same way (even if it looks same). If we want to be nearly sure that we compare the images that shouldn't differ on those small differences we have to convert them to binary images (black and white). Then we can say that we compare the shape on the image. To be able to do that we need to create a few methods which are located in the class below.
package com.testcraftsmanship.e2e.model.utils.image; import org.fluentlenium.core.domain.FluentWebElement; import org.openqa.selenium.*; import org.openqa.selenium.Point; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import javax.imageio.ImageIO; import java.awt.*; import java.awt.image.BufferedImage; import java.io.File; import java.io.IOException; import java.net.URL; import static com.testcraftsmanship.e2e.model.utils.text.ObjectsParser.getSelectorFromWebElement; public final class ScreenManager { private static final Logger LOGGER = LoggerFactory.getLogger(ScreenManager.class); private static final String REFERENCE_IMAGES_LOCATION = "reference-images/"; private ScreenManager() { } public static double getPercentageDiffOfImage( WebDriver driver, FluentWebElement webElementToCheck, String referenceImageName) { try { BufferedImage referenceImage = getReferenceImageFromResources(referenceImageName); BufferedImage webElementImage = getBinaryImageOfWebElement(driver, webElementToCheck); double difference = getDifferencePercent(referenceImage, webElementImage); LOGGER.info("Difference between web element {} and reference image {} is on level of {}%", getSelectorFromWebElement(webElementToCheck), referenceImageName, difference); return difference; } catch (IOException e) { throw new RuntimeException("There were some problems with reading reference or expected image.", e); } } private static BufferedImage getReferenceImageFromResources(String imageFileName) throws IOException { URL expectedFileUrl = ScreenManager.class.getClassLoader().getResource(REFERENCE_IMAGES_LOCATION + imageFileName); if (expectedFileUrl == null) { throw new IllegalArgumentException("There are no such image: resources/" + REFERENCE_IMAGES_LOCATION + imageFileName); } File expectedPicture = new File(expectedFileUrl.getPath()); return ImageIO.read(expectedPicture); } private static BufferedImage getBinaryImageOfWebElement(WebDriver driver, WebElement webElement) throws IOException { File fullPageImage = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE); BufferedImage fullPageBufferedImage = ImageIO.read(fullPageImage); Point webElementLocation = webElement.getLocation(); int webElementWidth = webElement.getSize().getWidth(); int webElementHeight = webElement.getSize().getHeight(); BufferedImage blankImageWithElementSize = fullPageBufferedImage.getSubimage(webElementLocation.getX(), webElementLocation.getY(), webElementWidth, webElementHeight); BufferedImage binaryImageOfWebElement = new BufferedImage( blankImageWithElementSize.getWidth(), blankImageWithElementSize.getHeight(), BufferedImage.TYPE_BYTE_BINARY); Graphics2D graphic = binaryImageOfWebElement.createGraphics(); graphic.drawImage(blankImageWithElementSize, 0, 0, Color.WHITE, null); graphic.dispose(); return binaryImageOfWebElement; } private static double getDifferencePercent(BufferedImage img1, BufferedImage img2) { final int highestValueOfPixel = 255; final int numberOfColors = 3; final double oneHundredPercent = 100.0; int width = img1.getWidth(); int height = img1.getHeight(); int width2 = img2.getWidth(); int height2 = img2.getHeight(); if (width != width2 || height != height2) { throw new IllegalArgumentException(String.format("Images must have the same dimensions: (%d,%d) vs. (%d,%d)", width, height, width2, height2)); } long diff = 0; for (int y = 0; y < height; y++) { for (int x = 0; x < width; x++) { diff += pixelDiff(img1.getRGB(x, y), img2.getRGB(x, y)); } } long maxDiff = numberOfColors * highestValueOfPixel * width * height; return oneHundredPercent * diff / maxDiff; } private static int pixelDiff(int rgb1, int rgb2) { final int maskToExtractOneColor = 0xff; final int shiftToGetRed = 16; final int shiftToGetGreen = 8; int r1 = (rgb1 >> shiftToGetRed) & maskToExtractOneColor; int g1 = (rgb1 >> shiftToGetGreen) & maskToExtractOneColor; int b1 = rgb1 & maskToExtractOneColor; int r2 = (rgb2 >> shiftToGetRed) & maskToExtractOneColor; int g2 = (rgb2 >> shiftToGetGreen) & maskToExtractOneColor; int b2 = rgb2 & maskToExtractOneColor; return Math.abs(r1 - r2) + Math.abs(g1 - g2) + Math.abs(b1 - b2); } }
Describing the class above I will start from the bottom. To method pixelDiff I am passing RGB value of the pixel from reference image and RGB value of the pixel on the same location on actual image. From the passed values I am extracting the value for color green, red and blue and return sum of differences for each color. Base on that I am getting information how much the pixels differ.
Method getDifferencePercent checks how much reference and actual image differs in percentage. If they have same size it goes through the whole image, compare every single pixel, sum all the differences and return the percentage of difference between images. This method can be also used to compare RGB images but as I said previously we can expect some differences then.
The next method is getBinaryImageOfWebElement. In that method we take screenshot of the whole browser view, finding the passed Web Element's location and dimensions. With use of those data we crop the web element from full screen image (the whole component has to be visible, if it is not we have to scroll the page). We have to create new image with the same size as the web element we want to compare. The newly created image has to have type TYPE_BYTE_BINARY so it will be converted to binary image. The last part is to rewrite the taken screenshot to new, binary image and return it.
Method getReferenceImageFromResources is super simple because it just gets the reference image from resources. Important thing is that reference image has to be in the same type as taken screenshot so in this case binary image.
In the method getPercentageDiffOfImage we just take all the pieces together and return the difference of the images in percentage. When the images are in binary format we can expect that difference should be on level of 0%;
When this class is ready we can create new assertions in which we can validate that Web Element looks as expected from graphical point of view. We can check for example if avatar uploaded with use of the web application is scaled or/and cropped as expected.