The resale value of a used smartphone is highly dependent on the quality of the screen. In this project, we demonstrate methods to automatically identify screen scratches and determine screen quality using a custom imaging and illumination layout and computer vision software.
Key elements of the design include:
- Position of illumination relative to the screen for stray light suppression
- Light scattering for uniformity of lighting
- Image processing/ computer vision algorithms with convolutional neural networks
The device is a box inside which the phone is placed. On the top side of this box, there are two LED strips with scattering plates on the front. The light should be dissipated equally on the bottom side to highlight scratches on the screen and to avoid direct reflections onto the camera which can lead to the appearance of light reflections on the resulting image. Schematic drawing of the device you can see on the image.
The camera module used in this device is OV2710 USB Camera Module 2MP. This module has a 160-degree wide-angle lens which is not suitable for our task so it was replaced with a new one with a 4 mm focal distance. The camera is connected to the laptop via USB.
Camera – https://www.aliexpress.com/i/32841227779.html
Lenses – http://www.m12lenses.com/4-0mm-F1-8-5MP-M12-Mount-CCTV-Lens-p/pt-0418-5mp.htm
General appearance of the device:
In this image, you can see the device connected to the laptop with a software for image grabbing and processing.
U-Net convolutional neural network
Now we will try briefly explain how the scratch detection algorithm works. Basically, the algorithm consists of 6 steps which are described below with samples of resulting images of each step.
These are initial images of the phones.
The first step is to detect area where scratches should be searched. For this purpose, we have trained U-Net convolutional neural network used for image segmentation. This network architecture performs this task very well. Result you can see on the images below.
The second step is image preprocessing made to highlight scratches
The third step is to remove the background from the image with a help of the screen mask we made in the previous step
The fourth step is cleaning the image (simple removement of small objects or random pixels) and binarization made by applying thresholding on pixel brightness.
The fifth step is the detection of scratch candidates with a help of a standard contours detection function.
The sixth step is the detection of possible scratches with the filtering of scratch candidates.
Using these methods of imaging, illumination and convolutional neural network based image processing, we have built a robust tool to identify scratches and allow for less subjective and more objective methods to determine smart phone resale value.