OpenCV Coin Detection Project

Introduction

This blog covers a course project I completed for Learn OpenCV for Faces, conducted by Satya Mallick. The project objective is to use a webcam to detect US coin currency on a table and classify each coin, counting the total change. The project is using OpenCV and Python (WinPython 3.65) running on a Acer laptop with Windows 10 OS.  I had zero experience with OpenCV before this class but have previously programmed in Python and completed a course in ML. Note, the OpenCV course also covers the material in C++ as well as Python.

Summary of Results

The simplest approach I started with to detect non-touching coins was to preprocess the image to a grayscale then threshold and apply findContour. This approach does not work with touching coins or partially occluded coins. I finally ended up evaluating both Hough Transforms and a Watershed segmentation algorithm. These two methods were chosen because they both handle partial occlusion. However, after looking at a specific case of a false coin a weakness in the Watershed algorithm was noted. The final algorithm selection was the Hough Transform. For more on the Watershed weakness see the appendix. In addition, to improve measurement accuracy of the coin diameters a calibration was made using a camera matrix and distortion matrix. Also, object classification uses both diameter and hsv color. The final implementation still is not as robust as I like and may be due to camera resolution.

The left photo shows results of identifying and classifying 15 non-touching coins and rejecting a slug, while the right shows touching and occluded coins.

 Non-touching                                     Occluded

                                             

A video of the occluded case is included here.

Overall Tasks

The first major task is to locate the objects (coins). Once the objects are identified then they can be classified using an algorithm to a) check physical size and b) Check object color to reduce false detection. We tried schemes with non-touching coins then applied methods to touching coins.

Perspective View

Identifying coins by their diameter is a challenge because the diameters are very close between dimes and pennies. Splitting the margin is only 0.57mm. The inherent camera perspective distorts the measurement of coins that are not centered in the field of view. As an example a penny has a diameter of 19.05mm and in the center of the FOV with a 16” distance to the camera the angle span is 2.68 degrees. If the coin is displaced 3 inches from the center of FOV the angle span is 2.59 degrees. This is equivalent to the coin being 18.50mm in diameter versus 19.05mm.

One idea considered for this is was to identifying points in the four corners of the FOV and use those for a 4 point perspective correction to the image. These points would be much smaller than the smallest coin and be arranged in an accurate rectangle. The thought of using the ID corners as the perspective correction was rejected because the card is not centered on the FOV and this could introduce distortion.

The final choice for a solution was to apply a camera calibration using a checkerboard pattern and the OpenCV function calibrateCamera. The checkerboard pattern was moved in several positions and used to compute the camera matrix and distortion matrix. The end result is the matrices are saved in a yaml file and used to correct the image for every frame taken. The python script to do the one time calibration is calibrate.py. The two images shown below are before and after calibration using a 9x7 checkerboard. The camera used in this project is a Logitech C310 webcam with 480x640 rgb images mounted on a tripod about 16 inches above a flat table.

 Original                                      Calibrated

                               

The corrections are difficult to see but the change in coin diameters was significant compared to the required accuracy. When running the video capture I could move a quarter around the field of view and see the measured diameter change depending on location. This error improved quite a bit after calibration. Below is a video capture during the time calibrate.py was running. A total of 10 calibration samples were taken, one of which is shown below.

 Calibration Checkerboard

Object Detection - Background color

An important part of the object detection is deciding the background to be used for capturing the image of the coins. The background I ended up selecting was a black velvet surface. Backgrounds that were rejected were white copy paper, green art paper and a black art paper.  Problems encountered were the the conditions of the coins. Pennies range from shiny copper to a dark oxidized copper. Similarly, dimes can be shiny or muted. Here are two images showing the extremes.

 black background                                white background

                                                      

Oxidized pennies have high contrast on the white background but poorer contrast on the black. Also, shiny dimes have low contrast on the white background and good contrast on the black. I captured contours on a black background and then computed the mean colors of each coin. Below is a table showing the RGB and HSV values along with the Euclidean distance to white and to black.

    Cluster Color Map

Notice on a white background the minimum Euclidean distance to white is 116.9 for a quarter. On the other hand the Euclidean distance to black is a minimum of 213 for a penny. Based on the accuracy of identifying coins and looking at the Euclidean distance to the background I chose the black velvet as the background of choice. Below is the grayscale image comparing the two backgrounds.

 Gray scales:

 black background                                 white background

                                                        

The ID-1 reference (credit card) was eventually changed to a security card (still ID-1). For identity concerns I did not want to publish by credit card. Secondly, I am only interested in the outer contour of the reference and the credit card had too many excess features within the contour and the shiny surface reflected lights creating image artifacts. I ended up using a security card which is ID-1 but has no other features.

Object Detection Image Processing

The object detection does not require color so the processing pipeline I used was the following:

• Scale the image size to (480 x 640)
• Apply camera and distortion correction
• Gamma correction 2.2
• Blur with (3,3) kernel
• Convert to gray scale

In the case of findContours the gray scale was then passed to the Canny filter

• Canny edge detection cv2.Canny(gray, 50, 200, 10)

The HoughCircles used the grayscale without further processing. The grayscale was used in the call to HoughCircles using param1= 175, param2= 35, minRadius=15 and maxRadius= 40. The circles structure returned was examined in a loop and the radius was used as part of the coin classification as well as the circle being used as a mask to extract color information for coin classification.

Object Classification - Color Space

The object classification of the coins uses coin diameter and coin average color. The diameter is taken from the HoughCircles radii while the color was derived from the HSV image. Specifically the blurred image, just prior to grayscale, was examined using the circles from the Hough transform as a mask. The image was converted to the hsv color space and the h and s components used to determine a Euclidean distance. From a test of several coins a hyperplane (6.4 * h -65 - s = 0) was used to separate pennies from dimes into two clusters. If the evaluation of the hyperplane is < 0 the decision is the object is a penny, if >0 the decision is it is not a penny. In principle this is similar to using a clustering algorithm followed by a linear SVM to find the best separating plane. In my simple case it was done manually on eight samples. Shown below is the cluster map of several samples of pennies, dimes and the background:

Color Space Cluster Map

For nickels, quarters and fifty cent pieces the diameter decides the coin value. For diameters in the range of dimes and pennies the color decides the coin value.

Eliminating False Positives

One obvious false positive would be a metal washer used to ‘fool’ the algorithm. To guard against this it was decided to look at the mean hsv color in a small circle at the center of each potential coin. This was compared to a sample of the background to detect a hole in the washer. The background is checked once after all contours are found. The area outside the extent of the combined contours is used to place a circle mask and sample the mean HSV color. The h-s values at the center of each potential coin are then compared against the background sample h-s values. Depending on which side of the hyperplane the center lies it is declared either True for no hole or False for a hole present.

Obviously without looking for detailed features on each coin a solid slug of copper or nickel of the proper diameter would be detected as a false positive.

Difficulties Encountered

I’ll comment briefly here are difficulties encountered during the project. As mentioned, the difference in diameters between coins is very small and this can lead to errors when measurements have noise due to lighting and/or shadows. Also I noted that as coins touched or were partially occluded this impacted the radius returned by HoughsCircles measurement. Approaches taken to reduce this uncertainty was to scale the input webcam image up by a factor of 1.33 to reduce the mm/pixel scale factor and calibrate the camera. One more thing to try might be to use a higher resolution camera, such as an iPhone.

An additional challenge was measuring (h,s) values at the center of coins and deciding if the value qualified as background to detect a hole. An area of the background was selected for each frame and sampled to define “background”. This value was compared against the sample at the coin center in a 5 pixel radius. Rather than a hyperplane in (h,s) I ended up using a threshold of h_sample = 0.4 * h_background to define a hole if h was larger and not a hole if h was less. See plot above.

For a discussion on the performance of the Watershed algorithm for this use case see my blog on the topic.

Python Code Source

Code for this project is checked-in to BitBucket along with a few mp4 files.

Note added Jan 22, 2021; I found a related helpful article by Adrian Rosebrock at Pyimagesearch that discusses measuring object sizes. It is located here.