OpenCV Skin Detector with C#

To go along with the skin detector we wrote in Perl a few days ago, and because I’ve been doing some work today with OpenCV & C#, I thought I’d go ahead and show you how to create a skin detector using OpenCV and C#. Just like the skin detector we wrote in Perl, OpenCV already provides us with tools to do this out of the box. Because of that, creating a skin detector using OpenCV and C# is a piece of cake. In our Perl skin detector, we simply measured the percentage of skin viewable in an image and wrote that percentage to the console. In this tutorial, we’ll actually detect skin in a video feed from a standard USB webcam and will overlay the skin in the video feed with a different color. So, let’s begin.

Just like in my other OpenCV articles, we will need to add references to our OpenCvSharp libraries and add OpenCvSharp to our using statements. For this program, we will also need to add OpenCvSharp.CPlusPlus to our using statements. We’ll find out why in a moment.

using OpenCvSharp;
using OpenCvSharp.CPlusPlus;

Once you’ve added your references, you will need to setup your capture device and a window to display the video on. For that, you will use the CvCapture object which you’ll call “FromCamera” on. The FromCamera method has 2 overloaded methods. The first overload simply takes a camera index as its parameter. This index is only really important when you have more than one camera attached to your computer. If you do have multiple cameras, you will need to set this to the index of the camera you want OpenCV to use. To get the index for the camera you want to use, begin with 1 and increment the number until you find the camera of choice. After you’ve setup your capture device, go ahead and construct a CvWindow for displaying your video feed.

using (CvCapture cap = CvCapture.FromCamera(1))
using (CvWindow w = new CvWindow(“Skin Detector”))

Next, you’ll need to create a while-loop which will be used to keep the window from closing until you click on the window and press a key.

while (CvWindow.WaitKey(10) < 0)

Inside your while-loop, you’re gonna need to setup a few IplImages. The first of these images is simply a reference to the current frame from your capture device. The second image is an image which will contain the color-overlay. The third image is just a copy of the first image (the frame from the capture device). This last image will be a combination of the first 2 images and is what we’ll display in our CvWindow in just a bit.

Now that you have references to your capture device and have each frame referenced as an IplImage, it’s time to use the OpenCvSharp.CPlusPlus namespace we added to our using statements at the beginning of this article. The CPlusPlus package contains a class called CvAdaptiveSkinDetector. This is the class that is going to locate the skin in our frames, just as the name implies. Once you’ve built your detector object, you will need to call the “Process” method. This method accepts the image source (the first image from above), locates the skin in the image, and then stores that skin reference in the second image from above. So, if you were to draw the second image to the window, you would only see your skin and nothing else.

CvAdaptiveSkinDetector detector = new CvAdaptiveSkinDetector(1, MorphingMethod.None);
detector.Process(imgSrc, imgHueMask);

That’s pretty cool, but we’re not stopping there. Instead, we are now going to add a private method that will take that second image, color it green (or whatever color you choose), and overlay that image on top of the third (destination) image from above. This method will need to accept 3 parameters. The first parameter will be the skin-only image from above. The second parameter will be the image that we want to draw the green onto. And the third parameter will be the color we want to use. Note: You can hardcode the color and remove it from the parameters list if you don’t want to pass it into this method or in case you never plan on changing the color programmatically.

In the code below, you will notice that the first thing I do in my new method is a check to make sure that both incoming images are the same size and I throw an ArgumentException if they’re not. Since I’m manually specifying my images in the previous method, I know that my images are going to be the same size. So, this check isn’t required for this example, but I’m including it anyways. The reason both images have to be the same size is because we’re going to manually loop thru ever pixel in the skin-only image, find each pixel that represents skin, and then find the same pixel location in the other image and change that pixel color to green. Here is the entire method for doing that.

After you’ve repainted the pixels in your destination image, all you’re left to do is to draw that image back to your window in the first method above. If you did everything correctly, you should be able to test the app at which point your skin should now be green like this:

The screenshot isn’t perfect because I did a screen grab from a moving video. So, there’s a little bit of “noise” in it. But, the video does look a lot better when it’s in real-time. Plus, the skin would be more prominent if I wasn’t wearing a white T-shirt. Or, maybe I just need to get out more and work on my tan? Here is the complete source code for this article. As always, if you have any questions or suggestions, feel free to post them in the comments below and I’ll help you any way I can. Until next time, HAPPY CODING!