How to convert OpenCV cv::mat to System Bitmap to System ImageSource

OpenCV is an open source computer vision library which can be used for a wide variety of things such as face recognition, people detection, motion tracking, video editing and image manipulation. For my undergraduate project, I made use of the OpenCV libraries to create a facial recognition system. It used a webcam to detect ID cards (in my project, I made use of ACU ID cards) and find the photograph on it, then used facial recognition to match the presented ID card with one in its database. More recently, I utilised the OpenCV libraries in a piece of research to build software to perform automatic data capture from hospital videos to save the human-effort required in data capture.

In my undergraduate project, the core output was the dissertation report and not the software, thus the software was created to perform its functionality but without a clean or user-friendly interface. As such, displaying information from the OpenCV libraries was done using the cv::imshow() method, rather than embedding the cv::mat into a GUI of my design. For my more recent software development, I opted to do this to produce a clean GUI that I could use to demonstrate my research to those outside my field.

To do this, I build a GUI using the WPF libraries in C#, and, as is common for most of the tools I create in the C languages, used a C++ engine to work with the OpenCV libraries. The GUI needed to display the frames of the video as it was playing so the user could see the video at the same time as it was being analysed. To do this, I had an image control on my WPF GUI which would be used to display the frames. However, this meant converting the cv::mat image to an image source to display in the image control.

This was done by converting the cv::mat to a System Bitmap first as a go-between the cv::mat and image source data types and then converting the bitmap to an image source.

Converting a cv::mat to a System Bitmap

The following code example is given in its CLI form as this was how I passed the cv::mat from the C++ engine into a Bitmap for the C# UI code.

Bitmap^ Engine::ConvertMatToBitmap(cv::Mat matToConvert) {
	return gcnew Bitmap(matToConvert.cols, matToConvert.rows, 4*matToConvert.rows, System::Drawing::Imaging::PixelFormat::Format24bppRgb, IntPtr(matToConvert.data));
}

Converting a Bitmap to an ImageSource

The following code example is given in its C# form and made use of the following libraries:

• System.IO
• System.Drawing
• System.Windows.Media
• System.Windows.Media.Imaging

private ImageSource ConvertBitmapToImageSource(Bitmap imToConvert)
{
    Bitmap bmp = new Bitmap(imToConvert);
    MemoryStream ms = new MemoryStream();
    bmp.Save(ms, System.Drawing.Imaging.ImageFormat.Bmp);

    BitmapImage image = new BitmapImage();
    image.BeginInit();
    ms.Seek(0, SeekOrigin.Begin);
    image.StreamSource = ms;
    image.EndInit();

    ImageSource sc = (ImageSource)image;

    return sc;
}

The resulting image source is then able to be applied to the image control in WPF and displayed in the GUI. To display a video in a GUI, simply use a timer to continue calling frames from the engine as they change and display them in the GUI.