Calico Emgu.CV
This page documents the use of the Emgu.CV library for accessing computer vision functions and a general interface to using webcams. It uses OpenCV.
You will need Calico 2.3.1 or better to use these examples.
These instructions are ALPHA, meaning that this is all experimental, and not well tested or debugged.
Contents
Overview
For those of you interested in integrating Computer Vision into your courses and research, this documents a bit of the development and plans for this.
First, we plan on having an interface to the Robot Operating System (ROS) in Calico eventually. By doing so, we hope to leverage a large amount of previous and sophisticated work in ROS, with the ease of Calico. However, currently ROS is really only supported on Linux, so this may not be a viable alternative for all users.
In the meantime, we are working towards making the OpenCV library available in Calico, through the Emgu.CV API. This has two levels: a language-neutral level (dlls in Calico/modules/), and then the low-level binaries (dlls in Calico/bin/).
Perhaps someday, we can just make Emgu.CV a standard part of Calico. For the time being, you'll have to download some additional components.
So far, everything we have tested on Windows7 is working ok... not perfectly yet. For example, not all webcams on all platforms have been tested.
What can you do with Emgu.CV? Find faces in your Fluke images, sophisticated object recognition (say, based on shape), find your location, motion detection, feature detection, machine learning, tracking, panoramic creation, and much more. Search the web for "opencv" for more information.
We'll be continuing to work on this over time.
Installation
From Calico
- Copy the emgu zip file for your Operating System to your computer from here:
- Unzip the file to a temporary location
- Run setup.py from inside Calico
- If you want to re-install (after it has been installed before), you can't directly on Windows, because you can't overwrite a file in use. But you can do this:
- StartCalico --nomodules --repl --nographics path\to\setup.py
Manual, Non-Calico version
The manual install of factory software will work, to a point. But there are additions that the Calico version (above) contains.
- Download Emgu.CV
- Copy the low-level (machine-dependent) opencv_*.dll libraries to Calico's bin folder.
- On Windows, use the 32-bit (x86) version even if you are on a 64-bit computer
- For example copy "C:\Emgu\emgucv-windows-universal-gpu 2.4.9.1847\bin\x86\*.*" to C:\Calico\bin
- Copy the high-level (managed) libraries to Calico's module folder
- For example copy "C:\Emgu\emgucv-windows-universal-gpu 2.4.9.1847\bin\*.dll" to C:\Calico\modules
Here is the code added to Image.cs, if you want to add that yourself:
public class ImageUtils {
public static Image<Structure.Gray,Byte> MakeImageGray(System.Drawing.Bitmap bitmap) {
return new Image<Structure.Gray,Byte>(bitmap);
}
public static Image<Structure.Rgb,Byte> MakeImageRgb(System.Drawing.Bitmap bitmap) {
return new Image<Structure.Rgb,Byte>(bitmap);
}
public static Image<Structure.Rgba,Byte> MakeImageRgba(System.Drawing.Bitmap bitmap) {
return new Image<Structure.Rgba,Byte>(bitmap);
}
public static Image<Structure.Bgr,Byte> MakeImageBgr(System.Drawing.Bitmap bitmap) {
return new Image<Structure.Bgr,Byte>(bitmap);
}
public static Image<Structure.Bgra,Byte> MakeImageBgra(System.Drawing.Bitmap bitmap) {
return new Image<Structure.Bgra,Byte>(bitmap);
}
}
Testing
In running these interactions, it is probably a good idea to NOT reset the shell when running a new Calico script. In Calico when you have a program script (say, Python) open, go to menu -> Script -> Script Options and UNCHECK "Reset Shell on Run". This will not clear variables, and will allow the webcamera connection to remain continuous through your session.
After installation, you should be able to run the following:
import Emgu import Myro import Graphics capture = Emgu.CV.Capture() image = capture.QueryFrame() ## draw the image obtained from camera Myro.show(Graphics.makePicture(image.Bitmap))
If that fails, perhaps you have the wrong architecture (32-bits vs. 64-bits). Although, the 32-bit version should work for all Windows versions.
Windows7: ok, so far
Linux: looks like Emgu.CV needs to be compiled on a 32-bit computer
Mac: not tested
Examples
Images in CV
To do any computer vision (CV) functions, you'll need to get an image into CV's Image format. Normally, one would:
pic = Myro.takePicture() bitmap = pic.toBitmap() image = Emgu.CV.Image[Emgu.CV.Structure.Gray,System.Byte](bitmap)
However, something is wrong, and IronPython cannot find the interfaces for Image. So, we have added some static methods to our own version of Emgu.CV that will allow you to make images given a bitmap:
- Emgu.CV.ImageUtils.MakeImageBgr(bitmap)
- Emgu.CV.ImageUtils.MakeImageBgra(bitmap)
- Emgu.CV.ImageUtils.MakeImageGray(bitmap)
- Emgu.CV.ImageUtils.MakeImageRgb(bitmap)
- Emgu.CV.ImageUtils.MakeImageRgba(bitmap)
So, you can take a picture, and get a CV image:
pic = Myro.takePicture() bitmap = pic.toBitmap() image = Emgu.CV.ImageUtils.MakeImageGray(bitmap)
Filtering
Myro.init("sim")
bitmap = Myro.takePicture().toBitmap()
image = Emgu.CV.ImageUtils.MakeImageGray(bitmap) ## get the image obtained from camera
filtered = image.PyrDown().PyrUp()
pic = Myro.makePicture(filtered.Bitmap) ## and back again, just testing
window = Myro.makeWindow("Emgu.CV", pic.width, pic.height)
pic.draw(window)
Face Detection from Fluke/Simulated Fluke
Here is the code:
import Emgu
import Myro
import System
import Graphics
Myro.init("sim") # or real robot, eg "com40"
pic = Myro.takePicture()
bitmap = pic.toBitmap()
image = Emgu.CV.ImageUtils.MakeImageBgr(bitmap)
pic = Myro.makePicture(image.Bitmap) ## display it, as an example of going back to Myro
Myro.show(pic, "Emgu.CV")
grayFrame = image.Convert[Emgu.CV.Structure.Gray,System.Byte]()
filename = calico.relativePath("../modules/haarcascade_frontalface_default.xml")
haarCascade = Emgu.CV.HaarCascade(filename)
detectedFaces = grayFrame.DetectHaarCascade(haarCascade)[0]
for face in detectedFaces:
print("Found a face at ", face.rect)
rect = Graphics.Rectangle((face.rect.X,
face.rect.Y),
(face.rect.X + face.rect.Width,
face.rect.Y + face.rect.Height))
rect.fill = None
rect.outline = Myro.Color("red")
rect.border = 4
rect.draw(Myro.getWindow())
Myro.show(pic, "Emgu.CV, Face Detection")
Face Detection from Webcam
NOTE: the format
image.Convert[Emgu.CV.Structure.Gray,System.Byte]()
is the Python way of converting C# generics:
image.Convert<Emgu.CV.Structure.Gray,System.Byte>()
Here is the code:
import Emgu
import Myro
import System
import Graphics
capture = Emgu.CV.Capture() ## Sometimes, the first image is black
image = capture.QueryFrame() ## get the image obtained from camera
pic = Myro.makePicture(image.Bitmap) ## display it
Myro.show(pic, "Emgu.CV")
grayFrame = image.Convert[Emgu.CV.Structure.Gray,System.Byte]()
filename = "C:/Emgu/emgucv-windows-universal-gpu 2.4.9.1847/bin/haarcascade_frontalface_default.xml"""
haarCascade = Emgu.CV.HaarCascade(filename)
detectedFaces = grayFrame.DetectHaarCascade(haarCascade)[0]
for face in detectedFaces:
print("Found a face at ", face.rect)
rect = Graphics.Rectangle((face.rect.X,
face.rect.Y),
(face.rect.X + face.rect.Width,
face.rect.Y + face.rect.Height))
rect.fill = None
rect.outline = Myro.Color("red")
rect.border = 4
rect.draw(Myro.getWindow())
Myro.show(pic, "Emgu.CV, Face Detection")
Continuously looking for faces:
import Emgu
import Myro
import System
import Graphics
capture = Emgu.CV.Capture()
image = capture.QueryFrame() ## draw the image obtained from camera
pic = Myro.makePicture(image.Bitmap)
window = Myro.makeWindow("Emgu.CV", pic.width, pic.height)
filename = "C:/Emgu/emgucv-windows-universal-gpu 2.4.9.1847/bin/haarcascade_frontalface_default.xml"""
haarCascade = Emgu.CV.HaarCascade(filename)
while True:
print("Taking a picture...")
image = capture.QueryFrame() ## draw the image obtained from camera
if image is None:
continue
try:
pic = Myro.makePicture(image.Bitmap)
except:
print("Error!")
capture = Emgu.CV.Capture()
continue
window.clear()
pic.draw(window)
grayFrame = image.Convert[Emgu.CV.Structure.Gray,System.Byte]()
detectedFaces = grayFrame.DetectHaarCascade(haarCascade)[0]
for face in detectedFaces:
print("Found a face at ", face.rect)
rect = Graphics.Rectangle((face.rect.X,
face.rect.Y),
(face.rect.X + face.rect.Width,
face.rect.Y + face.rect.Height))
rect.fill = None
rect.outline = Myro.Color("red")
rect.border = 4
rect.draw(window)
Myro.wait(.1)
