Chapter 7.5

From IPRE Wiki
Jump to: navigation, search

Manipulating Images and Robot Vision

Your robot has a small camera that can be used to take pictures. Myro includes several functions that allow you to capture and modify images. Lets start with the simplest, capturing a picture, assigning it to a variable, and then showing it on the screen:

pic = takePicture()
show(pic)

You can get some information about the size of the picture with the getWidth() and getHeight() functions:

picWidth = getWidth(pic)
picHeight = getHeight(pic)
print "The Picture is", picWidth , "pixels wide and", picHeight, "pixels high."

A normal myro picture is full color, which means that each pixel is made up of a blend of three primary colors (Red, Green, and Blue). Each pixel has a separate value (from 0 to 255) for each of the three primary colors. Note that the computer is using 8 bits (one byte) to store each color value. One byte is made up of 8 bits, and with 8 bits you can represent 2 8 (256) values, or a number between zero and 255.

So a completely red pixel would be represented as (255,0,0), or full red, and zero green and blue. Similarly a green pixel would be represented as (0,255,0) and a blue pixel would be represented as (0,0,255). Other colors are made by mixing various values of red, green, and blue. For example, white is made by combining all of the colors (255,255,255), and black is made by an absence of any color (0,0,0). If you combine green and blue with no red (0,255,255) you would get a teal color, while a combination of red and green with no blue (255,255,0) would produce yellow.

Because each pixel in the picture needs three bytes (one for the Red, Green, and Blue values), each pixel requires three bytes (or 24 bits) to store. This is why full color pictures are sometimes referred to as "24 bit".

Exercise 1: If our picture is 256 pixels wide, and 192 pixels high, how many total pixels does it have?

Exercise 2: How many bytes does it take to store the picture from Exercise 1?

Robot Explorer

If you do not need a full color picture, you can tell myro to capture a gray-scale image by giving the takePicture() function the "gray" parameter.

grayPic = takePicture("gray")
show(grayPic)

Did you notice that taking the gray-scale picture took less time than taking the color picture? This is because the gray scale picture only uses one byte per pixel, instead of three. The gray-scale picture does not have Red, Green, and Blue values for each pixel. Instead, it stores a single number (again, one bye, or a number between zero and 255) for each pixel that represents how dark or light the pixel is (zero is full black, and 255 is full white).

Because gray-scale images can be transfered from the robot to your computer more quickly than full color images, they can be useful if you want the images to update quickly. For example, you can use the joyStick() function combined with a loop that quickly takes and displays pictures to turn your robot into a remotely piloted explorer, similar to the mars rovers.

joyStick()
for i in range(25):
  pic = takePicture("gray")
  show(pic)

The above code will open a joyStick window so that you can control your robot, and then capture and show 25 pictures, one after the other. While the pictures are being captured and displayed like a movie, you can use the joystick to drive your robot around, using the pictures to guide it. Of course, if you removed the "gray" parameter from the takePicture() function call, you would get color pictures instead of gray-scale pictures, but they would take much longer to transfer from the robot to your computer, and make it more difficult to control the robot.

Saving and Loading Pictures

If you take a picture that you would like to save, you can save it as a JPEG file on your computers hard drive with the savePicture() function.

myPicture = takePicture()
savePicture(myPicture,"myPic.jpg")

You can use any name you want for your picture, as long as it ends with .jpg to tell Myro (and your computer) that it is a JPEG image file. Later, you can load the picture from disk with the makePicture() function:

mySavedPicture = makePicture("myPic.jpg")
show(mySavedPicture)

Manipulating Pixels

Myro provides functions that allow you to get the individual red, green, and blue values from a pixel, as well as set the values to numbers of your choosing. But before you get or set the value of a pixel, you need to select a specific pixel to change. To demonstrate, we will start out by making a blank picture that is 100 pixels high by 100 pixels wide, using the makePicture() function:

newPic = makePicture(100,100)
show(newPic)

Black100x100.gif

Note that the picture we made starts out will all pixels colored pure black. We can select a specific pixel (say, the one at X location 10 and Y location 5) using the getPixel() function. After we select a specific pixel, we can set its red value to full on (255) with the setRed() function.

onePixel = getPixel(newPic,10,5)
print getRed(onePixel)          #initial red value
setRed(onePixel,255)
print getRed(onePixel)          #new red value
show(newPic)

Black100x100-r.gif

Note that the red value of the pixel at (10,5) was initially zero, but after our call to setRed() the pixel value has changed to 255. Notice also that we had to call show() again before our change is displayed on the screen. If you look at the picture very carefully, you will see that one pixel near the top left corner is now red, instead of black. If you want to make that pixel white, you have to assign 255 to the green and blue colors as well:

setGreen(onePixel,255)
setBlue(onePixel,255)
show(newPic)

Black100x100-w.gif

Now the pixel at location (10,5) is pure white.

Drawing a Line

NOTE: if you would like to use a copy of the apple picture rather than using your own picture in the examples below, replace:

takePicture()

with:

makePicture("http://wiki.roboteducation.org/wiki/images/2/2b/Apple.jpg")

If we want to change a lot of pixels all at once, we can use a loop. For example, the following loop will change all pixels that have an X value of 10 and a Y value anywhere between 0 and 100 (but not including 100) to be red:

for yValue in range(0,100):
  aPixel = getPixel(newPic, 10, yValue)
  setRed(aPixel,255)
  setGreen(aPixel,0)
  setBlue(aPixel,0)

show(newPic)

The result is a vertical red line (at X position 10). Note that this piece of code will only work correctly for pictures that are exactly 100 pixels high (because we loop from zero to 100). But we can generalize this code to work on pictures of any size by replacing the 100 with a function call that tells us the actual height of the picture as follows:

for yValue in range(0, getHeight(newPic) ):
  aPixel = getPixel(newPic, 10, yValue)
  setRed(aPixel,255)
  setGreen(aPixel,0)
  setBlue(aPixel,0)

Drawing a line could be a useful function to use later, so we can prepare to re-use the code above by putting it into a function. But since we don't know the exact X position that the user will want to draw their line, we should make that into a parameter that the user can specify. Also, since we don't know the name of the variable that will hold the picture, that should also be a parameter:

def drawVerticalRedLine( picture, xPos ):
  for yPos in range(0, getHeight(picture) ):
     aPixel = getPixel( picture, xPos, yPos)
     setRed(aPixel,255)
     setGreen(aPixel,0)
     setBlue(aPixel,0)


Now we have a function that will draw a vertical red line on a picture of any height. Note that our function does NOT show the image, so a user would have to call our function, and then call the show() function to display the line on screen:

pic = takePicture()
show(pic)
wait(1)
drawVerticalRedLine(pic, 25)
show(pic)

Apple-red-line-25.jpg

Exercise 3: Write a function to draw a horizontal red line.

Moving a Line

You can make a special effect by drawing the red line moving across the picture one X position at a time with another loop:

myPic = takePicture()

for xPos in range(0, getWidth(myPic) ):
  print "Now Drawing at xPos: ", xPos
  drawVerticalRedLine(myPic, xPos)
  show(myPic)

Apple red draw.gif

Of course, this slowly covers up the existing picture with nothing but red pixels. If you want to only cover one line at a time, you have to draw on a copy of the original picture to keep from losing your picture data. The copyPicture() function will return a copy of a picture so that you can draw on a copy without covering up pixels from the original.

originalPic = takePicture()

for xPos in range(0, getWidth(originalPic) ):
  #Make a copy of the original to draw on
  copyPic = copyPicture(originalPic)
  print "Now Drawing at xPos:", xPos
  drawVerticalRedLine(copyPic,xPos)
  show(copyPic)

Apple-redraw-line.gif

Operating on all pixels in an image

The getPixel() function will return a pixel at a specific location in the picture. However, sometimes you want to do something to all pixels in the picture. the getPixels() method will return a generator that is similar to a list of all pixels. You can use the getPixels() method in a for loop to perform an operation on all pixels in the image. For example, we can turn the red color of all pixels all the way on with the following code:

myPic = takePicture()
show(myPic)

for eachPixel in getPixels(myPic):
  setRed(eachPixel,255)

show(myPic)

Because the above code does not change the green or blue values of the pixels, the picture is still recognizable, but all pixels have a reddish tint.


Apple.jpg Apple-red-tint.jpg

Robot Vision

By making a decision about each pixel in an image, you can locate specific areas in an image. For example, by looking for pixels that have a large value, you can locate bright areas in the image. You can even modify the image to outline bright areas. For example:

myPicture = takePicture()
show(myPicture)

for pixel in getPixels(myPicture):
  redValue = getRed(pixel)
  greenValue = getGreen(pixel)
  blueValue = getBlue(pixel)
  averageValue = ( redValue + greenValue + blueValue) / 3.0

  if averageValue > 175 :
    #Turn the pixel white
    setRed(pixel,255)
    setGreen(pixel,255)
    setBlue(pixel,255)
  else:
    #Otherwise, turn it black.
    setRed(pixel,0)
    setGreen(pixel,0)
    setBlue(pixel,0)

show(myPicture)

Apple.jpg Apple-bright.jpg


By changing the conditional test, we can instead look for areas that have a large amount of the color red. Red areas are characterized by having large red values, but smaller blue and green values.

myPicture = takePicture()
show(myPicture)

for pixel in getPixels(myPicture):
  redValue = getRed(pixel)
  greenValue = getGreen(pixel)
  blueValue = getBlue(pixel)

  if redValue > 175 and greenValue < 175 and blueValue < 175 :
    #Turn the pixel white
    setRed(pixel,255)
    setGreen(pixel,255)
    setBlue(pixel,255)
  else:
    #Otherwise, turn it black.
    setRed(pixel,0)
    setGreen(pixel,0)
    setBlue(pixel,0)

show(myPicture)

Apple.jpg Original Picture.

Apple-red-bigger-175.jpg Pixels with red values larger than 175 highlighted in white.

Apple-green-under-175.jpg Pixels with green values lower than 175 highlighted in white.

Apple-red-green-blue.jpg Pixels that have both red values larger than 175 and green and blue values lower than 175 highlighted in white.


We can make this code to find red pixels into a function that will return a black and white picture with white pixels representing "red" areas.

def findRedAreas(picture):

  for pixel in getPixels(picture):
    redValue = getRed(pixel)
    greenValue = getGreen(pixel)
    blueValue = getBlue(pixel)

    if redValue > 175 and greenValue < 175 and blueValue < 175 :
      #Turn the pixel white
      setRed(pixel,255)
      setGreen(pixel,255)
      setBlue(pixel,255)
    else:
      #Otherwise, turn it black.
      setRed(pixel,0)
      setGreen(pixel,0)
      setBlue(pixel,0)

  return picture

The result of testing for "red areas" is an image where most of the apple has been detected, but a lot of other pixels scattered around the image are also somewhat "reddish". If we want the robot to turn towards the direction with the most red pixels, we need to calculate the average X (horizontal) location of the pixels that have been marked (by turning them white).

If we examine every pixel in the image, and average the X coordinates of all the "detected" (or white) pixels, we can determine the middle of the red areas. To do this, we use two loops (one for the Y positions, and one for the X positions) to look at every pixel, and add up the X positions of all pixels that are turned "on" (or white). Because a white pixel has values of (255,255,255) and an "off" or black pixel has values of (0,0,0) we can take a shortcut and only test the value of one of the three colors.

def AverageXofWhitePixels(picture):
   sumX = 0.0           # We use floating point values.
   counter = 0.0        # We use floating point values.
   for xPos in range(0, getWidth(picture) ):
      for yPos in range(0, getHeight(picture) ):
          pixel = getPixel(picture,xPos,yPos)
          value = getGreen(pixel)
          if value > 0 :
             sumX = sumX + xPos
             counter = counter + 1

   averageX = sumX / counter
   return int(averageX)         #Return an Integer

Now, we can use the functions we have defined so far to locate the average X location of red pixels, and draw a line at that position:

myPicture = takePicture()
picture = copyPicture(myPicture)
picture = findRedAreas(picture)
XposAvg = AverageXofWhitePixels(picture)
drawVerticalRedLine(myPicture,XposAvg)
show(myPicture)

Apple-red-line.jpg

Because of all the other "red" pixels detected that were not on the apple, the red line is not exactly centered on the apple, but it is close enough that we could use the value in the XposAvg variable to figure out which way to turn the robot to face the apple.

Exercise 4: Write a program that will turn your robot towards red objects.

Animated GIF movies

The savePicture() function will also allow you to make an animated GIF, which is a special type of picture that in a web browser will show several pictures one after another in an animation. To save an animated GIF, you must give the savePicture() function a list of pictures (instead of a single picture) and a filename that ends in ".gif". Here is an example:

pic1 = takePicture()
turnLeft(0.5,0.25)
pic2 = takePicture()
turnLeft(0.5,0.25)
pic3 = takePicture()
turnLeft(0.5,0.25)
pic4 = takePicture()

listOfPictures = [pic1, pic2, pic3, pic4]

savePicture(listOfPictures, "turningMovie.gif")

The best way to view an animated GIF file is to use a web browser. If you are using Firefox, you can use the FILE->Open File menu, then pick the turningMovie.gif file. The web browser will show all frames in the movie, but then stop on the last frame. To see the movie again, press the "reload" button.

You can also use code in a loop to make a longer movie with more images:

pictureList = []  #Start with an empty list.
for i in range(15):
  pic = takePicture()
  pictureList = pictureList + [pic]   #Append the new picture
  turnLeft(0.5,0.1)

savePicture(pictureList,"rotatingMovie.gif")

Rotating-movie.gif