Question on color-blob tracking

[kiteswing]

Here, I have two questions on the color-blob tracking.

1. Colormap

When the mouse shows the RGB to be (16, 16, 16), and I add it into the colormap. The color map set the red green blue range to be (16 48 ), (16 48 ), (16 48 ).

So, what RGB will be tracked? Can I say, besides RGB(16, 16, 16), all the colors in this range. like RGB(48, 48, 48 ), RGB(48, 32, 32) will also be tracked?

2. Bounding box

There is blob of black in my picture. After I enable tracking with the color map shown in the 1st question, there are many bounding boxes that keep moving, even though I didn't move the camera at all. The bounding boxes seem very unstable.

So, my question is: what is the meaning of bounding boxes here? Can you give me any description of how the bounding boxes works? Or, how the tracking process works?

Thank you very much!

[Guest]

Hello,
In short, the answer to question 1) is yes.

The AVRcamVIEW program automatically adds a little "wiggle" room to the values of each color added to the color map. This helps detect the colors even when there is a little variation in one of the color channels. You can manually remove this wiggle room if you would like so it is more precise. However, the finest granularity you can do with the AVRcam is a range of 16 for any single color channel. So by manually setting a color channel to a single value for its high and low values (say, red channel to 16 for both high and low), you are effectively saying that and red values between 16 and 31 will be accepted.

Tracking black objects can be a difficult task because there is usually so much background black in a normal scene. The AVRcam works best with colors that are very distinct from the background in which they are working. Have you tried placing the black object in front of a large white piece of paper, and doing a frame dump to see the resulting view from the camera's perspective? You should be able to track black objects in this case. As a side note, I have tracked white objects on a black background very successfully. This is how I did the line-following in the video shown at http://www.jrobot.net/Projects/EyeBo3_with_AVRcam.mpg

The "instability" of the bounding boxes happens if there is any camera movement, any motion in the scene, any change in the lighting conditions, etc. The lighting conditions changing can sneak up on people, not realizing how the world is changing from the camera's perspective. I would suggest taking lots of snapshopts in different configurations to really understand how the camera views its world.

As far as the tracking process, the AVRcam scans each image line pixel by pixel, looking for contiguous blobs of RGB triplets that match what was set in the color map. As each line is completed, it attempts to connect any color blobs found in the most recent line with any currently tracked objects in an object tracking table, by vertical connectedness. If it finds connectedness, it updates the bounding box of the currently tracked object. If no connectedness exists, it will add a new object to the tracked object list.

Good luck, and post back if you have more questions.

[kiteswing]

John, Thank you so much for your quick and detailed response. It is so helpful to me.

Here, I have some more questions.

1. What is the meaning of the funcions “CamIntAsm_acquireTrackingLine" and "CamIntAsm_acquireDumpLine"? What is the value in "currentLineBuffer", "previousLineBuffer“ and "colorMap"? And, what is the runlength in the buffer?

My understanding is: CamIntAsm_acquireDumpLine is used to read the 176 color data into the currentLineBuffer, and put the previous 176 colors into perviousLineBuffer. CamIntAsm_acquireTrackingLine is used to read the 176 color data into currentLineBuffer and update the value in colorMap. Is it right?

2. Will “CamIntAsm_acquireTrackingLine “ publish the event “FEV_ACQUIRE_LINE_COMPLETE” in the end? In your code, I can not find anywhere that explicitly publishs this event, which brings the action of FrameMgr_processLine().

3. Can you explain the the Bayer format that arrange the pixels? Is it like the following?

Line 1: g r g r g r g r …………
Line 2: b g b g b g b …………
…………
Line 144: b g b g b g b …………

Lets use g(mxn) representing the green value of pixel n in line m. Can I say that pixel stacks in the packet is structured as

Packet1: g(1x1), b(2x1), g(2x2), r(1x2), g(1x3), b(2x3)………
Packet2: g(3x1), b(4x1), g(4x2), r(3x2), g(3x3), b(4x3)………
……………..

4. I am confused on the dataToSend in the function FrameMgr_processLine(). Why when i is even, the low 4 bits currentLineBuffer is in the high 4 bits of dataToSend, and when i is odd, it is opposite? A color value can be 0~255, but 4 bits in only 0~15. So, why you just use four bits to represent to green/red/blue value? what is the true RGB value?

I have so many questions. Thank you in advance for your help!

[Guest]

1. What is the meaning of the funcions “CamIntAsm_acquireTrackingLine" and "CamIntAsm_acquireDumpLine"? What is the value in "currentLineBuffer", "previousLineBuffer“ and "colorMap"? And, what is the runlength in the buffer?

CamIntAsm_acquireTrackingLine is the function that is used to perform the sampling of the digitized RGB values from the camera during tracking, and the needed mapping of the RGB triplet into an actual color as well as the run-length encoding of the data. The _acquireDumpLine is the routine used when the user performs a dump frame operation (snapshot) to send an entire image out to the user. These two functions are total separate, and when performing tracking the dump function is never used, and vice versa.

During color tracking, the currentLineBuffer is a buffer used to hold the run-length encoded color information calculated when a line is sampled. The previousLineBuffer is only used during frame dumping, in addition to the currentLineBuffer. They are used to sample the Nth and the N+1th line in the image. The data actually comes out of the camer in the following format:

Line N: ...GGGGGGGGGG...
Line N+1:...RBRBRBRBRBR...

So an RGB triplet is composed of data from lines N and N+1.
The colorMap is used as a lookup table to determine what color a particular RGB triplet maps to...brown, yellow, purple, etc., depending on how the user sets the map up.

The runLength in the buffer is the the number of consecutive RGB triplets in a particular set of lines that map to the same color (i.e., brown, yellow, purple, etc).

2. Will “CamIntAsm_acquireTrackingLine “ publish the event “FEV_ACQUIRE_LINE_COMPLETE” in the end? In your code, I can not find anywhere that explicitly publishs this event, which brings the action of FrameMgr_processLine().

No, the function doesn't explicitly publish the FEV_ACQUIRE_LINE_COMPLETE. This event is published from the Timer1 overflow (look at SIG_OVERFLOW1 in CamInterfaceAsm.S). Timer1 is setup to overflow after a complete line of pixels is sampled (Timer1 is being clocked by the PCLK line coming from the camera). Thus, when all the pixels in the line have been sampled, the Timer1 counter will overflow, interrupt, and the ISR will publish the event.

3. Can you explain the the Bayer format that arrange the pixels? Is it like the following?

See above...

4. I am confused on the dataToSend in the function FrameMgr_processLine().

dataToSend is used only during line dumping. The top nibble of the byte corresponds to the sample in the currentLineBuffer, and the lower nibble corresponds to the sample in the previousLineBuffer. Yes, only 4-bits of each color are sent (the upper 4...the lower 4 aren't even hooked up to the mega8). Among other things, during dumping this allows corresponding samples from two adjacent lines to be sent in a single byte. So, with 144 total lines in an image, we can compress it during dumping so it fits in 77 lines worth of data.

Keep studying the source code...all the answers are there

Good Luck!