Scan image addition

[Jan Valcke]

Hi specialist,

Im a hobby robotics engineer that will be part of the melexis safty trophy 2006 http://www.trophy.melexis.com. In this contest the main problem is to track road signs and traffic lights. Now i think the AVRcam can do the job with some modifications in the embedded software.
To use the AVRcam to track the traffic lights will not a problem with the standard configuration. It is only a color to scan for so no problems. To track the road signs is another story. A road sign exsist of a main color(blue or red) and on top a symbol like an arrow or text in white color.

So my question is how can i modify the source code to recognize different symbols on the tracked color object(blue or red). Would it be possible to scan each line of the boundry box of the object to see changes of color at a specific location in the box. with this information i coud analyse what the symbol means. Are there exsisting algoritms for such an application. Anyway are here on this form people that can give me hints how a can realise this in sofware because i dont have understanding of program.
I need some assistance to explore the software.

Many thanks for helping me out here.

[Guest]

To track the road signs is another story. A road sign exsist of a main color(blue or red) and on top a symbol like an arrow or text in white color.

So, it sounds like it should be possible to configure the camera to look for blue, red, and whatever color the symbol would be (as well as white to cover the text). This seems doable, as long as the symbol maps to a specific color, or will always be found relative to some other color (i.e., a yellow symbol is always right above a red sign...you would then post-process the tracking packets to look for red and yellow within a certain distance of each other). Remember: the system provides bounding box information currently, so each tracked object appears like a rectangle to whatever entity is connected to the system. If you want to do shape detection, I would perhaps consider an alternate solution, or, as you said, modify the code to do additional shape detection (but this may be too much for the current system to handle).

[eddy]

A few questions for you:

How fast do you need to recognize the signage? Are you going to be moving pretty fast or very slow?

Can you use more than one camera? Like you mentioned, tracking stop lights works with the default config. Would you use a second camera to recognize the road signs?

I think this is doable but not at 30fps. If you could get by with 1fps, you'd have a lot more processing time to handle the shape recognition.

Developing this type of software application could be a real challenge even for a veteran programmer. Does anyone else on your team have programming experience with either PCs or microcontollers?

Question for John: how quickly could you transfer a raster image to another processor using SPI, TWI, RS-232, etc? Another processor could analyse each line of the raster and capture the 2 points (hopefully there is only 2) that define the edge of the shape. This data would be small enough for a microcontroller to handle - basically the coordinates of the outline of the shape.

What if you uploaded the raster image to a small laptop or PC to handle the shape recognition and feed it back to the microcontrollers?

That's about all the ideas I have!

[Guest]

Question for John: how quickly could you transfer a raster image to another processor using SPI, TWI, RS-232, etc? Another processor could analyse each line of the raster and capture the 2 points (hopefully there is only 2) that define the edge of the shape. This data would be small enough for a microcontroller to handle - basically the coordinates of the outline of the shape.

There are some possibilities here, especially if the user can deal with reducing the frame rate somewhat. The AVRcam goes through and samples the incoming pixels, maps each RGB triplet to one of the 8 possible colors (or to the "not currently tracked" color if it isn't in the color map), and then run-length encodes each line of pixels. All of this is done on the fly, in 16 clock cycles, in assembly code. After each line is complete, the AVRcam has a run-length encoded version of the entire line. In a super noisy image (with lots of color changes), this run-length line can be pretty big, but normally, it is quite small, maybe 3 or 4 trackable changes at the most (which results in 6 or 8 bytes of encoded data, since it stores the color + the number of pixels of that color that follow). If you could live with a reduced vertical resolution (say, half the current number of lines, around 77), and with a reduced frame rate (say, 15 fps instead of 30), you would probably be able to send out the run-length encoded version of each line that would give you enough information to do the actual shape recognition desired. This would be the only data sent back over the user-interface (there would be no tracking packets...this would be an entirely new type of packet).

As Eddy mentioned, even if this information was streaming out of the AVRcam, the post-processing required to do the shape detection is non-trivial. Thats not to say that it can't be done, but it is definitely a significant effort.

Good luck, and keep us posted...

[Jan Valcke]

Hi,

By tought, maybe it can be simpler done. Is it possible with the standard configuration that 2 boundry boxes of different color overlap each other?

The symbols like the arrows are not placed symetric on the main bord. So with an arrow to the right the white boundery box will be more to the right then the middle line of the main blue bord. An arrow to the left will be more to the left of the main bord. If i measure the difference between the left and right of the boundry boxes than i can dicide what the symbol means. There will also be an arrow that go straight ahaed. The diffrence of each side will be minium with that type.