Agreed, working in YUV is definitely a better color space to use compared to RGB. It is a much better space when the user is dealing with variations in lighting conditions. The AVRcam will work fine in the YUV space (just requires a single register tweak in the OV6620). The main problem I ran into is that I can't visualize YUV like I can RGB. Comparing luminance and chroma values are essentially meaningless in my head when it comes to figuring out how to represent, say, light orange. So the decision to use RGB wasn't based on technical issues, rather usability issues.
One thing to note here: the AVRcamVIEW PC application is set up to use RGB values only. Thus, dumping an image in YUV color space will look odd.