Kodak Competitive Analysis, DVC-323 vs Intel, Connectix, US Robotics
Kodak scientists dissect the competition...
With the DVC-323, Kodak knew they had a good thing, and wanted to be sure reviewers understood just how good. To that end, we received a "Benchmark CD" along with our test sample of the camera itself. We thought the results Kodak found would be interesting to our readers, and so are passing it along here. Along with a performance analysis of the various cameras, there's an excellent general discussion of what to look for in a digital video camera.
The following is the exact text from the "readme" file on Kodak's benchmark CD. We've inserted links to JPEG versions of their still-image files. (Unfortunately, we don't have the ability to link-in the video files which were also on the disk.) While the original images on the CD were in the uncompressed Windows "BMP" format, we have converted them to JPEG for display on the web. To avoid loss of image quality though, we have used a minimal JPEG compression (a setting of "8" in Photoshop). This results in images somewhat larger than would normally be saved by the camera applications in normal operation. The only consequence will be somewhat longer download times.
NOTE: The following information is provided by Eastman Kodak Company, and The Imaging Resource makes no representation as to its accuracy or veracity. Text and linked images below this line are copyright Eastman Kodak Company, and may only be used with their permission.
All the cameras in this study are USB (Universal Serial Bus) devices except the U.S. Robotics camera which requires installation of a video capture board. The Kodak cameras were a random sample from a retail shipment. All other products were obtained through retail stores or mail order. The following cameras were tested:
* Connectix QuickCam VC
* Intel Create & Share Camera Pack
* Kodak DVC300
* Kodak DVC323
* U.S. Robotics Bigpicture Video Kit
Eastman Kodak Company authorizes the use and distribution, in paper or on-line, the image and video files on this CD for editorial, product promotions, and sales training. Use for other commercial purposes requires prior written permission. Permission requests should be sent to:
Eastman Kodak Company
Contracts Administration - D&AI
901 Elmgrove Road
Rochester, NY 14653-5241
Fax #: 716-726-0500
The balance of this README is organized as follows:
1. How this CD is organized
2. How the cameras were tested
3. What to look for when comparing the cameras
4. Our comments on comparing the cameras
1. HOW THIS CD IS ORGANIZED
(Ed: Only some of the still image files from the CD are included here)
There are 3 main folders: Stills, Videos, and NTSC chart. Inside each folder
are samples organized by camera. The "Stills" and "Video"
folders show a "-f" or "-i" after each camera to indicate
the type of lighting the samples were shot under ("-f" = fluorescent,
and "-i" = incandescent). The "NTSC" chart folder contains
samples from shooting the EIAJ test chart A. The NTSC folder is again organized
by camera with a "-stl" or "-vid" after each camera to indicate
the type of file the NTSC chart was used for ("-stl" = still, and
"-vid").
2. HOW THE CAMERAS WERE TESTED
All cameras were tested on identical high end PCs to minimize effects due to
PC hardware speed limitations. All stills and video clips were made using the
best/highest quality settings that were available with each camera and their
respective software packages. Each camera's automatic exposure (brightness)
and color balance (white balance) settings were used. Any cropping or variation
in the frame rates are situations that occurred upon actual capture of those
respective stills/videos. Settings and framing were matched as closely as possible
upon initial set up and focusing of the images. Each camera was set up in the
same manner, and under identical lighting conditions. All of the raw video files
were compressed using the Radius Cinepak codec since that is included on all
Windows '95 systems.
A level of 100 lux was used for the incandescent lights. Our incandescent lights were three drafting lamps with white reflectors, using standard 60 watt soft white household light bulbs. These were pointed at the scene from above and behind the camera. Lighting was adjusted to provide 100 lux of illumination in the scene. The lamps were approximately 6 feet from the subject. This corresponds to a fairly bright home environment, such as a kitchen or well lit home office. Incandescent home lighting often ranges from 50 lux (fairly dim) on up to 200 lux.
A level of 200 lux was used for the fluorescent lights. Our fluorescent lights were Sylvania Cool White deluxe super saver tubes. These were selected because they are common fluorescent lights. A fixture with six tubes was placed approximately twelve and one half feet from the subject. The light fixture was pointed at the scene from above and behind the camera. Lighting was adjusted to provide 200 lux of illumination in the scene. This corresponds to a nominal office. Office lighting often ranges from 150 lux on up to 400 lux. The NTSC chart shots were only done under fluorescent lighting.
3. WHAT TO LOOK FOR WHEN COMPARING THE CAMERAS
The key attributes to look for are:
Aliasing:
It is where single edges appear unnaturally jagged and sometimes with artificial
coloration. It is the artifacts that occur where high frequency content (fine
detail) in the scene is reproduced (incorrectly) at a lower frequency. Textures
and sets of lines (such as in a resolution target) will have artificial coloration
and low frequency patterns in the image. A digital image with no aliasing will
reproduce a fine texture as auniform area.
Color Reproduction:
It is the ability to reproduce colors accurately, or at least in a natural way.
Desaturated colors are colors that tend to collapse toward gray and become less
colorful. The Macbeth color checker chart is the key item to look at in the
scenes. This chart has a row of neutral patches (white to black) on the bottom.
The next row above it is a row of saturated primary colors: blue, green, red,
yellow, magenta, and cyan. The top two rows include various real world colors
of interest: flesh tones, foliage, and such.
Falloff:
It is the darkening of an image as you get away from the center of the image.
All cameras have some amount of darkening of image areas off the optical axis
of the lens. This can come from several sources, but is usually controlled by
the optical design of the camera.
Frame Rate:
The number of video frames captured per second, referred to as frames per second
(fps).
Geometric Distortion:
It is producing an image where the geometric patterns don't match the original
scene. For example, straight lines in the scene become curved in the image,
or squares in the scenes become something else in the image. Geometric distortion
is commonly tested with a flat target using squares, rectangles, and straight
lines. The most common forms of geometric distortion are anisotropic magnification
and barrel or pincushion distortion. Anisotropic magnification means different
vertical and horizontal magnification, producing an image where a square becomes
a rectangle. Barrel distortion is the term for corners being pulled in toward
the center of the image. Pincushion distortion is the term for corners being
pushed outward from the center of the image. All of the images on this CD show
some amount of barrel distortion, which is common in cameras with a fairly wide
viewing angle.
Noise:
It is the appearance of random or patterned grains or specks in the images or
videos. In photographic imaging, this is typically called grain.
Resolution:
It is the ability to resolve, or distinguish, the finest possible details. It
is often tested with sets of lines, and is usually expressed as line pairs per
picture height or a similar metric. Resolution targets often include sets of
lines of varying width - fine at one end, and thicker at the other end, with
a somewhat fan-like appearance. In the NTSC chart used here, the numbers on
the chart near the resolution lines (e.g., 200, 300, 400) correspond to lines
in one picture height. For example, the lines labeled 200 are spaced so that
200 alternating white and black lines would just about fill the height of the
image - 100 white lines, 100 black lines. The largest frequency ramp featured
in the center of the target runs from 200 lines per picture height to 400 lines
per picture height. The resolution target used here also has sections with lines
of constant spacing, providing an area to check resolution at a fixed frequency.
Sharpness:
It is the clarity of the edges, especially in details. Sharpness is related
to resolution, but more precisely is the overall impression of crisp edges and
details. On this CD, only the 640x480 captures of the NTSC chart show any significant
divergence between sharpness and resolution.
WYSIWYC:
(What You See Is What You Capture). All of the cameras provide a live video
preview window on your monitor to compose the stills and videos you capture.
You want the full contents of that live preview window to actually be contained
in your digital file. Also if you videoconference, you want to know what the
person on the other end of your videoconference is really seeing.
4. OUR COMMENTS ON COMPARING THE CAMERAS
We recommend viewing the still images and videos using a good quality 24 bit
(True Color) display. Viewing them on a 16 bit (High Color) display will introduce
additional noise and quantitization artifacts. A "Compare" folder
has been included for the Stills. It contains a single file which simultaneously
shows the images from each camera. To simultaneously compare videos, go to Programs-Accessories-Multimedia-and
open Media Player. Open one of the avi files. Under Edit-Options, turn on "auto
repeat" and select play. Go back to Accessories and open a second copy
of Media Player. Repeat the prior steps for your second avi file. Keep doing
this until you have the videos from all 5 cameras simultaneously running (if
your CPU is still alive!). All of these videos can be somewhat jerky in playback
if the PC you are using is not fast enough. You can check for consistent video
timing on a slower PC by single stepping through the video, looking at each
frame (in Media Player, select "Scale", set it to "Frames"
and then use the slider to view 1 frame at a time) . Look for small and even
differences in fan position between each frame. The second hand of the clock
is not a sweep second hand, so it's motion is jerky in all videos.
STILL PICTURES - Doll Scene
One of the major differences between the cameras is color reproduction. A number
of the cameras show desaturated colors.
Another clear difference between the cameras is sharpness. For sharpness look for clear details in the hairband and the eyes on the doll. The resolution chart in the lower left corner has other details to compare sharpness. Also look at the starburst. The Kodak DVC300 and DVC323 show significantly more sharpness. In fact, they clearly resolve the coarse (rightmost) section of the aliasing target, indicating resolution superior to the other cameras.
Compare highlights by looking at the doll's dress and the tablecloth.
Just to the right of the Macbeth chart is a target with fine vertical lines to check for aliasing.
All the cameras produce images that are slightly less saturated under incandescent light than under fluorescent, though this difference is fairly small compared to the differences between the cameras.
The automatic white balance algorithm in the Kodak DVC323 driver is significantly better than the one initially released with the Kodak DVC300 which is why the Kodak DVC323 is showing better color balance.
Some of the cameras are showing very visible noise under incandescent light. This difference in noise is also present in the fluorescent captures, but is less obvious.
All of the cameras except the Intel do a very good job of WYSIWYC. The actual file from the Intel camera crops off the entire bottom and the entire right hand side of your live video preview. Look at the right hand side of the Doll scene and see how the chair and frequency chart are just about cut in half compared to the other camera files. Look at the bottom and see that the table surface is not there.
STILL PICTURES - NTSC Chart
(Ed. note: NTSC stills not available on Imaging Resource site.)
All of the cameras were set-up to make sure that the digital file they captured
included the full NTSC chart. The sharpness and geometric distortion differences
between the cameras shows up even more clearly than in the doll scene. The image
from the Connectix VC is quite soft, and it doesn't clearly resolve even 200
lines per picture height. That is, the 200 line sample areas show aliasing artifacts,
which get worse in the higher frequency areas. The Kodak DVC300 is much sharper,
and clearly resolves to between 300 and 400 lines per picture height. The Kodak
DVC323 has similar resolution, though higher contrast in the DVC 323 makes its
image look better and appear sharper. The Intel camera shows less sharpness
than the Kodak cameras but probably more than the Connectix VC. The U.S. Robotics
camera shows good sharpness, though less than the Kodak DVC323.
The Connectix VC shows aliasing artifacts at 200 lines per picture height, and doesn't really resolve it clearly. The aliasing gets worse with higher frequencies, up to about 400 lines per picture height, when it just loses the lines completely. The Connectix shows significant coloration in several areas, where the high frequency detail is mistaken for color information rather than fine detail. The Kodak DVC300 shows some aliasing from about 400 lines per picture height up through about 600 lines per picture height, but it is not very noticeable. The Kodak DVC323 aliasing is similar to the DVC300. The Intel camera shows slight aliasing between 200 and 400 lines per picture height, but relatively little artificial coloration. The U.S. Robotics camera shows some aliasing above about 250 lines, but shows less of it than the Connectix VC.
Another difference shown by the NTSC chart is geometric distortion. The chart is a precise rectangle with 3:4 aspect ratio, which should perfectly fill these images. In the Connectix VC images, significant barrel distortion is present. Also, the Connectix VC camera has slightly different magnification in the horizontal and vertical directions. This can be seen by looking at the size of the large square defined by the inner edges of the 4 gray scales. In the Connectix VC image, the square is compressed about 7% in the horizontal direction. In the Kodak DVC300 and DVC323, very little geometric distortion is present, either in magnification or barrel distortion. The Intel camera shows more barrel distortion than the Kodak cameras, but pretty consistent horizontal and vertical magnification. The U.S. Robotics camera shows the most barrel distortion, and also shows some horizontal and vertical magnification difference. The large square is stretched by about 3% in the horizontal dimension.
OVERALL RANKING OF CAMERAS FOR STILL PICTURES:
In our opinion, the Kodak DVC323 is the best camera due to its superior sharpness,
low geometric distortion, and excellent color. The Kodak DVC300 probably would
be second, though it's automatic white balance could use improvement. The U.S.
Robotics camera would probably come next. The Intel and Connectix VC cameras
are a fair distance behind the U.S. Robotics, due to their limitations with
color, sharpness, and geometric distortion.
VIDEOS - Fan Scene
All of the cameras were set-up to make sure the actual digital video files being
captured had the same scene contents. Again, all of the camera live video previews
are very good at WYSIWYC except the Intel. It needed quite a bit of repositioning.
All videos on the CD have some contouring and other degradation visible due
to compression using the Radius Cinepak codec. You can usually see this contouring
in the smooth gradient in the white paper around the Macbeth chart. The original
(uncompressed) raw camera videos do not show steps in this area, but rather
a smooth gradient. Radius Cinepak compression was used on all files so that
the videos on this CD can be played on any Windows '95 computer. A number of
the uncompressed files can only be played back in their own application software.
One simple difference between the cameras is frame rate. The numbers are from checking the time span of (and the number frames) in the video as shown in the Microsoft Media Player application. Almost all of the camera/software products allow a user to make a trade-off between frame rate and video quality. Although higher frame rates can be obtained for the video capture sizes below, our settings were trying to obtain the fastest frame rates for "identical visible best" video quality.
Connectix VC |
10 fps |
|
|
|
|
|
|
|
30 fps, though tagged as 15 fps. In Microsoft's Media Player this video is labeled as being about 20 seconds long (and it has about 300 frames). However, as the clock in the video shows, it spans about 10 seconds of real time. |
Fan movement becomes more smooth as the frame rate increases, though this may be less apparent if played on a slower PC. The streamers on the fan show more differences. The streamers move more smoothly and are less blurred at higher frame rates.
One obvious visual difference between the videos is the sharpness. Look at the numbers on the clock.
Another difference between the cameras is aliasing. This is also apparent in the numbers on the clock and in the resolution target under the clock. The Kodak DVC323 has significantly less aliasing than the Kodak DVC300. The Intel camera shows almost no aliasing, but has less sharpness than any other camera. This is particularly clear in the wires around the fan blades and the circles at the corners of the resolution target. Both are almost lost in the Intel 320x240 video. The U.S. Robotics camera clearly has the best sharpness for 320x240 video.
The different videos also show the differences between color reproduction of the cameras.
Another difference between these videos is falloff. Some of the cameras show a darkening of the corners or edges of the image. This effect is visible to varying levels in each of the videos and still captures. The effect is relatively unimportant when illumination is even, but becomes objectionable when scene illumination falls off toward the side. For close up shots of faces, this effect is insignificant. For wider scenes, adjustments to the lighting can easily minimize the effect.
VIDEOS - NTSC Chart
The most obvious differences between the videos are in sharpness and aliasing.
The Connectix VC shows significant aliasing and other artifacts. Observe the mis-registration of some colors, the jaggedness of the diagonal lines, and the coloring of the frequency ramps. The Kodak DVC300 shows much better sharpness but quite a few aliasing artifacts. The Kodak DVC323 has sharpness similar to the Kodak DVC300, but significantly less aliasing. It especially shows less coloration of the frequency ramps. The Intel is similar to the Kodak DVC323 for sharpness and aliasing. The U.S. Robotics has the best sharpness but shows aliasing artifacts comparable to the Kodak DVC323 and Intel cameras.
Another difference shown by the NTSC chart is geometric distortion. The chart is a precise rectangle with 3:4 aspect ratio, which should nearly perfectly fill these images. In the Kodak DVC300 and Kodak DVC323, this is essentially true - very little geometric distortion is present. The Connectix VC, Intel, and U.S. Robotics cameras all show varying amounts of barrel distortion.
Also, the Connectix VC and Intel cameras show slightly different magnification in the horizontal and vertical directions. This can be seen by looking at the size of the squares in the center of the image.
OVERALL RANKING OF CAMERAS FOR VIDEO:
In our opinion, the U.S. Robotics analog video camera appears to be the best.
However, you have a higher price and the hassle of installing a video capture
card. Kodak's DVC323 is the next best. It is the first USB digital video camera
that approaches the video performance of a high quality analog video camera
and capture card solution.
The Kodak DVC323 is also the only USB camera that performs well for both 160x120 and 320x240 video. The Intel is fine for 160x120 but worse at 320x240. The Connectix is fine for 320x240 but worse for 160x120.
Follow Imaging Resource: