The Library of Congress / Ameritech National Digital Library Competition
(1996-1999) |
QUALITY REVIEW OF DOCUMENT
IMAGES
INTERNAL TRAINING GUIDE
National Digital Library Program, Library of
Congress
September 1996
Contents:
The purpose of this training guide is to describe for the image reviewer the imaging
process, document the standards for quality review of images, provide step by step instructions
for the quality review part of the process, and define terms related to the imaging process.
Imaging is the process by which digital images of objects (such as books, maps, papers,
and photographs) are created to reproduce the object. We create digital images so that they can
be placed online. The images are created digitally by "scanning" the original. The scanning is
done by a machine that works in a manner comparable to a copier; the object has a "picture"
made of it. Objects can be scanned at different settings controlled in the scanning software and
adjusted by the scan operator, depending upon the requirements for each image. Settings include
choices like bitonal, halftone, and different levels of grayscale as well adjustments for brightness,
density, and contrast.
The imaging for the Library is done by outside contractors. The contractors provide
scanning equipment, scanning personnel, project managers, and other experts who design the
programs or assist technically. Their job is to scan the original document, process it, do a
thorough quality control check, place it on an optical disk as an image file organized by batch,
and return it to us.
This is the point where you, the image reviewer, come into the picture. You will be
responsible for checking the contractor's work for the types of errors that will be discussed in the
imaging standards section of this document. If you find errors or have doubts about any image
you should make a note of it and print out the image. Once you have completed reviewing a
batch, prepare a memo for the quality review supervisor that lists the errors or questionable
images and include the printout with your memo. The quality review supervisor makes a
judgement based upon an examination of the original and if the contractor has made mistake, e.g.,
scanning a printed halftone without treatment (e.g., with a special pattern called diffuse
dithering), the image is returned to the contractor, corrected to our specifications, and delivered
to us as rework.
At this point in the development of the National Digital Library Program, printed
documents like book pages are scanned to produce one of three types of images: bitonal, bitonal
with diffuse dithering, and grayscale.
Bitonal images are produced when the original is simple text (typography) or line art,
e.g.,
drawings or charts rendered without tones of gray. Such originals have only two shades, black
and white, that need to be shown. Pure black and white can be encoded with a single bit for each
dot (a dot may also be referred to as a pixel, short for picture element) where "0" is white and "1"
is black. The NDLP typically renders bitonal images at a resolution of 300 dots per inch.
This may be contrasted with the handling of digital images of printed halftones. Printed
halftones are also bitonal originals in that they are created by black ink on white paper. But the
grid pattern of dots that permits the human eye to see them in shades of gray requires that they be
treated differently. Halftones are the black and white photographs that you see in the newspaper.
If you look closely, you will see they consist of black dots; densely packed to simulate black,
sparsely packed to simulate gray, and absent to simulate white.
When the printed halftone's pattern of dots encounters the raster or grid used by the
scanner, the result is an image with dark waves called moiré patterns, a form of aliasing.
To
inhibit such an outcome, one or more processes may be used. One example is diffuse dithering;
a diffuse pattern of dots randomizes the scanner's pattern and thus smooths out the resulting
image and suppressing the moiré pattern. Another example is a more complicated process
of
descreening and rescreening. The NDLP typically renders images including a printed halftone at
a resolution of 300 dots per inch.
The third type of image you may encounter is called grayscale. In a typical grayscale
image each dot may have two or more values (remember, in a bitonal image each dot can have
only one value). If a 2-bit depth is selected, each dot is capable of representing four different
levels of gray. If an 8-bit depth is selected, each dot is capable of representing 256 different levels
of gray. The objects commonly scanned as grayscale in the National Digital Library Program are
manuscript pages or photographs, and the spatial resolution may run from 100 to 300 dpi.
Images are stored in files that adhere to certain format rules or guidelines. In addition,
various compression algorithms may be used.
For National Digital Library Program document images, three file and compression
formats predominate. Most bitonal images are stored in TIFF files with CCITT (fax) Group 4
compression. The filename extension is .tif. When the Xerox 5200 scanner is used to produce
diffuse dithered imges of printed halftones, the PCX format (with its "native" compression) is
used. However, a PCX file may contain an undithered image because we have permitted
contractor to use PCX for all illustrations. The filename extension is .pcx. Uncompressed
grayscale images are stored in the TIFF format; compressed grayscale images are processed with
the JPEG (Joint Photographic Experts Group) algorithm and stored in the JFIF (JPEG File
Interchange Format). The filename extension is .jpg.
The guidelines in this section will not be a detailed
list of every possible problem you
will encounter in the review of images because there are so many that any list would be very long,
very dull, and, in the end, incomplete. Rather the goal is to provide you with a set of guidelines to
use to assist you in making judgments on images while performing quality control.
The signs of poor imaging include:
- Stark black and white contrasts on anything except simple line
drawings. A simple line drawing is a drawing where the lines are
not compact (less than one/24 inch apart) and any attempts by the
publisher to have grayed shading in the illustration is minimal;
- Loss of distinct lines, including hairlines, profiles, skylines,
horizons, etc., that are a result of a lack of dithering or too light a
scan.
- Loss of features such as, in the case of an illustration of a person,
eyes, ears, and eyebrows. If you are looking at the illustration of a
building pay attention to the windows, the landscaping, etc.;
- Skewing that is significant and not the result of the tight binding of
the book;
- Text that is cut off, including page numbers;
- Illustrations without a caption.
Other obvious problems include alignment problems, compressed text, or lines
or blotches that
appear not to be a part of the illustration. What we call alignment problems are instances where an
image has shifted so that the borders are curved or jagged. A good indicator that the blotches and
lines are not part of the illustration is that they often appear numerous times over many successive
images.
Examples of these problems are attached to the paper version of this document.
Depending upon the image, we use one of three types of software to view images,
ViewDirector, PhotoStyler, and HiJaak Pro. ViewDirector is the preferred software for viewing
TIFF images. Both TIFF and PCX can be viewed in ViewDirector, but it does not scale images
to 1:1 with reliability. For undithered images this is not a problem, the reviewer can still get a
very good sense of the image quality. ViewDirector's advantage is that it is fast.
PhotoStyler is the software used to view PCX and JPEG images. PhotoStyler's
disadvantage is its inability to allow the viewing of TIFF images when compressed with CCITT
compression. Its advantage is that it does allow the viewing and the printing of an image at a 1:1
scale.
A third software package that is useful is HiJaak Pro. It allows the viewing of files in both
the
TIFF format and PCX format, and enables the reviewer to open many files at a time ( I've opened
more than twenty at a time.). In addition, its resolution is sharp and clear for both file formats.
HiJack's disadvantage is its lack of flexibility for manipulating the image and the effort it
sometimes takes to print an image at 1:1.
This section will guide you step-by-step through the process of image quality review.
- When the optical disk and paperwork are received from
the
contractor , a file is created based on its batch number. Batch
numbers tend to group together based on the collection being
scanned. For example, the Margaret Mead collection we are
creating starts with batch number 130. Other batches will be
numbered differently, say batch 150 and above. The batch files are
stored in accordion folders over my desk and are stored
sequentially. The optical disks are stacked next to the folders and
are numbered by disk, not by batch number.
- Get the file folder for the batch you want to work on from the
accordion folder.
- Check the contents of the folder. You should find a directory
list of the files in that batch, a letter telling you which shuttle disk
the batch is on, and a log in which the contractor has indicated any
problems they encountered during the scanning of the object. You
should refer to the log when you encounter a questionable image to
see if the contractor has noted the same image. If they have, note
their log entry in your memo.
- Place the shuttle disk in the appropriate optical disk drive. The
optical disk drives are at the side of the terminal and resemble 8-track tape players in
appearance. The disks you will be working
with can function in any of the drives we have, but sadly, none of
your 8-track tapes will.
- From the window on the computer screen select the appropriate
software icon for your viewing purposes and open it with a double
click. This will vary between two different software packages
depending on the types of images that you will be performing
quality review on. If you are reviewing TIFF files, you will open
the ViewDirector software. If you are looking at PCX files, you
will open PhotoStyler software. You may also use HiJaak Pro for
looking at both TIFF and PCX files.
- Once you have opened the appropriate software for your file
type, choose the drive path that contains the optical disk. You can
do this by pulling down the drive window in the dialogue box and
clicking the letter of the optical drive that contains the optical disk.
The letters should be on the optical drives over the mouth of the
drive.
- Choose the batch you are reviewing from the directory and open
it with a double click.
- After opening the batch, the files contained in that batch will
display in the directory. Select the file you wish to review and open
it with a double click.
- The individual image files will now be displayed for the file you
selected. Make sure the filenames adhere to our filenaming
convention, which means there should be eight characters in the
filename. Select the first image file from the directory. For TIFF
files this should be the target page, that is, the page that contains
the title of the document you will be reviewing. Make sure that the
target page matches the title page. The title page will usually be the
second TIFF file you view, but it sometimes appears later. The
important thing is to make sure they match. In the PCX format the
first image file will be (hopefully) the first illustration in the
document.
- Do a 100 percent quality review of all images that have a ".
PCX" extension and a 10 percent check of all images that have a
".TIF" extension. Avoid just going to every tenth ".TIF" image
file, but select image files as randomly as you can. If you see a
problem with one file, check the surrounding image files. In many
instances problems occur in groups and we are very concerned with
finding out the extent of problems so that they can be fixed. You
may also look at clumps of image files, but your 10 percent check
should range over the whole document in some way or another.
- Look for image quality problems, following the approach described in
the section "Imaging Guidelines
for the National Digital Library." Note any problems you see,
print out a copy of the image and make a note of the problem to be
included in the memo that you will write describing your review. It
is a good idea to zoom in and out as necessary and otherwise move
around the image to get a better look at it. The more ways you can
look at an image, the better your quality review will be.
- Check if the relationship between filename and pagenumber
changes. If it does, find out why. For instance, if file
xxx001.tif is page 1, this relationship would be expected to
be maintained throughout the document.
Also, check the last image in the directory just to see if it appears
as though it is the last page of the document.
- Write your memo. The batch name is the subject. List any
problems (including questions) you encountered by image filename
and attach printouts to the memo. An example of a review memo is
attached to this document.
- Deliver the batch folder and memo to my desk for my review.
- Your part is done unless I have questions or comments about
one or more of the images you cited as problematic, in which case I
will get in touch with you.
This list may seem long, but most of these steps you will know after your
first time
through the process.
This section will acquaint you with some basic terms used in imaging. It's important to
know what they are and when they apply to an image.
- Algorithm
- A rule (often mathematical) governing computer
processes.
- Artifacts
- Visual effects (usually thought of as defects) introduced into a digital
image in the course of scanning or compression that do not correspond to the
image scanned.
- Batch
- In the imaging process as we do it, the grouping of books, photographs, or
other objects into an organizational unit for our reference only. Each collection
has its own distinct series of batch number assignments.
- Binary image
- A computer image where each dot may have just two values, either
black ( 0', for the absence of light) or white ( 1' for the presence of light).
Suitable for printed text or line art because only the values of black and white are
needed for these types of images. Synonym for bitonal image.
- Bit-mapped image
- An image created from a series of bits that form pixels. Each
pixel can vary in color or gray-scale value.
- Compression
- A process that reduces the file size for processing, storage,
transmission, and display. Compression may be lossless (e.g., CCITT) or lossy
(JPEG). The quality of the image may be affected by the compression techniques
used and the level of compression applied.
- Diffuse dithering (see dithering)
- Digital image
- An image composed of bits and bytes.
- Digitizing
- To convert an image into binary code. Visual images are digitized by
scanning them and assigning a binary code to the resulting bit-mapped image data.
- Dithering
- A technique for creating an image with apparent levels of gray by
grouping black and white dots into cells. According to the number of black and
white dots and their arrangement in the cell, the eye perceives the cell as a single
gray, not as a group of black and white dots. Commonly used by laser printers to
simulate grey images. The NDLP use of diffuse dithering represents a special case
in which the scanner's dot pattern is randomized.
- DPI
- Dots per inch. It is a measurement of the scanning resolution of an image or
the quality of an output device. It expresses the number of dots a printer can print
per inch, or a monitor can display, both horizontally and vertically. A 600-dpi
printer can print 360,000 (600 x 600) dots on one square inch of paper.
- Dynamic range
- It is the number of possible colors or shades of gray that can be
included in a particular image. 8-bit images can represent as many as
256 colors; 24-bit images can represent approximately 16 million colors.
- Eight-bit image
- An image where each dot may have any one of either 256
possible gray values or 256 possible color values. The value is calculated by
raising 2 to the 8th power.
- Grayscale
- The range of shades of gray in an image. The grayscale of scanners
and terminals are determined by the number of shades between black and white
that they can recognize and reproduce.
- Halftone image
- An image in which combinations of dots are used to create an
impression of grays or colors by grouping and density. For example, the eye will
see shades of gray in black dots on a white background. Where the dots are large,
dense and possibly overlapping, the eye sees dark gray or black; where the dots are
small and sparse, the eye sees light gray or white.
- Image file
- The stand-alone file of each individual page image.
- Optical Disk
- Generic term for any disk read by a laser, including all CD-ROMs.
In the NDLP production environment, the term is a shorthand for the write-once
Panasonic disks sometimes used for contractor delivery, especially the "shuttle"
disks used to transfer image files between us and the contractor.
- PCX
- For our purposes it is the file format in which we scan and store illustrated
images. The images can either be dithered or undithered.
- Pixel
- The picture elements that make up an image, similar to grains in a
photograph or dots in a halftone. Each pixel can represent a number of different
shades or colors, depending upon how much storage space is allocated for it.
- Raster
- the grid pattern applied when an image is scanned.
- Resolution (spatial)
- Images measured in terms of pixels dots per inch or other
linear measure. The higher the number of pixels, the higher the resolution.
- Resolution (tonal)
- another way of describing the numbers of bits of information
per pixel, used of grayscale or color images.
- Rework
- The delivery of contractor work that responds to a quality review memo
sent by us. It should address all the problems indicated in the review memo.
- Twenty-four bit image
- a color image where each dot may have any one of 16.7
million color values. The 24 bits are typically made up of three eight-bit values,
with each eight bit value representing a separate color (usually red, green, and
blue). Very high tonal resolution.