Digital Image Processing

What Is Digital Image Processing?

An image may be defined as a two-dimensional function, f(x, y), where x and y are spatial (plane) coordinates, and the amplitude of at any pair of coordinates (x, y) is called the intensityor gray level of the image at that point. When x, y, and the amplitude values of f are all finite,discrete quantities, we call the image a digital image. The field of digital image processing refersto processing digital images by means of a digital computer. Note that a digital image iscomposed of a finite number of elements, each of which has a particular location and value.These elements are referred to as picture elements, image elements, pels, and pixels. Pixel is theterm most widely used to denote the elements of a digital image. Vision is the most advanced of our senses, so it is not surprising that images playthe single most important role in human perception. However, unlike humans, who are limited to the visual band of the electromagnetic (EM) spectrum, imaging machines cover almost the entireEM spectrum, ranging from gamma to radio waves. They can operate on images generated bysources that humans are not accustomed to associating with images. These include ultra-sound,electron microscopy, and computer-generated images. Thus, digital image processingencompasses a wide and varied field of applications. There is no general agreement amongauthors regarding where image processing stops and other related areas, such as image analysisand computer vision, start. Sometimes a distinction is made by defining image processing as adiscipline in which both the input and output of a process are images. We believe this to be alimiting and somewhat artificial boundary. For example, under this definition, even the trivialtask of computing the average intensity of an image (which yields a single number) would not beconsidered an image processing operation. On the other hand, there are fields such as computervision whose ultimate goal is to use computers to emulate human vision, including learning andbeing able to make inferences and take actions based on visual inputs. This area itself is a branchof artificial intelligence (AI) whose objective is to emulate human intelligence. The field of AI isin its earliest stages of infancy in terms of development, with progress having been much slowerthan originally anticipated. The area of image analysis (also called image understanding) is inbetween image processing and computer vision.

There are no clear-cut boundaries in the continuum from image processing at one end to computer vision at the other. However, one useful paradigm is to consider three types of computerized processes in this continuum: low-, mid-, and high-level processes. Lowlevelprocesses involve primitive operations such as image preprocessing to reduce noise,contrast enhancement, and image sharpening. A low-level process is characterized by the factthat both its inputs and outputs are images. Mid-level processing on images involves tasks suchas segmentation (partitioning an image into regions or objects), description of those objects toreduce them to a form suitable for computer processing, and classification (recognition) ofindividual objects. A mid-level process is characterized by the fact that its inputs generally are images, but its outputs are attributes extracted from those images (e.g., edges, contours, and theidentity of individual objects). Finally, higher-level processing involves “making sense” of anensemble of recognized objects, as in image analysis, and, at the far end of the continuum,performing the cognitive functions normally associated with vision and, in addition,encompasses processes that extract attributes from images, up to and including the recognition ofindividual objects. As a simple illustration to clarify these concepts, consider the area ofautomated analysis of text. The processes of acquiring an image of the area containing the text,preprocessing that image, extracting (segmenting) the individual characters, describing thecharacters in a form suitable for computer processing, and recognizing those individualcharacters are in the scope of what we call digital image processing.

Fundamental Steps in Digital Image Processing

Image acquisition is the first process shown in Fig. Note that acquisition could be as simple asbeing given an image that is already in digital form. Generally, the image acquisition stage involvespreprocessing, such as scaling.Image enhancement is among the simplest and most appealing areas of digital image processing.

Basically, the idea behind enhancement techniques is to bring out detail that is obscured, or simply to highlight certain features of interest in an image. A familiar example of enhancement iswhen we increase the contrast of an image because “it looks better.” It is important to keep inmind that enhancement is a very subjective area of image processing. Image restoration is an area that also deals with improving the appearance of an image. However, unlike enhancement, which is subjective, image restoration is objective, in the sensethat restoration techniques tend to be based on mathematical or probabilistic models of imagedegradation. Enhancement, on the other hand, is based on human subjective preferencesregarding what constitutes a “good” enhancement result.

Color image processing is an area that has been gaining in importance because of the significantincrease in the use of digital images over the Internet. Wavelets are the foundation for representing images in various degrees of resolution.

Compression, as the name implies, deals with techniques for reducing the storage required to save an image, or the bandwidth required to transmit it. Although storage technology has improved significantly over the past decade, the same cannot be said for transmission capacity.

This is true particularly in uses of the Internet, which are characterized by significant pictorial content. Image compression is familiar (perhaps inadvertently) to most users of computers in theform of image file extensions, such as the jpg file extension used in the JPEG (JointPhotographic Experts Group) image compression standard.Morphological processing deals with tools for extracting image components that are useful in therepresentation and description of shape.

Segmentation procedures partition an image into its constituent parts or objects. In general, autonomous segmentation is one of the most difficult tasks in digital image processing. A ruggedsegmentation procedure brings the process a long way toward successful solution of imagingproblems that require objects to be identified individually. On the other hand, weak or erraticsegmentation algorithms almost always guarantee eventual failure. In general, the more accuratethe segmentation, the more likely recognition is to succeed. Representation and description almost always follow the output of a segmentation stage, whichusually is raw pixel data, constituting either the boundary of a region (i.e., the set of pixelsseparating one image region from another) or all the points in the region itself. In either case,converting the data to a form suitable for computer processing is necessary. The first decisionthat must be made is whether the data should be represented as a boundary or as a completeregion.

Boundary representation is appropriate when the focus is on external shapecharacteristics, such as corners and inflections. Regional representation is appropriate when thefocus is on internal properties, such as texture or skeletal shape. In some applications, these representations complement each other. Choosing a representation is only part of the solution fortransforming raw data into a form suitable for subsequent computer processing. A method mustalso be specified for describing the data so that features of interest are highlighted. Description,also called feature selection, deals with extracting attributes that result in some quantitativeinformation of interest or are basic for differentiating one class of objects from another.Recognition is the process that assigns a label (e.g., “vehicle”) to an object based on itsdescriptors. We conclude our coverage of digital image processing with the development ofmethods for recognition of individual objects.

Components of an Image Processing System

As recently as the mid-1980s, numerous models of image processing systems being sold throughout the world were rather substantial peripheral devices that attached to equally substantial host computers. Late in the 1980s and early in the 1990s, the market shifted to imageprocessing hardware in the form of single boards designed to be compatible with industrystandard buses and to fit into engineering workstation cabinets and personal computers. Inaddition to lowering costs, this market shift also served as a catalyst for a significant number ofnew companies whose specialty is the development of software written specifically for imageprocessing.

Although large-scale image processing systems still are being sold for massive imaging applications, such as processing of satellite images, the trend continues toward miniaturizing and blending of general-purpose small computers with specialized image processing hardware. Figure shows the basic components comprising a typical general- purposesystem used for digital image processing. The function of each component is discussed in thefollowing paragraphs, starting with image sensing.

With reference to sensing, two elements are required to acquire digital images. The first is a physical device that is sensitive to the energy radiated by the object we wish to image. The second, called a digitizer, is a device for converting the output of the physical sensing device into digital form. For instance, in a digital video camera, the sensors produce an electrical outputproportional to light intensity. The digitizer converts these outputs to digital data. Specialized image processing hardware usually consists of the digitizer just mentioned, plus hardware that performs other primitive operations, such as an arithmetic logic unit (ALU), whichperforms arithmetic and logical operations in parallel on entire images. One example of how anALU is used is in averaging images as quickly as they are digitized, for the purpose of noisereduction. This type of hardware sometimes is called a front-end subsystem, and its mostdistinguishing characteristic is speed. In other words, this unit performs functions that requirefast data throughputs (e.g., digitizing and averaging video images at 30 framess) that the typicalmain computer cannot handle The computer in an image processing system is a general-purpose computer and can range froma PC to a supercomputer. In dedicated applications, some times specially designed computers areused to achieve a required level of performance, but our interest here is on general-purpose image processing systems. In these systems, almost any well-equipped PC-type machine issuitable for offline image processing tasks.

Software for image processing consists of specialized modules that perform specific tasks. A well-designed package also includes the capability for the user to write code that, as a minimum,utilizes the specialized modules. More sophisticated software packages allow the integration ofthose modules and general-purpose software commands from at least one computer language.

Mass storage capability is a must in image processing applications. An image of size 1024*1024pixels, in which the intensity of each pixel is an 8-bit quantity, requires one megabyte of storagespace if the image is not compressed. When dealing with thousands, or even millions, of images,providing adequate storage in an image processing system can be a challenge. Digital storage forimage processing applications falls into three principal categories: (1) short-term storage for useduring processing, (2) on-line storage for relatively fast re-call, and (3) archival storage,characterized by infrequent access. Storage is measured in bytes (eight bits), Kbytes (onethousand bytes), Mbytes (one million bytes), Gbytes (meaning giga, or one billion, bytes), andTbytes (meaning tera, or one trillion, bytes). One method of providing short-term storage iscomputer memory. Another is by specialized boards, called frame buffers, that store one or moreimages and can be accessed rapidly, usually at video rates (e.g., at 30 complete images persecond).The latter method allows virtually instantaneous image zoom, as well as scroll (verticalshifts) and pan (horizontal shifts). Frame buffers usually are housed in the specialized imageprocessing hardware unit shown in Fig.3.Online storage generally takes the form of magneticdisks or optical-media storage. The key factor characterizing on-line storage is frequent access tothe stored data.

Finally, archival storage is characterized by massive storage requirements butinfrequent need for access. Magnetic tapes and optical disks housed in “jukeboxes” are the usualmedia for archival applications.

Image displays in use today are mainly color (preferably flat screen) TV monitors. Monitors aredriven by the outputs of image and graphics display cards that are an integral part of the computer system. Seldom are there requirements for image display applications that cannot bemet by display cards available commercially as part of the computer system. In some cases, it isnecessary to have stereo displays, and these are implemented in the form of headgear containingtwo small displays embedded in goggles worn by the user.

Hardcopy devices for recording images include laser printers, film cameras, heat-sensitive devices, inkjet units, and digital units, such as optical and CD-ROM disks. Film provides the highest possible resolution, but paper is the obvious medium of choice for written material. Forpresentations, images are displayed on film transparencies or in a digital medium if imageprojection equipment is used. The latter approach is gaining acceptance as the standard for imagepresentations. Networking is almost a default function in any computer system in use today. Because of thelarge amount of data inherent in image processing applications, the key consideration in imagetransmission is bandwidth. In dedicated networks, this typically is not a problem, butcommunications with remote sites via the Internet are not always as efficient. Fortunately, thissituation is improving quickly as a result of optical fibre and other broadband technologies.

Image representation and its properties

We will use two principal ways to represent digital images. Assume that an image f(x, y) is sampled so that the resulting digital image has M rows and N columns. The values of the coordinates (x, y) now become discrete quantities. For notational clarity and convenience, we shall use integer values for these discrete coordinates. Thus, the values of the coordinates at theorigin are (x, y) = (0, 0). The next coordinate values along the first row of the image are represented as (x, y) = (0, 1). It is important to keep in mind that the notation (0, 1) is used to signify the second sample along the first row. It does not mean that these are the actual values ofphysical coordinates when the image was sampled. Figure shows the coordinate conventionused.

The notation introduced in the preceding paragraph allows us to write the complete M*N digitalimage in the following compact matrix form:

The right side of this equation is by definition a digital image. Each element of this matrix arrayis called an image element, picture element, pixel, or pel.

Geometric Transform

Geometric image transformation functions use mathematical transformations to crop, pad, scale, rotate, transpose or otherwise alter an image array to produce a modified view of an image. The transformations described in this chapter are linear transformations. For a description of non-linear geometric transformations, When an image undergoes a geometric transformation, some or all of the pixels within the source image are relocated from their original spatial coordinates to a new position in the output image. When a relocated pixel does not map directly onto the centre of a pixel location, but falls somewhere in between the centres of pixel locations, the pixel's value is computed by sampling the values of the neighbouring pixels. This resampling, also known as interpolation, affects the quality of the output image.

Cropping Images:

Cropping an image extracts a rectangular region of interest from the original image. This focuses the viewer's attention on a specific portion of the image and discards areas of the image that contain less useful information. Using image cropping in conjunction with image magnification allows you to zoom in on a specific portion of the image. This section describes how to exactly define the portion of the image you wish to extract to create a cropped image

Padding Images:

Image padding introduces new pixels around the edges of an image. The border provides space for annotations or acts as a boundary when using advanced filtering techniques. This exercise adds a 10-pixel border to left, right and bottom of the image and a 30-pixel border at the top allowing space for annotation. The diagonal lines in the following image represent the area that will be added to the original image. For an example of padding an image, complete the following steps.

Rotating Images:

The rotation operator performs a geometric transform which maps the position of a picture element in an input image onto a position in an output image by rotating it through a user-specified angle about an origin . In most implementations, output locations which are outside the boundary of the image are ignored. Rotation is most commonly used to improve the visual appearance of an image, although it can be useful as a pre-processor in applications where directional operators are involved.

Reflecting images:

The reflection operator geometrically transforms an image such that image elements, i.e. pixel values, located at position in an original image are reflected about a user-specified image axis or image point into a new position in a corresponding output image. Reflection is mainly used as an aid to image visualization, but may be used as a pre-processing operator in much the same way as rotation.

Local Pre-processing

Pre-processing is the name used for operations on images at the lowest level of abstractionbothinput and output are intensity images. These iconic images are usually of the same kind as the original data captured by the sensor, with an intensity image usually represented by a matrix or matrices; of image function values (brightness’s).

Pre-processing does not increase image information content. If information is measured using entropy , then pre-processing typically decreases image information content. From the information theoretic viewpoint it.can thus be concluded that the best pre-processing is no pre-processing, and without question, the best way to avoid (elaborate) mpre-processing is to concentrate on high-quality image acquisition. Nevertheless, pre- processingis very useful in a variety of situations since it helps to suppress information that is not relevant to the specific image processing or analysis task. Therefore, the aim of pre-processing is an improvement of the image data that suppresses undesired distortions or enhances some image features important for further processing, although geometric transformations of images (e.g., rotation, 5Caling, translation) are also classified as pre-processing methods here since similar techniques are used.

Image pre-processing methods are c1assified here into four categories according to the size of the pixel neighbourhood that is used for the calculation of a new pixel brightness. covering pixel brightness transformations (local pre-processing ill our sense), andimage restorationA considerable redundancy of information in 1ll0000t images allows image pre-processingmethods to explore image data itself to learn image characteristics in a statistical senseThese characteristics arc used either to suppress unintended degradation, such as noiseor to enhance the image. Neighbouring pixels corresponding to given object in real images have essentially the same or similar brightness value, so if a distorted pixel can be pickedout from the image, it can usually be restored as an average value of neighbouring pixels.

We shall consider methods that use a small neighbourhood of a pixel in an input image to produce a new brightness value in the output image. Such pre-processing operations are called also filtration (or filtering) if signal processing terminology is used.

Local pre-processing methods can be divided into two groups according to the goal of the processing. First, smoothing aims to suppress noise or other small fluctuations in the image; it is equivalent to the suppression of high frequencies in the Fourier transform domain. Unfortunately, smoothing also blurs all sharp edges that bear important information about the image. Second, gradient operators are based on local derivatives of the image function. Derivatives are bigger at locations of the image where the image function undergoes rapid changes, and the aim of gradient operators is to indicate such locations in the image. Gradient operators have a similar effect to suppressing low frequencies in the Fourier transform domain. Noise is often high frequency in nature; unfortunately, if a gradient operator is applied to an image, the noise level increases simultaneously.

Image restoration:

Pre-processing methods that aim to suppress degradation using knowledge about its nature arc called image restoration. Most image restoration methods are based on convolution applied globally to the whole image.

Degradation of images can have many causes: defects of optical lenses, non-linearity of the electro-optical sensor, graininess of the film material, relative motion between an object and camera, wrong focus, atmospheric turbulence in remote sensing or astronomy, scanning of photographs, etc. . The objective of image restoration is to reconstruct the original image from its degraded version.

Image restoration techniques can be classified into two groups: deterministic and stochastic. Deterministic methods are applicable to images with little noise and a known degradation function. The original image is obtained from the degraded one by a transformationinverse to the degradation. Stochastic techniques try to find the best restorationaccording to a particular stochastic criterion, e.g. , a least-squares method. In some casesthe degradation transformation must be estimated first.

It is advantageous to know the degradation function explicitly. The better this knowledge is, the better are the results of the restoration. There are three typical degradations with a simple function: relative constant speed movement of the object with respect to the camera, wrong lens focus, and atmospheric turbulence.

In most practical cases, there is insufficient knowledge about the degradation, and it must be estimated and modeled. The estimation can be classified into two groups according to the information available: a priori and a posteriori. If degradation type and/or parameters need to be estimated, this step is the most crucial one, being responsible for image restoration success or failure. It is also the most difficult part of image restoration. A priori knowledge about degradation is either known in advance or can be obtained before restoration. For example, if it is known in advance that the image was degraded by relative motion of an object with respect to the sensor, then the modeling determines only the speed and direction of the motion. An example of the second case is an attempt to estimate parameters of a capturing device such as a TV camera or digitizer, whose degradation remains unchanged over a period of time and can be modeled by studying a known sample image and its degraded version.

A posteriori knowledge is that obtained by analysing the degraded image. A typical example is to find some interest points in the image (e.g., corners, straight lines) and guess how they looked before degradation. Another possibility is to use spectral characteristics of the regions in the image that are relatively homogeneous.