2001-08-21 1Industrial Vision: Image acquisitionIndustrial Vision: Image acquisition

CCD cameraCCD camera

DigitalisationDigitalisation

Data acquisition cardsData acquisition cards

Vision software Vision software

Cameras and sensors; lensesCameras and sensors; lenses

2001-08-21 2Image acquisition: CCD cameraImage acquisition: CCD camera

2001-08-21 3

CCD: Licht wordt in lading omgezet. CCD chip bevat aan elkaar geschakelde capaciteiten. Ladingen worden doorgeklokt naar buiten. ADC kan niet op de CCD chip, dus steeds een analoge uitgang.

CMOS: elk beeldelement bevat 1 fotodiode en 3 transistoren: een voor lading-spanningsconversie, twee voor adressering.

CCD camera

Image acquisition: CCD cameraImage acquisition: CCD camera

2001-08-21 4

GatesGates

Photon

SiliconSilicon

ChargesCharges

PotentialPotentialWellWell

Image acquisition: CCD sensorImage acquisition: CCD sensor

2001-08-21 5Image acquisition: digitalisationImage acquisition: digitalisation

AD

8 BitGreyvalueVolts

0...255

Greyvalue

Volts

0,7

255

0,348

127

brighter

2001-08-21 6

DigitalisationDigitalisation

camera image 8 bit grayscale digital image

Pixel mask

Pixel = Picture Element

2001-08-21 7

DigitalisationDigitalisation

Grayscale image& numeric representation

255 255 255 255 253 88 74 73 72 72 75 175 255 255 255

255 255 255 255 250 82 75 74 73 74 73 190 255 255 255

255 255 255 255 231 80 73 72 72 72 76 197 255 255 255

255 255 255 255 232 83 73 73 73 76 75 172 255 255 255

255 255 255 255 226 79 75 74 74 76 75 184 255 255 255

255 255 255 255 220 84 75 73 76 79 74 159 255 255 255

255 255 255 255 224 83 76 74 77 75 75 156 255 255 255

255 255 255 255 207 90 75 76 78 77 81 172 255 255 255

255 255 255 255 252 107 75 75 80 79 79 162 255 255 255

255 255 255 255 249 136 77 76 89 81 99 217 255 255 255

255 255 255 255 255 183 78 75 80 81 120 248 255 255 255

255 255 255 255 255 249 86 76 74 84 201 255 255 255 255

255 255 255 255 255 255 115 77 77 98 251 255 255 255 255

255 255 255 255 255 255 193 80 78 143 255 255 255 255 255

255 255 255 255 255 255 217 85 78 173 255 255 255 255 255

255 255 255 255 255 255 248 97 79 220 255 255 255 255 255

255 255 255 255 255 255 255 119 80 224 255 255 255 255 255

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255

2001-08-21 8Video sourcesVideo sources

The video source can be:The video source can be:• Video cameraVideo camera• CamcorderCamcorder• Video recorder (VCR)Video recorder (VCR)• Television broadcastsTelevision broadcasts• X-ray equipmentX-ray equipment• Scanning Electron Microscope (SEM)Scanning Electron Microscope (SEM)• CT scannerCT scanner

2001-08-21 9

Composite video = signal containing bothComposite video = signal containing both video data (luminance + colour) video data (luminance + colour) and the timing and the timing (synchronisation)(synchronisation) information. It is the standard which information. It is the standard which interconnects almost all video equipment (TVs, laserdisc, videorecorders, interconnects almost all video equipment (TVs, laserdisc, videorecorders, camcorders) at home.camcorders) at home.

It can be transported over RCA connector or SCART (=Peritel=Euro) - It can be transported over RCA connector or SCART (=Peritel=Euro) - connector.connector.

Examples of composite video standards: Examples of composite video standards: • RS-170:RS-170:

• used in North America and Japanused in North America and Japan• Monochrome signalMonochrome signal• Spatial resolution: 640 pixels x 480 linesSpatial resolution: 640 pixels x 480 lines• Frequency: 60 fields/second (equivalent to 30 frames/second)Frequency: 60 fields/second (equivalent to 30 frames/second)

• NTSC/RS-330NTSC/RS-330• used in North America and Japanused in North America and Japan• Equivalent to RS-170 but colour information is superimposed on the Equivalent to RS-170 but colour information is superimposed on the

monochrome signal.monochrome signal.• NTSC = National Television System CommitteeNTSC = National Television System Committee

Signal types for Image acquisition boardsSignal types for Image acquisition boards

2001-08-21 10

Signal types for Image acquisition boardsSignal types for Image acquisition boards

More composite video standards: More composite video standards: • CCIRCCIR

• used in Northern Europeused in Northern Europe• Monochrome signalMonochrome signal• Spatial resolution: 768 pixels x 576 linesSpatial resolution: 768 pixels x 576 lines• Frequency: 50 fields/second (equivalent to 25 frames/second)Frequency: 50 fields/second (equivalent to 25 frames/second)• CCIR = Comité Consultatif International RadioCCIR = Comité Consultatif International Radio

• PALPAL• used in Northern Europeused in Northern Europe• Equivalent to CCIR but colour information is superimposed on the Equivalent to CCIR but colour information is superimposed on the

monochrome signal.monochrome signal.• PAL = Phase Alteration LinePAL = Phase Alteration Line

• SECAMSECAM• used in France, Russia and the Sovjet Republic Statesused in France, Russia and the Sovjet Republic States• Equivalent to CCIR but colour information is superimposed on the Equivalent to CCIR but colour information is superimposed on the

monochrome signal.monochrome signal.• SECAM = Séquencial Couleur Avec MemoireSECAM = Séquencial Couleur Avec Memoire

2001-08-21 11

Signal types for Image acquisition boardsSignal types for Image acquisition boards

S-Video (also called Y/C video): luminance (Y) and chrominance (C) are separate S-Video (also called Y/C video): luminance (Y) and chrominance (C) are separate signals. The Y signal contains timing (synchronisation) information. S-video can be signals. The Y signal contains timing (synchronisation) information. S-video can be transported over 4 pin mini DIN connector, or over SCART connector.transported over 4 pin mini DIN connector, or over SCART connector.

Some image sources produce “nonstandard” video signals:Some image sources produce “nonstandard” video signals:• Video and timing information can vary in format as well as in single or multiple Video and timing information can vary in format as well as in single or multiple

signals. They do not adhere to particular spatial resolutions, signal timing signals. They do not adhere to particular spatial resolutions, signal timing schemes, signal characteristics … Consult the documentation provided with schemes, signal characteristics … Consult the documentation provided with your video source.your video source.

Progressive scan (25-30 frames/sec) cameras produce non interlaced signals.Progressive scan (25-30 frames/sec) cameras produce non interlaced signals.

All previous camera signals are All previous camera signals are analogueanalogue..

DIGITAL CAMERAS: DIGITAL CAMERAS: No frame grabber required!No frame grabber required!• Cameras with FireWire (IEEE 1394) interface. Cameras with FireWire (IEEE 1394) interface.

Supported by Apple, Windows XPSupported by Apple, Windows XP• Cameras with USB interfaceCameras with USB interface

2001-08-21 12

FugaFuga

AllegroAllegro

2001-08-21 13Image acquisition boardsImage acquisition boards

• The video capture device is often called The video capture device is often called frame grabberframe grabber card. card.• Frame grabber puts a pixel mask over the image: the card converts the Frame grabber puts a pixel mask over the image: the card converts the

analogue image (or images) supplied by a video source into a digital array analogue image (or images) supplied by a video source into a digital array (or arrays) of data points.(or arrays) of data points.

• It is a plug in card (PCI) with It is a plug in card (PCI) with AD AD convertor. The ADC must have video convertor. The ADC must have video speed: 20 MHz or higher (30 or 25 video frames per second, 300 kB [640 x speed: 20 MHz or higher (30 or 25 video frames per second, 300 kB [640 x 480 x 8 bit] per frame.480 x 8 bit] per frame.

• Other features: Other features: • input multiplexer (to select one of the 4 inputs)input multiplexer (to select one of the 4 inputs)• Colour notch filter = chrominance filter (to acquire monochrome signals Colour notch filter = chrominance filter (to acquire monochrome signals

from colour sources)from colour sources)• Programmable gain stage (to match the signal into the ADC input Programmable gain stage (to match the signal into the ADC input

range)range)• Timing and acquisition control (to synchronise grabbing with sync Timing and acquisition control (to synchronise grabbing with sync

pulses of incoming signal: PLL or Digital Clock Synchronisation)pulses of incoming signal: PLL or Digital Clock Synchronisation)• Camera control stage (to send to the camera or to receive from the Camera control stage (to send to the camera or to receive from the

camera setup and control signals, e.g. horizontal and vertical sync camera setup and control signals, e.g. horizontal and vertical sync signals, pixel clock and reset signals)signals, pixel clock and reset signals)

• Most cards provide digital I/O for input or output operations, to Most cards provide digital I/O for input or output operations, to communicate with external digital devices (e.g. industrial process). This communicate with external digital devices (e.g. industrial process). This saves a separate I/O board.saves a separate I/O board.

2001-08-21 14Block diagram of analog frame grabberBlock diagram of analog frame grabber

© Data Translation

2001-08-21 15Image acquisition boardsImage acquisition boards (continued) (continued)

• Plug-in cards (image grabber, frame grabber card) for analogue camerasPlug-in cards (image grabber, frame grabber card) for analogue cameras• Are plugged in at a VME or PCI busAre plugged in at a VME or PCI bus• Are delivered with Windows 98 or NT driversAre delivered with Windows 98 or NT drivers• Accept cameras according to the EIA (30 frames/sec) or CCIR (25) Accept cameras according to the EIA (30 frames/sec) or CCIR (25)

standardsstandards• Good cards have their own processor (DMA data transfer to PC) and Good cards have their own processor (DMA data transfer to PC) and

large RAMlarge RAM• Others (cheaper ones) use the PC processorOthers (cheaper ones) use the PC processor• They accept the signals: S video, composite video TV or VCR signals They accept the signals: S video, composite video TV or VCR signals

(NTSC/PAL/Secam)(NTSC/PAL/Secam)• Some cards have camera control outputSome cards have camera control output

2001-08-21 16Image acquisition - CamerasImage acquisition - Cameras

• Sensor types:Sensor types:• LineLine• ArrayArray

• Interface standardsInterface standards::• CCIR / RS-170 (B&W, 50-60 fields/sec.)CCIR / RS-170 (B&W, 50-60 fields/sec.)• PAL / SECAM / NTSC (Colour)PAL / SECAM / NTSC (Colour)• Progressive scan (25-30 frames/sec.)Progressive scan (25-30 frames/sec.)• FireWire (IEEE 1394)FireWire (IEEE 1394)• USBUSB

• Sensor technology:Sensor technology:• CCD (Charge Coupled Device)CCD (Charge Coupled Device)• CMOS (Complementary Metal Oxide Semiconductor). CMOS (Complementary Metal Oxide Semiconductor).

A CMOS camera produces a 1000*1000 pixel imageA CMOS camera produces a 1000*1000 pixel image

2001-08-21 17Spatial resolutionSpatial resolution

• The number of rows (N) from a video source generally corresponds The number of rows (N) from a video source generally corresponds one-to-one with lines in the video image. The number of columns, one-to-one with lines in the video image. The number of columns, however, depends on the nature of the electronics that is used to however, depends on the nature of the electronics that is used to digitize the image. Different frame grabbers for the same video camera digitize the image. Different frame grabbers for the same video camera might produce M = 384, 512, or 768 columns (pixels) per line. might produce M = 384, 512, or 768 columns (pixels) per line.

• a CCIR / PAL image source can result in max 768 x 576 pixel imagea CCIR / PAL image source can result in max 768 x 576 pixel image

• a RS-170 / NTSC source can result in max 640 x 480 pixel imagea RS-170 / NTSC source can result in max 640 x 480 pixel image

• Depending on video source or camera used, the spatial resolution can Depending on video source or camera used, the spatial resolution can range from 256 x 256 up to 4096 x 4096.range from 256 x 256 up to 4096 x 4096.

• Most applications use only the spatial resolution required. For fast Most applications use only the spatial resolution required. For fast image transfer and manipulation, often 512 x 512 is used. For more image transfer and manipulation, often 512 x 512 is used. For more accurate image processing, 1024 x 1024 is common.accurate image processing, 1024 x 1024 is common.

• The pixel aspect ratio (pixel width : pixel height) can be different from The pixel aspect ratio (pixel width : pixel height) can be different from 1:1, typical 4:3. Some frame grabbers don’t convert video data into 1:1, typical 4:3. Some frame grabbers don’t convert video data into square pixels but into rectangle ones. This creates the effect of a circle square pixels but into rectangle ones. This creates the effect of a circle appearing ovular, and squares appearing as rectangles.appearing ovular, and squares appearing as rectangles.

2001-08-21 18Spatial resolutionSpatial resolution

• Example 768 x 512 (aspect ratio 3 : 2)

Brightness resolutionBrightness resolution• Brightness resolution = bit depth resolution: number of gray levels Brightness resolution = bit depth resolution: number of gray levels

(monochrome) or number of colours(monochrome) or number of colours

• RS-170 / NTSC image: 8 bits = 256 gray levelsRS-170 / NTSC image: 8 bits = 256 gray levels

• A standard RS-170 image is 307 kB large: 640 x 480 x 8bit.A standard RS-170 image is 307 kB large: 640 x 480 x 8bit.

768 pixels

512 rows

2

3

520 rows max CCIR

2001-08-21 19Interlaced / non interlaced formatsInterlaced / non interlaced formats

• A video signal consists of a series of lines. Horizontal sync pulses separe the lines from each other.

• All composite video sources (RS-170/NTSC, CCIR/PAL) and some nonstandard video sources transmit the lines in interlaced format: first the odd (first field), afterwards the even lines (second field).

• Vertical sync pulses separate the fields from each other.

• Some nonstandard video sources transmit the lines in non-interlaced format = progressive scan. Only one field, containing all the lines, is transmitted.

• Progressive scan is recommended for fast moving images.

• If one is planning to use images that have been scanned from an interlaced video source, it is important to know if the two half-images have been appropriately "shuffled" by the digitization hardware or if that should be implemented in software. Further, the analysis of moving objects requires special care with interlaced video to avoid "zigzag" edges.

2001-08-21 20Image sensors - InterlacingImage sensors - Interlacing

Interlaced image (TV)

Field 1 Field 2

2001-08-21 21Industrial Vision softwareIndustrial Vision software

• Labview Labview

• ……

2001-08-21 22

Image enhancementImage enhancement

• Neighbourhood operationsNeighbourhood operations• Sharpening / blurringSharpening / blurring• Edge detectionEdge detection

-1 -1 -1-1 9 -1-1 -1 -1

ResultResult

1 1 11 1 11 1 1

1 0 -11 0 -11 0 -1

111000-1-1-1

ImageImage

2001-08-21 23

Image enhancementImage enhancement

• Contract enhancementContract enhancement• ThresholdingThresholding• Histogram equalisationHistogram equalisation

• See course Digital Image Processing, R. CatthoorSee course Digital Image Processing, R. Catthoor

2001-08-21 24Image acquisition: moving objectsImage acquisition: moving objects

• Resolution calculationResolution calculation• Cd resolution = cross detectionCd resolution = cross detection• Md resolution = moving detectionMd resolution = moving detection• Example cd resolution:Example cd resolution:

• Image width = 1900 mmImage width = 1900 mm• Required resolution = 0.25 mmRequired resolution = 0.25 mm• Number of pixels: 1900/0.25 = about 8000Number of pixels: 1900/0.25 = about 8000• This requires minimum 2 line scan CCD camerasThis requires minimum 2 line scan CCD cameras

• Example md resolution:Example md resolution:• Required resolution = 0.25 mm (same as cd)Required resolution = 0.25 mm (same as cd)• Object velocity = 1600 mm/secObject velocity = 1600 mm/sec• This requires 1600/0.25 = 6400 lines per second, or a camera with This requires 1600/0.25 = 6400 lines per second, or a camera with

6.4 kHz line scan frequency.6.4 kHz line scan frequency.

2001-08-21 25Image acquisition: cameras and sensorsImage acquisition: cameras and sensors

CameraCamera• Contains Contains CCD CCD or or CMOSCMOS chip. It produces an analogue signal chip. It produces an analogue signal• CCD chip contains opt. sensors with bucket memory and shift registerCCD chip contains opt. sensors with bucket memory and shift register• The shift register is controlled by a pixel clockThe shift register is controlled by a pixel clock• Synchronous reset cameraSynchronous reset camera: the pixel clock runs independently: the pixel clock runs independently• Asynchronous reset cameraAsynchronous reset camera: the shift register is set externally: a picture : the shift register is set externally: a picture

is captures immediately after an external trigger.is captures immediately after an external trigger.• A A Matrix cameraMatrix camera captures interlaced or progressive scan (=non- captures interlaced or progressive scan (=non-

interlaced) imagesinterlaced) images• Interlaced camera: sensor can be slower; reduced flickeringInterlaced camera: sensor can be slower; reduced flickering• Standards: an image has a 2x3 ratioStandards: an image has a 2x3 ratio• A A Linescan cameraLinescan camera captures one single line only captures one single line only• Sensors producing an interlaced image are not good for moving images Sensors producing an interlaced image are not good for moving images

(comb effect)(comb effect)• Now there are digital cameras: The CCD is coupled to a built in ADCNow there are digital cameras: The CCD is coupled to a built in ADC• Digital camera’s allow digital remote camera settings; no frame grabber Digital camera’s allow digital remote camera settings; no frame grabber

card is required.card is required.

2001-08-21 26 Image acquisition: cameras and sensorsImage acquisition: cameras and sensors

CCD or CMOSCCD or CMOS• CCD chip bestaat uit lichtsensoren in de vorm van een emmertjes CCD chip bestaat uit lichtsensoren in de vorm van een emmertjes

geheugen, en een shift register. Dit laatste wordt gecontroleerd door geheugen, en een shift register. Dit laatste wordt gecontroleerd door een pixel clockeen pixel clock

• Het licht wordt in lading omgezet. CCD chip bevat aan elkaar Het licht wordt in lading omgezet. CCD chip bevat aan elkaar geschakelde capaciteiten. Ladingen worden door deze capaciteiten geschakelde capaciteiten. Ladingen worden door deze capaciteiten naar buiten geklokt. Is er 1 pixel stuk, dan is er geen informatie over naar buiten geklokt. Is er 1 pixel stuk, dan is er geen informatie over de volledige lijn die erop volgt.de volledige lijn die erop volgt.

• Een ADC kan niet op de CCD chip, dus steeds een analoge uitgang.Een ADC kan niet op de CCD chip, dus steeds een analoge uitgang.

• CMOS: elk beeldelement bevat 1 fotodiode en 3 transistoren: één CMOS: elk beeldelement bevat 1 fotodiode en 3 transistoren: één voor lading-spanningsconversie, twee voor adressering. Bij CCD is voor lading-spanningsconversie, twee voor adressering. Bij CCD is de ganse cel beschikbaar voor belichting, dus 10x meer gevoelig de ganse cel beschikbaar voor belichting, dus 10x meer gevoelig dan CMOS. De donkerstroom is 10x hoger bij CMOS.dan CMOS. De donkerstroom is 10x hoger bij CMOS.

• Elke lichtgevoelige cel is individueel uitleesbaar. Is er 1 pixel stuk, Elke lichtgevoelige cel is individueel uitleesbaar. Is er 1 pixel stuk, dan bijft de rest van de lijn intact.dan bijft de rest van de lijn intact.

• Dynamisch bereik is hoger. Bij CCD is er bij overbelichting een Dynamisch bereik is hoger. Bij CCD is er bij overbelichting een “overlopen” van ladingen.“overlopen” van ladingen.

• CMOS is sneller uit te lezen.CMOS is sneller uit te lezen.

2001-08-21 27

Property CCD CMOSDynamic range Up to 45 dB Up to 140 dBPixel read out Frame/field wise Random/Area wise

Manufacturing costs High LowIntegration complexity High Low

On-chip-processing Very difficult Relatively easyPower usage High LowDistortions Blooming/smearing Fixed pattern noiseTechnology Well known Well known

Image technology Well known Experimental

Image sensors - CCD versus CMOSImage sensors - CCD versus CMOS

CCDCCD CMOSCMOS

2001-08-21 28

OpticsOptics

• Choices:Choices:• MagnificationMagnification• Depth of fieldDepth of field• Distance to objectDistance to object• Distance to image sensorDistance to image sensor• Light intensityLight intensity• Optical resolutionOptical resolution• Telecentric Telecentric

• Artefacts:Artefacts:• Added sizeAdded size• Added weightAdded weight• Image distortionImage distortion

2001-08-21 29

LensLens• Choose a lens with right focus distanceChoose a lens with right focus distance• A “telecentric objective” sees only the surface plane of a prismatic A “telecentric objective” sees only the surface plane of a prismatic

objectobject

PricePrice• B/W Camera: 500 EURB/W Camera: 500 EUR• Frame Grabber card with software: 300 EUR Frame Grabber card with software: 300 EUR

Image acquisition: cameras and sensorsImage acquisition: cameras and sensors