Cognex: Identifying Component Drift using geometric shapes & patterns

The purpose of this blog is to detail the process of setting up a program or a 'job' which will be able to identify and locate different components on an electronic development board with respect to each other. I will do so in several steps. 

Step 1: Get Connected

Open Insights Easy Builder and click the "Get Connected" button. This will display a panel similar to the one below. As you may see a camera has been set up on my network called "APP-OFFICE-COGNEX". I have this camera set up on the network through a switch with the assigned IP Address 192.168.1.21. Click connect. 

Step 2: Set Up Image

Click the Set Up Image button to open the image configuration panel. The main section of interest is the "Edit Acquisition Settings". There will be a "Live Video" button, you should click this and ensure your image is focused and the subject is central. This is important because we will be creating an Ideal Model which the job will reference in real time.

Once you are happy with the quality of your Image, the lighting, exposure and trigger interval change the trigger type to "Continuous". This will save us having to manually trigger an image acquisition and will allow us to view results on the image as they happen.

An important item to check before you move on from this step is "Focus Region", a button found at the bottom of Edit Acquisition Settings. Clicking this will have prompt a scalable selection box. Adjust this to your liking, below I have selected most of the image because the breadboard takes up most of the screen and I expect a drift variance of the board itself between inspections. 

Step 3: Finding a reference point

Typically with most vision systems you'll need a reference point. One which will tell the program where to expect other items. You can use this pattern to provide location references which will be critical to your process. To do so open the spreadsheet view by pressing Contol+Shift+V. This will open a spreadsheet with just the image and its acquisition settings. They're interesting to look at and you can change them from here but I would recommend setting up your image in the EasyBuilder because you can visually confirm if something is in focus without a transparent spreadsheet in the way. 

Press control+shift+v to open the spreadsheet view.

Find an empty cell with a few rows free above and below. Click it and type "FindPattern". A drop down will appear, just press enter or click it to select. This will open a property sheet for finding patterns. I will break down what these properties do and what to change. 

Image specifies a reference to a spreadsheet cell that contains an Image data structure; by default, this parameter references A0, the cell containing the AcquireImage Image data structure. This parameter can also reference other Image data structures, such as those returned by the Vision Tool Image functions or Coordinate Transforms Functions. Don't change this.

Fixture defines the Model and Find Regions of Interest (ROI) relative to a Fixture input or the output of a Vision Tool function's image coordinate system. Setting the ROI relative to a Fixture ensures that if the Fixture is rotated or translated, the ROI are rotated or translated in relation to the Fixture. 

The default setting is (0,0,0), the top leftmost corner of the image. 

Row

The row offset, in image coordinates.

Column

The column offset, in image coordinates.

Theta

The angle of orientation, in the image coordinate system.


Because this is our reference point we wont select anything for this option.

Model Region specifies the rectangular image region that are used to extract the model pattern.

The x-offset of the origin, in fixture coordinates.
The y-offset of the origin, in fixture coordinates.
The dimension along the region's x-axis.
The dimension along the region's y-axis.
The orientation, in fixture coordinates.


Double click on the model region text. This will minimize the spreadsheet view and open the image with a red selection box on it. Move the selection box to the item of interest and resize it accordingly. 

There is a few things I’ll point out here.  Firstly notice the polarity of the red arrows. This is how the camera processes the image. You can rotate these boxes to your liking but having them uniform in your program will shorten processing time.

Secondly notice I have selected the area to the left and below the image. Why did I do this? This item is going to be a reference point for the rest of the program. I know that it wont move during inspection. If it does its considered an automatic fail because the reference point cant be found. I’ve selected a little extra area to the left and below the item because the straight line transition from dark to light (Left of IC chip) and the 8 pinholes (below of IC Chip) are easily identifiable-location confirming features.

 Press enter. This will return you to your property sheet.

Notice how now the model region parameters have been populated by my selection.

Model settings specifies area model or edge model training.

In this section ensure the model type is "Edge model" and that the Coarseness and accuracy and both set to medium. You don't need to change anything else at the moment. 

Model Type

0 = Area model

Area model training creates patterns based on a uniform sampling of greyscale pixel values from the Model Region. The area model similarity metric is based on the normalized greyscale correlation coefficient.

1 = Edge model (default)

Edge model training creates patterns based on a sampling of points biased to the immediate vicinity of the greyscale discontinuities typically found on objects' boundaries. The edge model similarity metric is based on the normalized comparison of greyscale derivatives.

Coarseness

Specifies the size of the smallest features in the trained model

0 = Fine

Smallest features approximately 4 pixels in size.

 

1 = Medium (default)

Smallest features approximately 4 to 8 pixels in size.

2 = Coarse

Smallest features larger than 8 pixels in size

Accuracy

Specifies the size of the smallest features in the trained model

 

0 = Accurate

Specifies higher accuracy/reliability and slower execution speed

 

1 = Medium (default)

 

 

2 = Fast

Specifies lower accuracy/reliability and faster execution speed.

Offset Row

Specifies the row offset (-479 to 479; default = 0) from the model's center to the response point, as measured in the model's local coordinate system.

Offset Column

Specifies the column offset (-639 to 639; default = 0) from the model's center to the response point, as measured in the model's local coordinate system.

Force Train

Enables/disables automatic model training on spreadsheet updates. When Force Train is OFF, the model is trained only when the Model Region or Model Settings are changed. When ON, the model are re-trained every time the spreadsheet executes

Patterns

Specifies whether the FindPatterns function searches using its own model, or a model contained in another FindPatterns function. The default value is 0, which means the current FindPatterns function uses its own model. Patterns can also be referenced to a Patterns data structure, which specifies that searching uses the local model trained by another FindPatterns function. This alternative behavior allows a single model to be shared by multiple instances of PatFind searches in the spreadsheet.


At this point its worth checking where the centroid of your pattern is. This will be the point at which your pattern will be located. It will provide a starting point to be measured from later. Double click "Model Settings". A red crosshair will appear on the image. Move this to a point you desire. I have moved it to the center of the integrated circuit chip

Find region specifies the region of the image that is used to define where the search for the pattern is performed. A rectangular image region is created, that can be transformed and rotated. With this parameter selected, optionally press the Maximize Region button on the property sheet's toolbar to automatically stretch the region to cover the entire image.

Find Region

X

The x-offset of the origin, in fixture coordinates

Y

The y-offset of the origin, in fixture coordinates

High

The dimension along the region's x-axis

Wide

The dimension along the region's y-axis

Angle

The orientation, in fixture coordinates.


Double click the 'Find Region" text and readjust the selection area to cover most of the image. This is asking 'Where will I search for the pattern' put simply. 

  

Miscellaneous other you may leave these all to default but its worth considering "Angle Range" if you expect your item to be inspected at a large variety of rotations. This does affect processing speed but is a good trade off depending on what you are inspecting.

Number to Find

Specifies the maximum number of pattern matches to return (0 to 255; default = 1).

Angle Range

Specifies the function's +/- tolerance to pattern rotation (0 to 180; default =15). The function still finds patterns that are outside of the specified range, but the accuracy may be somewhat compromised. An Angle Range setting greater than 0 reduces the speed at which patterns are located.

Scale Tolerance

Specifies the function's tolerance to changes in pattern scale

0 = OFF (default)

Disables scale searching. When scale searching is disabled, the function still find patterns that are within about 10% of the original model size, but the response is degraded as the discrepancy increases.

1 = ON

Enables scale searching. When scale searching is enabled, the function performs limited scale adjustments in the range of plus and minus 10% of the original model size.

Thresh: Accept

Specifies the minimum acceptable pattern response score (0 to 100; default = 50). The function returns the Number to Find pattern responses that exceed the Thresh: Accept limit.

Thresh: Confuse

Specifies the maximum expected pattern response score (0 to 100; default = 70)for a pattern in the image that is not a true instance of the model. Setting this search parameter helps the function know which pattern responses to investigate and which can be safely ignored. This parameter is also known as a confusion threshold.

Timeout

Specifies the amount of time, in milliseconds (0 to 30000; default = 5000), that the function searches for patterns before execution is halted and #ERR is returned. Setting the value to 0 disables the setting and a timeout is not applied.

Show

Specifies the display mode for FindPatterns graphics on the image.

0 = hide all

All graphics are hidden, except when the cell containing the FindPatterns function is highlighted in the spreadsheet.

1 = result graphics only

A cross is displayed at the offset coordinate of the found model at all times.

2 = input and result graphics

Both the input image region and the cross are displayed at all times.

                                                                                           


Once you have configured these parameters press ok and a pattern cell will be filled along with accompanying information. If the row above the selected cell is empty titles for the results are populated. This is why I said to select a cell with nothing in the row or the row above. If you are doing multiple patterns its a good idea to select the cell below (row15, col A in my case) to reuse the result titles and keep your spreadsheet easy to read. 

Step 4: Locating your variable component 

Now time to model the component which is suffering from component drift. You will want to repeat step 3 and select the second object this time around. Be tighter with the selection this time around. 

Step 4: Locating Edges 

Next we want to select 2 edges of the board and find them by referencing the IC chip pattern. This both helps locate items of interest and will provide board length measurements later in the program. If the measurement between the edges ever strays out of tolerance it will indicate that the camera has moved or needs to be recalibrated. 

Type “FindLine” in an empty cell or find it in the expression dropdown tab. Press enter. A property sheet will appear. Some properties here are similar to FindPattern but I will comment on some differences here. 

This time we want to reference the IC chip location and angle to find the edge. 

To do this double click “Row” and select the first patterns Row Column and Theta values like so.

Press enter once selected. This will prompt you to make a selection with an adjustable red box.

Notice the red arrow pointing left to right. I have chosen this line because the is a very clear transition from dark to white. The polarity of this red arrow indicates which direction the camera processes the image. Keeping the similar to others in your program will aid processing times. 

Notice here Have selected black-to-white transition. The rest I left default as I was happy with the results. I will explain some of the other parameters anyway in case you would like to know more. 

Polarity

Specifies the polarity of the edge to be located, relative to the scan direction of the Region; the function will only report edges of the specified polarity.

0 = black-to-white

Specifies that the edge polarity is black-to-white.

1 = white-to-black

Specifies that the edge polarity is white-to-black.

2 = either (default)

Specifies that the edge polarity could be either black-to-white or white-to-black

Find By

Specifies the edge selection technique the function will use to select an edge, amongst multiple edge candidates.

0 = best score (default)

Specifies that the edge with the highest score receives the highest consideration.

1 = first edge

Specifies that the first edge detected receives the highest consideration.

2 = last edge

Specifies that the last edge detected receives the highest consideration.

Acceptance Threshold

Specifies the minimum acceptable edge contrast (0 to 100; default = 25). The contrast value is normalized from the greyscale histogram within the Region to a scale of 0 to 100. The function will reject any edge whose contrast score falls below this value.

Normalize Score

Specifies the pixel distance (1 to 50; default = 3) over which an edge transition takes place. Edge Width is used to filter the image before edges are extracted.

Angle Range

Specifies the function's tolerance to edge rotation (0 to 10; default = 0). The function will still find edges that are outside of the specified range, but the accuracy may not be as high.

Edge Width

Specifies the pixel distance (1 to 50; default = 3) over which an edge transition takes place. Edge Width is used to filter the image before edges are extracted.

 

Specifies the display mode for FindLine graphical overlays on top of the image.

0 = hide all (default)

All graphics will be hidden, except when the cell containing the FindLine function is the active cell in the spreadsheet.

1 = result graphics only

The line edge will be displayed at all times.

2 = input and result graphics

Both the input image region and the line edge will be displayed at all times.

3 = show all: input, result, and chart

The input image region, the line edge, and the edge response chart will be displayed at all times.

 


Note. Modifying the acceptance threshold allows you to fine tune your desired results. Represented here ************************graph***********

 

This graphic will appear in at the bottom of the spreadsheet while findline property sheet is open.

Repeat this step for the other edge. Remember to select the IC Chips location as the fixture address. This time select White-to-Black polarity because the edge is on a white-dark transition.

 

Step 4: Measurements & Calculations

Now that we have some identifiable features at known or expected points we may draft some measurements. First I want to measure the length of the board from the center point of the two edges. This will be useful to ensure the camera is position correctly in future uses of the program.

MidPointToMidPoint

Select a new cell and type “MidLineToMidline” and press enter. This will open the MidLineToMidlLine property window. As you might guess this takes the midpoints of two separate lines and measures the distance between them.

Double click X0 and select Row0 through Col1 of the first edge as the first line “Line 0”.

Do the same for the second edge for “Line1”

Click ok and the Distance measurement should look like this.

InRange

Select the cell I have highlighted to the right of the distance measurement. Type InRange() and fill in the prentices as prompted. This is how you set your tolerances.

In my case I selected cell G27 as my reference value, 450 as my minimum and 460 and my maximum tolerances. This will return a Boolean TRUE/FALSE depending on whether the value in G27 is in or outside of the set tolerances.

StatusLight

For aesthetic reasons you can then select the next cell to the right type statuslight and press enter. You will be prompted to fill in conditions on whether or not the calculation has returned a pass or a fail.

PointToPoint

This measurement will now measure the centroids of two patterns and measure the distances between them. This is the key quality attribute measurement in this job: The distance between the center of the IC and the buzzer.

For this select an empty cell and type PointToPoint and press enter.

In the PointToPoint property window you select the row and col for the respective patterns (The X&Y locations). Similar to how I have done it in the MidPointToMidPoint measurement.

Press Ok and repeat the steps for InRange and Status Light.

PlotString

The very last thing I wanted to do was show some representation on screen for whether or not the item would pass or fail inspection.

To do this You need to type “’PASS” and “’FAIL” in two separate empty cells. Don’t forget the ‘. Otherwise spreadsheet builder will think you are referencing a custom function and will return an error.

Next find another empty cell and type a short “If statement” like this “ If(H29=1,A32,A33) “ Where H29 is the Boolean result of the measurement , A32 is the cell where you have written “PASS” and A33 is the cell where you have written “FAIL”. This will change the content of the if statement cell to either PASS or FAIL depending on the value of the cell H29 (1 or 0 depending on InRange query).

Finally, find an empty cell and type PlotString. This will open the property window for plotting strings on the image screen. Select where on the screen you want it and press OK.

 

Results

Go online and observe your results. Initially you're expecting a 'PASS' because you're basing your models off of the image on screen. If not you should go back and recheck whatever parameters are failing or returning errors. 

Now move the second pattern (The buzzer in my case). I moved it a few pins to the left. Observe the "FAIL" icon.

 

Conclusion

This has been a short demonstration on how you might use pattern matching and geometric shapes to identify component drift using Cognex cameras and Insight software.  

Comments are closed