Similar SKU Treads were getting mislabeled before shipment. Some SKUs have very similar tread patterns but vary in width by only 1/2 inch. With the product being black rubber, imaging the product can be quite difficult, and with 100s of different SKUs, pretraining all the SKUs is not feasible.
An auto-guide interacts with the PLC to get the current SKU and opens/closes it to the appropriate width, allowing the tread to be centered over a combination of 3D laser displacement cameras. The tread is imaged as it passes over an encoder / grooved conveyor rollers. The image is converted into a 2D image and contrast-enhanced.
A Setup Wizard allowed the local QA staff to easily train a new SKU in less than 10 minutes. Vision Scripts are stored on a central repository, allowing all three finishing lines to use the same files.
Track and Trace dashboard allows QA to review inspection results and replay images in case of a future customer complaint.
Before applying cement, the smooth side of the tread surface must be inspected to ensure the quality adhesion of the tread to a tire carcass. Although a human inspector can easily find defects like blisters and cracks, small defects can be a challenge, and other surface defect anomalies may require shifting positions to find variation in surface sheen.
Our team shadowed human inspectors while collecting images of the product, annotated images, and then an image-based deep learning model was developed to detect surface anomalies. Through further data collection, annotation, and model iterations, the system gained the ability to classify the surface anomalies into several categories. Traditional vision (rule-based) algorithms were added to established pass/fail criteria based on shape, size, and brightness for each classification, thus reducing the need for 100% human inspection.
A shrink wrap product sleeve is placed onto the bottle before entering into a steam tunnel. Occasionally the shrink wrap label is not in the correct position or gets snagged. This causes an unaesthetic package that is not suitable for retail. Each bottle must be manually inspected prior to filling. Shrink wrap colors are anything across the rainbow and many shades in between, including shades of white placed on a white bottle.
The bottle is side-belt transferred to an exit conveyor, and the bottle is inspected with a GigE camera. Using a combination of UV and white LED lighting, along with a specialty cut filter, the UV lighting causes a blue shift in the color of the product sleeve, while the color of the white bottle appears to be grey. Due to the reliability with traditional color segmentation working with greys, an in-house custom color segmentation routine was used to segment the exposed bottom of the bottle that is not covered by the product sleeve. This allowed the bottle to be profiled and measured, allowing for a quantitative pass/fail decision.
When customers are finished with a laser ink cartridge, they often send it back using the return box provided with new cartridges. These cartridges are returned to centralized recycling centers across the country by the truck loads. Used cartridges are removed from their packaging and thrown onto a conveyor belt for manual sorting. As volumes steadily increase, additional workforce is required to aid in the sorting. Since the recycling center is credited for each cartridge processed and specific cartridges have a higher number of recyclable components than others, an accurate count of models is highly desired.
PC-based vision system monitors the conveyor for cartridges as they enter into the field of view. At this point, it makes note of the cartridge's encoder location and its orientation. Using a blob tool, the system pre-classifies cartridges based on their shape using a high-end multi-core processor for parallelized pattern matching. Subsequent secondary pattern matching is performed in some instances to identify other key markers when subcategorization is required. A local register retains counts of each cartridge identified. Encoder location, orientation, and cartridge ID are passed to a spider robot to pick and place the cartridges into large bins. Multiple instances and orientations of well over one hundred cartridges were trained into the system.
The inside of a tire is manually inspected for surface defects and penetration by foreign objects. It can be physically awkward and challenging to see small objects manually, so the vision system handles that task.
For this pilot project, a camera on a linear slide and rotating mirror constrained to a 12" x 3" package were mounted on the end of the linear arm and joysticked into the center of the tire. Laser gauges provide feedback to the operator for their ideal placement. Interior is laser profiled to calculate optimal camera distance such that the entire field of view is within focus. The mirror is centered to the field of view, and the tire is rotated while the camera acquires a series of images. After each series of images, the mirror is adjusted to the next field of view, and the process is repeated until the interior is fully imaged. Images are then presented to the operator on a 34" curve monitor for manual inspection. The operator reviews the images and draws boxes around possible defects. Afterward, this information is archived with an identifier. At a later date, this data can be used to train a Deep Learning system.
To prevent the injection of counterfeit products into the supply chain, the print shop manufacturer had a client request to serialize all the products that get placed onto consumer shelves, along with inner boxes, master cases, and pallets.
A custom ASP.Net application was developed to augment the manual process of packing products into inner boxes, master cases, and stacking them on pallets. The system generated barcode labels and printed them to a Zebra printer. These labels would get affixed to the product and boxes. The product was scanned with a Cognex Dataman handheld ID Reader prior to being backed into a box. Once the box was filled, the barcode box was scanned, and all the relational data was stored in a database. Similarly, the filled inner boxes would be added to the master case, scanned, and stored relational data. Master boxes were then added to a pallet, and once the pallet was full, a pallet flag containing all the master cases, along with total product and inner box counts.
QR code and human-readable text on the trading card would occasionally not match, either due to misconfiguration by the Operator or buggy remote software by the Vendor.
The system was deployed with surplus stock in less than two weeks, using 4VT's standard framework and a VisionPro Quickbuild script. Cards were fed onto a vacuum belt, triggered by a photoeye, and if there was no match, the system would blow the product off at the air gap between the vacuum belt and the accumulator belts at a rate of 10 cards per second.
The existing smart sensor did not have the tools to determine if label placement was out of spec. Quality Department requested the ability to make sure that the Lot/Date code is present.
The line was moved from another facility to a local one, and no documentation could be found. The existing controls system was reverse engineered, and the vision camera was upgraded to a Cognex InSight that mimicked the old system. Retrofit required the removal of some of the existing material handling components, which were removed, modified, and installed over a long weekend. The system was successfully brought online for the start of production.
Animal Health Sciences start-up was seeking a subject matter expert to develop a machine vision solution to determine the orientation of a baby chicken's head for the application of vaccine.
For this SBIR Phase 1 project, a custom lighting and optical solution were developed to acquire images of baby chicks as they passed under the camera. 1000 images were collected and annotated, 2/3rd of the images were then fed into Deep Learning algorithms. Detection performance was 97%, with an execution time of 19-22ms.
After the nail polish bottle has been capped, a consumer label is applied to the top of the cap that indicates the color name and product number. Space constraints prevented an inspection camera from being placed after the label applicator, so the Vision System needs to compensate for 360 orientation of the label.
With some blob morphology techniques, the orientation of the text can be determined. Afterward, an OCR tool was applied to read the text. If the text found did not match the HMI, the product was rejected.
A scheduling system dictates the components to be assembled on a diesel engine head. This information is written to an RFID tag that is attached to a carriage. The client needed a way to validate the correct engine head was loaded onto the carriage and get its information (production date, parent, and serial number) married to the RFID tag. (7) different head types will need to be handled by this system.
Multiple Cognex Dataman cameras are oriented in a work cell. As the carriage enters the work cell and stops, an RFID reader gets the part type and passes that to the HMI software, which triggers 1 or 2 cameras. The cameras extract the date, parent, and serial number for a 2D matrix dot pin barcode, and the HMI software writes the data to the RFID tag and an edge database. If the parent number is not valid for the current part type, the error is flagged for an operator to review. In the event of a poor barcode, the operator can enter the information manually.
Prior to a tray of vials being loaded in the freeze dryer, the number of vials needs to be counted and documented. Depending on the size of the vials, the number of vials on a tray can range from 100 to 225. Descrepencies between the manual count and actual count occur from time to time.
Due to the need for a continuous laminar airflow on the vials within the cleanroom, the vision system components could not be mounted directly above the vial tray. High-intensity side lights were deployed on the side of the tray. The vials would act as a light pipe allowing for sufficient contrast for the smart camera to identify and count the vials. The HMI of the software provided the Operator with an overlay for each bottle located, and a foot pedal allowed the Operator to acknowledge the count. The images were archived, and a PDF-generated report was saved to network storage at the end of the batch.
2D barcode labels are automatically applied to the tub base of a washing machine. Occasionally the label has poor print quality or is not applied correctly.
Using Cognex Dataman Barcode reader a custom script was created to handshake between the barcode printing/label applicator. After the label was applied the Dataman Reader verified that the label was readable and the data matched.
A labeling system was added to a mixed nut packaging line. A single line can process 16 different product lines at 120 - 150 jars per minute. Some products use very similar labels, with differences attributed to what country the product is being shipped to. Occasionally the wrong labels are loaded into the labeling system. If this goes unnoticed, a large number of jars need to have their labels removed, and new ones applied.
Cognex InSight cameras were mounted downstream from the labeler to view the front and backside of the jar. Multiple pattern matching zones were established to ensure no two different labels would generate a false positive and stored in a lookup table. Ethernet/IP communications was established between the PLC and the smart camera for effortless product changeovers via a PanelView HMI.
A large national conveyance company installed a conveyor and carton routing system. The Cognex Datamans (image-based barcode reader) ran well, but once the refrigeration system went online, the label reading on the cartons became unreliable.
Moisture and the cold environment caused the cartons' labels to wrinkle, and excessive packing tape by newly trained operators created hotspots on labels. An off-the-shelf cross-polarizing lighting assembly was unavailable at that time, so an existing dual lighting unit was modified with polarizing film and a polarizing filter added to the camera's lens.
A Temp labor team assembled packaging for box displays for a Trading Card game. Occasionally cards or accessories were incorrect or missing.
The product enters a multicamera light tunnel and a QA engineer uses a Wizard to draw ROIs for each card and accessory. Cognex VisionPro's PatMax was used for its robust pattern matching algorithms. The system would stop the conveyor once it exited the light tunnel if there was a mismatch or no match, and turn on a stack light.
A filler machine adds powder to a dose pouch. If the douce pouch is not properly opened, then the product is spilled onto the bottom of the machine.
Cognex InSight program was developed to profile the outside shape of the dose pouch. If the center gap distance and area were too small then the filling is skipped for that dose pouch.
Inkjet printer applies a lot and date code on round vitamin bottles. Jams or other printing issues can make the lot or date code unreadable.
Smart Camera was added to the system downstream from the Inkjet printer. System was presented with multiple instances of character from 0 - 9 and trained. PLC pushed the current Lot/Date code to the camera and the Smart Camera verified that it was a match.
2D barcode label was applied to the end of a syringe. Labels that are wrong or placement is out of specifications need to be rejected.
Cognex Smart camera was added to an existing label applicator machine. The syringe was rotated and 4 images were acquired. The 2D matrix on the barcode was verified against the SKU read in from the PLC. The label position was gauged relative to the tip of the syringe and all 4 images were used to calculate the label skew.
Pocked waffle packs are used by pick and place machines in an electronic assembly. End user desired to have a machine that could handle various gold solder pad sizes and have 1 and only 1 gold solder pad placed into a single pocket for a waffle pack.
Gold solder pads were loaded into an Asyril vibratory feeder that would spread the product out for a vision system to identify a singulated solder pad. The robot would pick the solder pad up via suction cup and present it to another camera that would identify orientation and offsets. Using the x,y, and rotation offsets the gold solder pad was placed into the waffle pack.
Seeds are collected and manually placed into a 96-well plate with gaps left open for control samples. These blocks enter into a machine that crushes the sends, which are then transferred over to another machine for genetic testing. The process of manually placing the seed in most, but not all wells of the test plate, is tedious and prone to mistakes. Also, it is important that one and only one seed is deposited in the appropriate well.
Seeds were dispensed into a small backlit bowl, A camera identified a single seed, with sufficient free space around it, so that a pick and place robot with a vacuum tip can move the seed from the bowl into one of the test wells of the 96-well plate.
A secondary camera, verified only one seed made it into the well of the plate. Certain seeds were prone to clog or gum up the surface of the vacuum tip and the secondary check verified compliance.
Motor components are assembled in stages, with the base attached to a carriage that moves through the assembly line. After the rotor is mounted in the stator the alignment needs to be checked.
Due to inconsistencies in the placement of the motor base on the carriage and the tightness of tolerances, a single top-down view and gauge approach were not viable. A stereo camera setup allowed for the end of the rotor and stator to be individually identified in 3D space. From there concentricity measurement and plane parallelism could be calculated.
Catheters are placed into a vaccuum-formed pouch and then sealed to Tyvek. A CAPA event required automated inspection of catheter packaging inspection to ensure no portion of the catheter made it past the seal.
Multiple images were rapidly acquired as the product indexed past the sealing process and the seal zones were analyzed with a Cognex Smart Camera.
Product number, manufacturing date, and index are pressed into the surface of the railroad bearing. Since the characters are on a dial, if an individual character doesn't properly roll to the next character a duplicate serial number can be generated of the character is not fully formed.
The product was rolled until a laser sensor located the first character. A Cognex InSight line scan camera imaged the full length of the string and a pattern match was performed on each character.
Due to the variance in surface finish and impression depth (as the character die wore down), multiple images over the whole production were collected and a composite image for each character was generated. The composite image of the character was used to generate the pattern classifier.
Car interiors are often customized for their material, trim, and user features. Variations may be subtle and batch runs are small. An end-of-line inspection system is needed to ensure the product that was assembled matches the build ticket.
A camera with a bright field and dark field configuration was mounted to the end of an ABB robot. The operator loads an assembled car door interior onto an open nest and presses the Inspect bottom. The robot would move to 20 different locations on the backside of the car door interior, triggering the camera/lights, and then requested the operator to rotate the fixture. The robot then moved to 6 different locations, triggering the camera/lights for each.
A mock display is presented to the Operator for pass/fail of all zones. The operator had the ability to select a zone to see the image taken and individual pass/fail results for each Poka-Yoke item.
After the bearing raceway has been machined to a sized blank, the possibility exists that the surface may have underfills, pitting, or other blemishes. Such non-conformances need to be removed and repaired before the additional value is added to the product.
The product is placed into an encoder roller system via an overhead robotic system and is imaged with a line scan camera. If variations in the surface finish are detected, the product is moved to a secondary inspection area, other wise it is moved onto the next step in the process.
A robotic system applies Loctite to various holes on an engine block. Insufficient application of Loctite can create quality issues.
An additive in the Loctite causes it to fluoresce if illuminated with UV lighting. Cognex InSight camera with a Blue bandpass filter image the Loctite application for presence and bead width and continuity.
Prior to loading the tire for shipment, the highpoint is located and the sidewall of the tire is marked with a red dot, white dot, white P, white N or a sticker. Storage and material handling of the tire can cause the highpoint mark to get rubbed off or faded.
Cognex VisionPro was used to identify and verify the presence/absence of the various types of highpoint marks based on the product SKU.
Tires of similar sizes can be produced with different tread patterns. A color band is applied on the tread surface based on the product SKU to ensure the wrong product is not mixed in with the shipment.
Cognex VisionPro was used to blend the color band together (because of the gap between tread lugs) and a color match is performed to verify it matches with the current SKU.
A fuse block assembly for a car or truck has similar frames, but what goes into them can vary. A component can easily be missed, such as a fuse or diode or the wrong relay model installed.
A GUI was developed to allow an Engineer to train a new component (color, pattern matching, OCR) and add it to a component family. Once populated with components, an Engineer can create a product SKU and create a Region of Interests (ROIs) for each component location on the fuse block. The engineer can choose the allowed orientation of the component for the ROI.
With a fully configured product, an Operator can recall the SKU from a part list and batch inspect multiple fuse blocks. Any component that is missing, in the wrong orientation, pattern score is too low, or OCR is mismatched, then the ROI is highlighted in red. Components that pass are highlighted in green.
Key blanks for major retailers are plated, stamped and channels cut. Non-conformances can include poor plating, poor stamping, misaligned stamping, and misaligned channels.
The system was trained with the average composite image of 10 or more keys. Cognex PatInspect and additional image processing were used to determine the difference image, then blob analysis determined pass/fail result.
Opiate Lozenges are sonic welded onto a plastic stick, inspected, and transferred to a packaging tray. An embossed number on the lozenge indicates the dosage of the product. The wrong or unreadable embossed number is considered a non-conformance. A chipped or cracked lozenge is also considered a non-conformance.
Cognex VisionPro was used to pattern match the embossed number; too low scores were rejected. Image processing tools masked out the embossed number, and edge detection algorithms were used to identify cracks/chips.
The product is placed by a robot into a thermoformed package. If the product is not properly placed into the individual pockets it can create a problem with the top foil seal.
Cognex InSight and pattern matching were used to verify the product was seated correctly in the individual pockets. The product line was stopped for non-conformance.
The lot and date code are printed on a label, and the label is applied to the bottom of the stick holding the opiate lozenge. Missing label, missing print, or incorrect lot/date code is considered a non-conformance.
Cognex VisionPro was used for verifying label placement and OCV of lot and date codes.
After the products have been loaded into a vacuum-formed tray, a top foil is applied. If the wrong foil has been loaded into the machine, and if gone unnoticed for a while, a whole batch of products would need to be destroyed. Sealed and unsealed products can not be repackaged.
A Cognex Smart camera was used to pattern match key features of the print on the top foil. Mismatches will signal to the PLC to stop the line.
A large print shop was contracted for coupons. The coupons are serialized and encoded with a 1D barcode to prevent fraud.
Cognex Dataman with an HMI was used to ensure readability. The operator entered the start and end sequence of the serialization. Dataman would send a reject/stop signal to the PLC if the barcode was not readable or out of sequence.
RSS stacked codes are printed on the side of a pharmaceutical package. Readability is a must.
.Cognex Dataman was used to read the RSS stacked code, and when the quality grade of the printed barcode fell below a threshold grade, an alarm was signaled.
Ice containers getting filled with the wrong product, if uncaught until later can create a costly recall.
Prior to filling multiple Cognex barcode readers oriented around the ice container looked for a 2D barcode and verified that it matched the SKU provided by the PLC
Ice containers getting filled with the wrong chocolate-covered ice cream nuggets, if uncaught until later, can create a costly recall.
Two Cognex InSight cameras were set up on opposite sides of the oval container. Pattern matching was done on the picture label to verify the match based on the SKU. On the ingredient label, the UPC code was verified against the SKU.
Brazing paste is dispensed onto a sprocket gear before entering the oven. Too much, too little, or not in the correct position will cause a poor weld.
A Cognex InSight Camera was used to determine the orientation of the sprocket with pattern matching and provide offset information to the dispensing robot. The robot would dispense two blobs of paste between the teeth of the sprocket on opposite sides. After the application, the smart camera applied blob analysis to verify the amount of brazing paste dispense was within tolerance.
From new to end of life, welding tips used to spot weld the housing to the band create varied results. From time to time, a weld would fail to fully engage and would produce a light or no weld.
5000 images were collected and annotated, then applied to a neural network-based descision engine (predecessor to modern deep learning algrothims). System performed well after new tips were broken in after 2000 cycles. (Average life cylce of tips 35k - 40k)
Tablet press machines over time require routine maintenance, and scheduling downtime can be a challenge. Having chipping or caping the tablet is considered a non-conformance and should be separated.
Cognex InSight camera located and fixtured image processing tools on the tablet. Blob detection tools were then used to locate edges in areas that should be quiet. Results were signaled to the PLC.
On pills/tablets with a surface coating, contract pharmaceutical manufacturers will print a brand/ID instead of making an imprint. Missing text or faded print is considered a non-conformance.
Cognex InSight smart camera located the print, and pattern matching scores were determined for individual characters. Pattern match scores below a specific threshold were rejected.
International mail brought in through airline processing centers are often in large sacks with a 4”x6” card attached to it. Depending on the country of origin, some of these cards are hand written and others are printed. The individual at the processing window needs to manually enter in all of the information on the mail tag into a data entry system before it can be thrown onto a conveyor belt to be processed. This creates a bottle neck in the work flow, since only so many transfer windows can be manned and managed to allow baggage/cargo trucks to be unloaded.
A portable prototype imager with a flat surface allowed the mail handlers to position the mail tag for image acquistion. At the start of a new load the mail handler would enter in the Airline and Flight number, along with the number of pieces. The mail handler would take an image of that tag and then the local Vision PC would attempt to read all of the text on the tag, then present the image along with data to the mail handler. Once approved, the data would be saved to a database. If tag was not readable, the image was sent to a remote location where it was presented to a person for manual entry. This allowed for reconcilation of a load and moving mail bags as quickly as the manual imaging process would allow.
Supplier for the machines that manufacture the Permanent Residency Cards needed to ensure that the text printed on the card matches what is contained in the database. It was also to ensure the person’s face was printed clearly, and verify the presence of several security features.
Standard lighting techniques and Optical Character Recognition routines where used to validate the text and data that was contained in the database. Image correlation was used to validate the person’s face printed on the cards. Multiple different lighting techniques and LED wavelengths were used with different image processing alghrothims to verify the presence of all security features.
Supplier for the machines that manufacture the Driver’s Licenses needed to ensure the text printed on the card matched what is contained in the database. Also needed to ensure the person’s face was printed clearly, and verify presence of the engraved birth date on the card.
Standard lighting techniques and Optical Character Recognition routines where used to validate the text and data that was contained in the database. Image correlation was used to validate the person’s face printed on the cards. Photometric stereo imaging technique with IR LED lights was used to extract the engraving and filter out the print on the driver’s license.
Dashboard displays manufactured by automotive suppliers are hand assembled with different Legend Tags (door open, oil temp, washer fluid low, etc.). Before the product is packaged and shipped for assembly into a vehicle, 100% inspection is required to prevent quality chargebacks.
PC-based vision system reads the 2D barcode on the cluster assembly and queries a database. The database provides information on what Legend tags are to be installed in each of the 10 locations. Using pre-trained patterns, the system verifies all 10 locations on the cluster and prompts the Operator of any mismatches.
Dashboard displays manufactured by automotive suppliers must have all the gauges (speedometer, tachometer, engine temp, etc.) calibrated before shipping.
The operator places the product into a test docking station, and the gauge is provided a signal for 3 different values. The PC-based vision system determines the angle of the needle on the gauge and provides the value back to the test docking station. The 3 data points are used to calibrate the gauge, and values are saved to the EPROM of the dashboard display.
Vinyl floor manufacturer was looking to increase the consistency of their product by detecting foreign color chips, shade breaks, and "eye poppers." Vinyl flooring is made by grinding up various color chips, heating them, and rolling them flat. Since broken tiles are often ground and put back in the mix, any tiles left over from the previous batch run could introduce undesirable variances.
A PC-based vision system segmented the image for the tile. During the training, the process scanned and identified all the grouped colored signatures, including the background average color—tiles with background shade breaks and non-conformances were rejected. After multiple consecutive rejects, an operator was alerted.
An Automotive manufacturer needed to verify the correct roof has been placed on the vehicle prior to robotic welding.
PC-based Vision system identified key features of 6 different possible roof configurations and verified a match based on the VIN barcode.
An Automotive manufacturer needed to verify the correct fender has been placed on the vehicle prior to robotic welding.
PC-based Vision system identified key features of 4 different possible fender configurations and verified a match based on the VIN barcode.
Tier 1 automotive supplier manufactures car frames. Mounts that are bent or incorrectly mounted on the frame need to be detected before additional value is added to the product.
PC-Based vision system located and profiled the mount that was welded to the frame. An alert was raised if the location or deformation was outside of tolerance by more than 1/4 inch.
Molded fenders made from thermoplastic olefins (TPO) require an adhesion promoter (Adpro) prior to the application of automotive paint.
A PC-Based vision system projected a laser line onto the TPO fender. A raw fender would create a diffraction pattern when reflecting the laser line, whereas the application of Adpro creates a diffuse reflection. Image processing analysis detected the signature difference between the two, and the system would stop the line if a non-conformance were detected.
Fenders are welded to a truck bed by welding robots. Misaligned fenders need to be detected and corrected before the truck bed proceeds to the next step in the process.
The PC-Based vision system used structured light to detect and measure fender location on the truck bed. Rejects were flagged, and an alert was raised to the Operator.
Metal truck panels after forming enter into a rust inhibitor dip tank. Panels that are not properly secured can "float" off the carriage into the dip tank. Subsequence carriage entering into the dip tank can have their panels damaged.
A PC-based vision system was used to verify all the locking pins holding panels to the carriage were in the correct position. If an improper position locking pin was detected, the line was stopped, and an alarm was raised.