In factories, AI vision was first used for defect detection, then assembly inspection, and now SOP monitoring. Add routine tasks such as code reading and positioning, and a natural question follows: if one system can do all of these, can it cover an entire factory’s vision needs? My presentation at VisionChina Shenzhen on October 27 explored exactly that question. This article turns the presentation into a written account and adds the calculations and project evidence that time did not allow me to cover on stage.
My answer is yes—but only when three conditions are met at the same time: the visual foundation model must deliver few-shot learning and strong generalization; the Vision Agent must be general-purpose, easy to use, and fast to deploy; and the architecture must use one-to-many edge deployment to create compelling economics. Without any one of these, factory-wide vision either cannot be built or cannot be scaled. One major challenge remains unresolved: imaging. I return to it at the end.
QUESTION
What Must Factory-Wide Vision Actually See?
Vision tasks in a factory fall broadly into three groups: inspecting product defects, verifying assembly status, and monitoring whether operations follow the SOP. There are also conventional tasks such as code reading and positioning. Historically, these jobs have been split across different devices and vendors: one system for appearance inspection, another for assembly error-proofing, and yet another for SOP monitoring. Every SKU change could require collecting new samples and reconfiguring the program.
For one platform to cover all of them, the technology must answer two questions. First, is it practical to use? Can one model handle all three task types, and can a product change avoid a new sample-collection effort from scratch? Second, is it affordable? Can the economics support factory-wide rollout rather than one or two pilots? Everything that follows addresses these two questions.
PRODUCT FORMS
Two Product Forms: Edge Agent and Local Agent
Vision Agents come in two product forms. Most capabilities are the same; the difference lies in how they are integrated.
| Item | Edge Agent (LBZN-VA-I) | Local Agent (LBZN-VA-I-F) |
|---|---|---|
| Deployment | Centrally deployed beside the line; cameras connect over industrial Ethernet | Integrated inside the machine |
| Best suited for | Multiple lines, multiple stations, and mixed tasks | Vision inspection and motion control within one machine |
| Controller | Results are written to the line PLC through Modbus or similar interfaces | Built-in PLC; no external PLC required |
| Connectivity | Sized by camera resolution, sampling rate, concurrent tasks, and latency | 16 camera channels and 31 lighting channels |
Factory-wide coverage relies primarily on the edge agent, which is the focus of the sections below.
EDGE DEPLOYMENT
Why the Edge: Neither at the Endpoint nor in the Cloud
In the convolutional era, models were trained on large datasets, usually in the cloud or on dedicated training servers. A visual foundation model is pretrained and needs only a few on-site samples for adaptation, so training can be completed at the edge. Inference and training run on the same device—integrated training and inference. This is the first prerequisite for edge deployment.
The second prerequisite is networking. The edge and endpoints communicate over conventional industrial Ethernet: camera images go directly to the agent, and results return to station displays, PLCs, and alarm lights. A copper Ethernet segment can run up to 100 meters; fiber can bridge longer distances. This allows one agent to connect stations along a line and across multiple lines, regardless of overall line length.
Upstream, the edge agent connects to MES or SCADA. MES sends the recipe, identifies the current SKU or model, and the agent switches accordingly. Downstream, each station has at least one camera. Operator stations add a display that identifies an incorrect step and prompts rework. Stations that divert or reject products also add a PLC and actuator. These are the endpoint resources.
The Two Extremes of One-to-One Endpoint Deployment
Why not place compute at every endpoint? A one-to-one architecture tends toward two extremes: with one camera, most compute sits idle; with ten cameras, the device cannot keep up. GPUs remain expensive resources. Provisioning each station for its own peak demand leaves much of that capacity unused most of the time.
One-to-One at the Endpoint
- Dedicated compute at every station, sized for peak load
- Underused with few cameras; overloaded with many
- Maintenance targets increase linearly with station count
One-to-Many at the Edge
- One integrated training-and-inference device shared by all stations
- Staggered workloads; channel count sized by aggregate throughput
- Centralized recipes, models, and backups
Capacity Must Be Calculated from Throughput
How many stations can one agent support? The answer is not a slogan; it is calculated from image dimensions, inference throughput, and cycle time. Consider the tray project (see “One-to-Many—Many More” for the full derivation). Inference uses 700×700 tiles, device throughput is estimated at 200 images per second, and the on-site acquisition cycle is 20 seconds. Each 4K line-scan camera produces the equivalent of one 4096×4096 frame per cycle, split six ways in each dimension into 36 tiles.
The actual deployment stops at 16 lines, preserving 28% headroom for recapture and review, new-product trials, and future expansion. No matter how capable a system is, if it is too expensive it remains a pilot and cannot scale. Edge deployment offers the best economics: one agent powers many—many more.
GENERAL-PURPOSE
General-Purpose: One Foundation Model for Defects, Assembly, and SOPs
At the core of the Vision Agent is a visual inspection foundation model—a Transformer-based foundation model rather than a convolutional network. The first major difference is sample efficiency.
Convolutional models require large datasets—often thousands of images for one station. In a one-to-many deployment, the number of stations multiplies, and there may be ten or twenty SKUs. A rough calculation of 20 stations × 20 SKUs × 1,000 images yields about 400,000 images to collect and annotate, making deployment difficult and expensive. With a pretrained foundation model, each defect or target for a specific SKU can start with only 1–5 samples.
| Item | Small Convolutional Model | Visual Inspection Foundation Model |
|---|---|---|
| Starting samples | Thousands per station | 1–5 per target class |
| Changing SKU | Collect and train again | Add a few samples and fine-tune on site |
| Training location | Usually cloud-based or on a training server | On the edge device with integrated training and inference |
| Task scope | A dedicated model for each scenario | One foundation model shared by defect detection, assembly inspection, and SOP monitoring |
| Output | Different models for bounding-box detection, classification, and other tasks | One form only: pixel-level semantic segmentation |
It is not trained exclusively for one domain. It can detect defects, verify missing, incorrect, or reversed assembly, and monitor SOPs. The model has just one output form: pixel-level semantic segmentation, without separate branches for bounding-box detection. Segmentation is the most general representation: a defect is a region, a component is a contour, and an operator’s hands and tools are contours as well. All can be represented in the same way, then interpreted by event logic and state machines.
One distinction is important. Using a generative model to create defect images for a small convolutional model, or using a large language model as a machine interface, sits outside the inspection pipeline and serves only as assistance. The foundation model discussed here operates inside the inspection pipeline and produces the decision directly. Because it already needs very few samples, it does not rely on synthetic data augmentation.
EASE OF USE
Easy to Use: Drag-and-Drop Orchestration with Fewer, Broader Operators
On-site configuration requires no coding. Drag operators from the library onto the canvas, connect them, and assign the camera, model, and output target to each node—the workflow is complete. Compared with earlier drag-and-drop tools, this approach differs in two ways: the interface is clean, and the operators are highly abstracted and few in number.
- Camera operator: one module connects industrial cameras from many brands. We call it a “macro-operator.”
- Preprocessing and semantic segmentation: segmentation is the only model operator; defects, components, and actions are all segmented here.
- Event detection and state machine: segmentation results become events—such as an object appearing, disappearing, or remaining—and are evaluated in process order.
- External interfaces: Modbus, TCP/IP, serial, HTTP, and MQTT deliver results to PLCs, MES, or station displays.
Defect detection, assembly inspection, and SOP monitoring use the same operator chain. Only the model classes, event definitions, and state machines differ. This is why one agent can run all three task types in parallel.
RAPID DELIVERY
Rapid Delivery: Start with Only 1–5 Samples
Rapid delivery also comes primarily from the foundation model. Annotation requires only painting over the target, and each class can start with 1–5 samples.
As a result, a 3C electronics project is typically delivered in seven days and may include several SKUs. The first food-container line went live in ten days. Delivery time depends on on-site equipment and engineering scope: stations with imaging and hardware already in place are fastest, while projects requiring new optical and mechanical structures take longer.
CASE 1 · ONE-TO-SIXTEEN
Case 1: One Vision Agent for 16 Disposable-Tray Lines
Sixteen production lines, each with five 4K line-scan cameras—80 cameras in total—all connect to one edge agent. Because 4K line-scan images are large, people often assume a foundation model will be too slow and must run in the cloud. The on-site cycle time is 20 seconds. As calculated above, 80 cameras produce 2,880 tiles per cycle, consuming 72% of the device’s throughput budget.
Cost: Replacing 16 Systems with One
Per-Line Deployment · 16 Lines
- 16 inference systems and software licenses, about RMB 30,000 per line
- 16 maintenance targets, each upgraded and backed up separately
- Total acquisition cost: about RMB 480,000
Centralized Deployment · Same 16 Lines
- 1 Vision Agent including platform license, about RMB 150,000
- One platform, shared compute, and one maintenance target
- About RMB 9,000 per line
RMB 480,000 versus RMB 150,000—a reduction of about 70%. This comparison includes only inference compute and software acquisition. Cameras, lighting, frames, sorting equipment, and on-site engineering are required in both architectures and are excluded.
Accuracy: Full Manual Inspection Across 18 Shifts
| Metric | Definition | Acceptance Limit | Measured |
|---|---|---|---|
| Miss Rate | Defective items found by full manual inspection among products classified as acceptable by the machine | ≤ 1.50% | 0.39% |
| False-Reject Rate | Machine-rejected products confirmed acceptable by manual review, divided by total shift output | ≤ 5.00% | 3.28% |
| Sample Size | Item-by-item full manual inspection over 18 shifts | — | 74,916 Items |
CASE 2 · ONE-TO-TWENTY
Case 2: One Vision Agent for 20 Automotive Engine Assembly SOP Stations
SOP monitoring verifies whether operators follow the documented procedure. This automotive engine assembly line has 20 SOP stations, all handled by one Vision Agent. Each station verifies 5–15 steps individually. If a step is missed, audible and visual alarms trigger immediately and the display identifies the omission. The project went live in 20 days.
Viewed another way, 20 stations with 5–15 steps each means one device tracks the states of 100–300 process steps simultaneously. A state machine follows the required sequence and raises an immediate alarm if a step is skipped or missed.
CASE 3 · ONE-TO-FIVE
Case 3: One Agent for Five Stations in 3C Smart-Hardware Assembly
What if there are only a few stations? Vision Agents now come in three capacity tiers; even the smallest can support ten SOP stations. This 3C project has just five stations and two task types—assembly inspection and SOP monitoring. It uses the smallest tier and still has spare capacity.
| Item | Selection | Notes |
|---|---|---|
| Camera | 5 MP, global shutter | Field of view and small targets determine position and focal length |
| Lens | 12–36 mm zoom | One model covers different camera distances |
| Pixel scale | 0.06–0.23 mm/px | 1 mm spans approximately 4–17 pixels |
| Lighting | Existing line lighting | No additional close-range light |
| Compute | Centralized in a cabinet beside the line | Stations only acquire images and display results |
Pixel scale determines the smallest visible detail. At 0.06 mm/px, 1 mm spans about 17 pixels; at 0.23 mm/px, about 4 pixels. Centimeter-scale targets such as manuals, films, and foam occupy roughly 43–167 pixels per centimeter. For focal length, working distance, and field of view at each station, see Ryan’s “Multi-Station Assembly Inspection and Process Monitoring in the 3C Industry”.
To accelerate delivery, we designed a standard articulated mount. Cameras and displays use the same mount, so the installation team only adjusts position and angle on site instead of fabricating custom fixtures for every station.
What Each of the Four Station Types Checks
| Station | Inspection | Annotated Samples |
|---|---|---|
| Manual insertion | Insert one white and two yellow manuals in sequence | About 61 |
| PET film application | Align the circular hole, then press twice | About 89 |
| Bracket and foam | Assembly correctness: bracket and conductive foam both present | About 36 |
| Scan and insert into housing | Recognize the work surface and advance through ten steps | About 98 |
MORE APPLICATIONS
24 Production Applications: One Foundation, Three Task Types
Historically, one vision system could rarely handle different task types, and SOP monitoring was especially difficult. The presentation summarized 24 production applications: 9 automotive, 3 pharmaceutical, 6 home-appliance, 2 in 3C electronics, and 4 in other industries. By task, they include 4 defect-detection applications, 4 assembly-inspection applications, 15 SOP-monitoring applications, and 1 fertilizer-mixture ratio inspection. One Vision Agent can support all of these factory vision tasks.
| Industry | Application | Task | Further Reading |
|---|---|---|---|
| Automotive | Injection-molded part defect detection (foam between two panels) | Defect Detection | Visual Defect Inspection of Injection-Molded Parts: Imaging and Lighting |
| Automotive | Transmission assembly SOP monitoring | SOP Monitoring | — |
| Automotive | Fuel-tank assembly SOP monitoring | SOP Monitoring | Automotive Fuel-Tank Installation Process Monitoring |
| Automotive | Seat wiring assembly inspection | Assembly Inspection | Online Monitoring for Missing Automotive Seat Wiring Connections |
| Automotive | Body-in-white painting SOP monitoring | SOP Monitoring | Automotive Painting Process Monitoring: Body-in-White Paint Booth |
| Automotive | Component insertion SOP monitoring | SOP Monitoring | — |
| Automotive | Wiring-harness fuse-box SOP monitoring | SOP Monitoring | New-Energy Assembly Process Monitoring: Wiring-Harness Fuse-Box Station |
| Automotive | Body wax-injection SOP monitoring | SOP Monitoring | — |
| Automotive | Component assembly inspection | Assembly Inspection | — |
| Pharmaceutical | Capsule defect detection | Defect Detection | AI Visual Inspection for Capsule Appearance |
| Pharmaceutical | Cleanroom gowning SOP monitoring | SOP Monitoring | Pharmaceutical Industry: Gowning Recognition in a Grade B Cleanroom |
| Pharmaceutical | Ophthalmic-equipment accessory-box packing SOP monitoring | SOP Monitoring | — |
| Home Appliances | Motor junction-box SOP monitoring | SOP Monitoring | Component Assembly Process Monitoring: Motor Junction-Box Station |
| Home Appliances | Product-kit packing SOP monitoring (missing fuse or manual) | SOP Monitoring | — |
| Home Appliances | Product-accessory packing SOP monitoring | SOP Monitoring | — |
| Home Appliances | Kettle-packing SOP monitoring | SOP Monitoring | Missing-Item Detection at a Home-Appliance Packing Station |
| Home Appliances | Cable presence and wiring-sequence inspection for microwave and combi-steam ovens | Assembly Inspection | — |
| Home Appliances | Anti-static fabric assembly inspection behind large displays | Assembly Inspection | Home Appliances: Monitoring the Anti-Static Fabric Application Process |
| 3C | Router SOP monitoring | SOP Monitoring | — |
| 3C | Electronic-component SOP monitoring | SOP Monitoring | SOP Process Monitoring: Electronic-Component Test Station |
| Other | Nonwoven hook-and-loop fastener defect detection | Defect Detection | AI Appearance Inspection for Hook-and-Loop Fastener Patches |
| Other | Swim-cap SOP monitoring | SOP Monitoring | Process Monitoring for Swim-Cap Final Inspection |
| Other | Fertilizer blend-ratio inspection (three-color nitrogen, phosphorus, and potassium mixture) | Ratio Inspection | Flatten First, Then Measure the Ratio: Vision-Agent Sampling Inspection for Fertilizer Blends |
| Other | Infant-formula scoop defect detection (flat and contoured surfaces) | Defect Detection | Six-Side Visual Appearance Inspection for Infant-Formula Scoops |
These applications come from different projects with different imaging conditions, cycle times, and accuracy requirements. “—” indicates that no public article is currently available.
The infant-formula scoop deserves special mention. It combines flat and contoured surfaces, making it particularly difficult to capture completely with a fixed top-down camera. This leads directly to the final limitation.
ABOUT LINGBANG
Lingbang Intelligence: It Started with Data Collection
Lingbang Intelligence was founded in 2016. We began accumulating industrial images in 2017 and have collected approximately one billion to date, forming the basis for training our own foundation model.
| Year | Milestone |
|---|---|
| 2016 | Lingbang Intelligence founded |
| 2017 | Launched our first AI vision equipment and began collecting industrial images (about one billion to date) |
| 2022 | Selected by China’s Ministry of Industry and Information Technology to lead a national machine-vision R&D initiative |
| 2023 | Began training the visual inspection foundation model in January |
| 2024 | Released the visual inspection foundation model; launched VisionAgent 1.0 in October |
| 2025 | Released VisionAgent 2.0 |
| 2026 | Released VisionAgent 3.0 at this forum |
| 2027 | Plan to launch VisionAgent 4.0 at VisionChina Beijing or Shanghai |
When training began, ChatGPT was not yet widely known in China; broad discussion started around March 2023. Lingbang is a national-level “Little Giant” enterprise specializing in advanced and innovative technologies. The company holds 62 granted patents and 20 software copyrights and participated in drafting GB/T 46886—2025, General Technical Requirements for Intelligent Inspection Equipment.
LIMITATION
The Remaining Limitation Is Imaging: From Trial-and-Error Lighting to Computational Imaging
Factory-wide vision still has a major limitation: imaging for AI-based industrial inspection. Once an image is captured, model processing is already highly general—the foundation model handles the 4K imagery in the tray project both accurately and quickly. The hard part is ensuring that the image actually reveals the defect.
Industrial imaging today is still largely empirical: mount a camera above the part, move lights around it, and see whether the defect becomes visible. The problem lies in the samples. A customer initially provides some samples and the lighting is tuned to them, but unseen defects appear after deployment and the imaging setup cannot reveal them. Requiring a frozen reference sample set is difficult because customers rarely have every defect type available. This is the dilemma of traditional imaging.
The way forward is general-purpose imaging—in other words, computational imaging. The presentation used a simple example: capture four images and computationally derive two outputs, one representing shape variation and the other reflectance variation such as stains and contamination.
Its physical basis is the diffuse-reflection, or Lambertian, model: pixel intensity I = ρ·(n·l), where ρ is surface reflectance, n is the surface normal, and l is the illumination direction. Capture one image under each of several known lighting directions and every pixel yields a system of equations from which ρ and n can be solved. Three directions are theoretically sufficient; a fourth improves robustness against specular highlights and shadows. The recovered n forms the shape map, while ρ forms the reflectance map. Relief defects appear in the shape map; stains and color differences appear in the reflectance map. Imaging no longer depends on a lighting angle discovered by trial and error for one defect type—that is what “general-purpose” means.
Lingbang has not yet solved this problem well enough, so we hope to work with industry peers to close the gap. In imaging, the whole industry should move toward general-purpose methods and away from trial-and-error practice. Otherwise, too many projects will be abandoned—walk through production lines in large factories and you may find half the cameras powered off. We should stop creating failed deployments and use the foundation-model revolution to raise the ceiling of the machine-vision industry.
Visual foundation models: few-shot learning and strong generalization;
Vision Agents: general-purpose, easy to use, and fast to deploy;
Edge deployment: one-to-many—many more, with compelling economics;
Factory-wide vision: defect detection, assembly inspection, and SOP monitoring.