Detection of PCB Soldering Defects using Template Based Image Processing Method

: In this study, a predefined template-based image processing system is proposed to automatically detect of PCB soldering defects that negatively affect circuit operation. The proposed system consists of a scaled inspection structure, a camera, an image processing algorithm merged with Fuzzy and template guided inspection process. The prototype is produced using a plastic material, depending on the focal length of the camera and the PCB size. Image processing step comprises two steps. Firstly, solder joints are determined and boxed using Fuzzy C-means clustering algorithm. Then, the center of each joint is determined. In the next step, a joint template is created that contains solder joints information. This joint template contains information about the joints that includes possible touching odds to other joints. Template accelerates the algorithm diverting to closest joint that may include defect. Finally, each joint is only inspected regarding template guide that based on neighbor joints. Proposed method includes a scaled inspection structure related to focal length of camera. During the every query, PCB must be located same coordinates via mechanical guiding on the structure to obtain same picture. Thus, taken picture could be same every trying. The proposed method is executed 85 times on same sample PCB in case of any fake output error. In order to obtain commercial success, mechanical structure was improved and for inspected PCB success was obtained 100%.


Introduction
PCB is a skeleton of a circuit to carry current, hold the components and connects to components to each other's. For this reason, it is very important in terms of continuity of technological development and operation of electronic circuits. With the development of technology, demands for electronic products to contain more features and to be produced in smaller sizes have emerged [1]. This demand causes the PCB area to be smaller or contain more electronic material in the same area. The increase in the number of electronic materials leads to a shortening of the distance between the materials and thus to soldering problems. Soldering problems often lead to major problems such as failure of the circuit or damage to circuit elements [2]. Detection of soldering mistakes reduces the cost by preventing damage to the circuit, and increases production quality and efficiency. Examination of soldering defects manually made by inspectors, various problems have arisen. These faults, which are related to the human factor, usually lead to slower production, lower quality standards and increased cost [3]. In order to overcome these problems, computer vision systems are used for PCB soldering defect inspection [4,5]. The main components of computer vision systems consist of an external component, a camera and an image processing algorithm [6]. Computer vision systems come to the forefront with features such as rapid inspection, objective decision making, working without fatigue, working under difficult conditions [7]. Image processing is the fundamental part of computer vision. So, it is widely used in literature and reel world applications to find an effective solution for sophisticate problems [8,9]. In the PCB soldering defect detection process, soldering joints must be determined via image processing algorithms. For this operation, object detection is applied first. Then, object recognition is performed to eliminate joint-like misleading objects so that joint regions can be detected correctly. The goal of this study is to find short circuit problems caused by PCB soldering process. During the first testing of electronics device could be damage in case of solder bridges. Thus, especially soldering joints which are connects each other reluctantly cause boards damage. For this reason, inspection before the power up has vital importance. In order to avoid man human-induced faults a computer vision system is proposed, which detects problems that may occur if two or more joints touch each other due to soldering. According to designed PCB some joints are already short-circuited to each other via circuit tracks or soldering area and these joints due to soldering mistake is not a problem for the operate of circuit. The proposed computer vision algorithm does not perceive these soldering defects as error. But, if joints that are not interconnected by circuit paths are shorted to each other due to soldering problems, this creates a problem for the circuit and this problem is perceived by the proposed algorithm as a soldering defect. As the first step of the soldering defect analysis, the fuzzy c-means algorithm [10] is used to determine the joints in the image. Then, the contact between the joints is inspected according to the previously defined joint template according to regarding template guiding. The organization of the rest of this paper is as follow; Section 2 contains the description of the problem and details of the proposed computer vision algorithm. In the Section 3, experiments and experimental results are discussed. In Section 4, conclusion is summarized. Short circuiting causes problems for electronic circuits. A number of tests are usually done for this. But these tests are costly and take a long time. One of the fastest methods for short circuit inspection is visual inspection. However, it can be quite misleading because of the PCB complexity. Because soldering joints can be in contact with each other using circuit paths that are difficult to follow. In such a case, a short circuit caused by soldering will not cause a problem and reworking this circuit will cause both money and time loss. The two types of problems that this study suggests are; the first is the determination of the joints that do not have a problem with each other, and the second is the determination of the pixels that will become a problem if they are in contact with each other. These two type joints are shown in Figure 1. To overcome the soldering problem, a template based computer vision technique with joint contact information is proposed. For this, a prototype is produced in accordance with the camera field of view and the PCB dimensions. The images obtained with the help of the camera on the prototype are transferred to the computer and processed. Finally, obtained soldering defects are presented to user via monitor. Proposed computer vision system is shown in Figure 2(b).  When PCB was located to end of the plastic rails, camera takes same pictures to evaluate. Thus, this trick is very important to improve evaluation results.

Fuzzy C-Means Clustering for Joint Detection
Fuzzy C-Means (FCM) groups the data in a data set according to the similarity rate. Therefore, the similar data is assigned the same group. FCM was originally proposed by Dunn [11] and advanced by Bezdek et al. [12]. Basically it is an iterative clustering algorithm that minimizes the squared error objective function. For clustering, each data is assigned to a group according to its membership value.
Then, the fuzzy membership matrix is updated according to Equation 3 where, A is one of the members set area, and sub parameters i,j,k, m,n and c are characterized to show numerical values of members. Fig. 3 shows how fuzzy makes defuzzication to obtain output. Fuzzy implementation is especially used and focused to find borders of soldering areas. In order to improve proposed method success Fuzzy was merged our aim. Especially any error during the understanding the border of a soldering area, reduces system performance and ability.

Joint Template Usage for Joint Contact Inspection
In order to obtain effective solution of this problem in real life, the inspection time and fully success are so importance to obtain robust behavior for inspection created method. Technological solutions that makes slow down production speed are generally not preferred. For this reason, rather than very heavy and complicated systems, simple and fast systems are preferred. Fig.4 shows prepared structure to take photo to scrutinize the PCB. The inspection of soldering defects for a PCB can be done by examining all joints against all other joints. In this process, the relationship of each joint to all other joints is examined. Even though the result is quite successful, the inspection time is quite long for this process. The solution makes inspection time longer would be useless because of the long time. Instead, the joints that should not touch a joint are identified and only the relationship with these joints is examined to reduce inspection time without losing total success. In this study, a joint template-based inspection method has been presented for rapid detection of soldering defects. In order to prepare the template, every joint exeminated by manually which joints can connect the other neighbor joints reluctantly. Thus, template includes real possible solder defects locations that come into connect. During the inspection, each joint is examined only for dangerous joints. The steps for creating a template are as follows; 1-For each joint, the joints could which are likely to come into contact reluctantly are identified. Because there is no possibilities a short circuit between the target joint and the joints in far regions. 2-If the two joints are connected together by circuit tracks, then no inspection is performed between those joints. 3-Inspections are performed between close joints that are not interconnected by circuit paths. An example template is shown in Fig. 5. In the Fig. 5, the numbers on the left and above indicate the numbers of the joints. The colors in the squares where the joints intersect have certain meanings. The red color indicates that the two joints should not touch each other. From Fig. 5, soldering area, numbered 7, cannot touch area 2, on the contrary, even if area 7 touch the area 1, this is not crucial problem to search. Non-colored soldering areas have no possible to touch each others. The blue color indicates that the two joints are already connected to each other via circuit tracks. A joint is only inspected with the joints that have the color relationship specified in the template. Because there is no possibilities for soldering defects with other joints. We can explain the inspection using template process for the first joint as follows; the first joint has only a short circuit with the second joint and the seventh joint. For this reason, only these two probes must be inspected. But, according to the template, the first joint and the seventh joint are already short-circuited with circuit paths. For this reason, they do not need to be examined. Only one examination is performed between the first joint and the second joint. Thus, instead of eight reviews, only one review process is sufficient for the first joint.

EXPERIMENTS AND EXPERIMENTAL RESULTS
In experiments, PCB images are converted to gray level images. The value of each pixel is assigned to the range 0-255. This way it is easier to cluster each pixel. The raw RGB image has 1296x780x3 pixels and after gray level conversion image size turns 1296x780 pixels. The clustering of the pixels is done with FCM. 3 was selected as the cluster number. The first cluster represents the empty PCB areas, the second cluster represents circuit paths, and the third cluster represents soldering joints. In order to improve searching speed, picture related to PCB is converted to gray scale.
Making gray protects all possible features and abilities in gray mode and could be executed in one time to obtain same resolution.  Fig.7) includes all joints. Template makes easier and faster searching due to result oriented working. Fig. 7 shows exeminated PCB. Left side (a) is target PCB which has no problem. Middle picture in Fig. 7b shows that area 24 and joint 25 have connection each other. But regarding the template this not a problem (blue boxed in the Fig.  7b). On the contrary, area 14 and 22 creates short circuit that makes malfunction on the circuit (red boxed in Fig. 7c). Thus, red colored touching in Fig.7c right side is a crucial problem that must be aware and be corrected by. The control process is automatically performed according to the color boxes in the template. In Figure  5, a joint is not inspected by the remaining 49 joints on the template. It is only inspected with the joints where the red boxes in the row or column are found. Same PCBs were prepared as target and different soldering defected PCB (created solder defect on soldering areas, intentionally). For different soldering defects examination executed 85 times for every kind soldering problem. Some of these faults do not adversely affect circuit operation, and some of them have adverse effects. Soldering faults with negative effects are enclosed in the red box, while soldering problems without negative effects are marked with the black box as normal joints. These results are presented to user with a monitor. Soldering joint inspection results are shown in Figure 7. Soldering defects detection accuracy, calculated based on the catching of soldering errors. All soldering faults in the PCB were detected successfully.

CONCLUSION
In this study, a fast and successful computer vision method is presented for PCB soldering defect inspection. The purpose of the work is to create a faster and more successful application for soldering defects in PCBs. Because slow inspection methods slow down production speed and create monetary loss. In the playground, RGB image is first converted to gray level image. Then, the image is clustered with FCM to determine the soldering joints. A template is created manually to quick inspection to query soldering joints. All joints are inspected efficiently with the joints specified in the template. In this way, the process time is reduced by dramatically via guided template. All soldering defects in experiments have been detected correctly. In order to obtain zero errors, inspected PCB checked by manually and powered for run in real life. Success was obtained as 100% for inspected PCB model. Sample PCB is one side soldered and suit for trough hole components. Fuzzy improves border finding and template makes faster the proposed solution. Results encourage us to use the proposed method for wide, big and complex PCB to find soldering problems.