Training Product-Unit Neural Networks with Cuckoo Optimization Algorithm for Classification

In this study Product-Unit Neural Networks (PUNN) which is the special class of feed-forward neural network has been trained using Cuckoo Optimization algorithm. The trained model has been applied to two classification problems. BUPA liver disorders and Haberman's Survival Data have been used for application. The both data have been obtained from UCI machine Learning Repository. For comparison, Backpropagation (BP) and Levenberg–Marquardt (LM) algorithms have been used. Classification accuracy, sensitivity, specificity and F1 score have been used as statistics to evaluate the success of algorithms. The application results show that the PUNN trained with Cuckoo Optimization algorithm is achieved better classification accuracy, sensitivity, specificity and F1 score.


Introduction
Artificial neural network (ANN) is a mathematical model which was inspired from the biological neural networks. ANN is an artificial intelligent model that utilizes information coming from different nodes known as neurons in this model [1]. ANNs are very helpful to solve complex problems. They also show good performances on analysing non-linear systems. ANNs have been applied over a wide range of fields for modelling and predicting. The reason why the ANN is frequently used is presenting some properties such as learning from examples and showing some ability for generalization beyond the training data [2]. A Product unit neural network (PUNN) a special type of an artificial neural network. PUNNs are an alternative to classical neural networks. In PUNNs, additive nodes were replaced by multiplicative ones. [3]. PUNNs have a number of advantages.
• They have great predictive power.

•
They are a product network allows for simplification.

•
They enable insight in relation between inputs and outputs.

•
They are transportable [4]. Guerrero-Enamorado and Ceballos-Gastell [5] intended aims to obtain empirical information about the behavior of an Evolutionary Product-Unit Neural Network (EPUNN). In the application part they used 21 data….sonra hangi fiil olacak 21 data sets for the classification task, eleven noisy data sets, sixteen imbalanced data sets, and ten missing values data sets. Hervás et al. [3] proposed a classification method based on product-unit neural networks. They applied an evolutionary algorithm to specify the structure of PUNN and to estimate the model's coefficients. Zhang et al. [6] investigated the convergence of BP algorithm for PUNN. Martı´nez-Estudillo et al. [7] presented a model of evolution of PUNN to cope the local minima on the error surface problem. The presented model improves both the weights and the structure of these networks by means of an evolutionary programming algorithm. The performance of the model is evaluated using five benchmark functions. Dulakshi et al. [8] investigated the applicability of PUNN to hydrological time series prediction. They noted that PUNN achieved more accurate results than ANN. Martı´nez-Estudillo et al. [9] proposed a classification method based on PUNN. They applied an evolutionary algorithm to specify the basic structure of the PUNN model. The paper is arranged as follows: Section 2 briefly reviews the theoretical background of study. Section 3 introduces the databases which are used for application aim. Section 4 describes the application results. In the section 5 the paper is concluded.

Product-Unit Neural Networks
In PUNN, the summation units of the hidden layer are replaced by the product units with weights. is a weight between th neuron of input vector and th neuron of hidden layer. Thus the th neuron of hidden layer is is a weight between th neuron of hidden layer and th neuron of output layer. Thus the input of th neuron of output layer is Where is an activation function.

Cuckoo Optimization Algorithm
Cuckoo Optimization Algorithm (COA) inspired by the life of a bird family which is called cuckoo has been presented by Rajabioun [10]. Fig. 2 shows a flowchart of the COA. The basic motivation of this algorithm was a cuckoos special egg laying and breeding style. The algorithm starts with an initial cuckoo population. The initial cuckoos lay eggs to nest of host bird. Some of these laid eggs are more alike to the eggs of host bird's and thus they have the chance to grow up. The less alike eggs are determined and are killed by host birds. The suitability of the nests in the area is determined through the number of grown eggs reveal. So the task of COA is to find better location in which more eggs can survive. In this study COA has been used to determine the weights of PUNN. The method begins with a random initial population and in each iteration, the population modifies via COA principles. The each individual presents the weights of PUNN. Accuracy (5) equation has been used as fitness function.
Where TP and FP are the number of correctly and incorrectly identified, TN and FN are the number of correctly rejected and incorrectly rejected cases, respectively.

BUPA liver disorders
BUPA liver disorders dataset has been used in implementation phase. The data sets have been obtained from the UCI Machine Learning Repository [11]. The data were collected by Richard S. Forsyth in BUPA Medical Research Ltd. The data set consists of 345 data. Data has 6 features which can be seen in Table 1. Of these, 200 individuals suffer from alcoholism and 145 don't have this problem.

Haberman's Survival Data
Haberman's Survival Data is other database which was used in implementation phase. The data sets have been obtained from the UCI Machine Learning Repository [11]. The data were collected by Tjen-Sien Lim. The data set consists of 306 data. Data has 3 features which can be seen in Table 2. Of these, 225 individuals survived 5 years or longer and 81 are the patients which died within 5 year.

Performance Metrics
Accuracy, sensitivity, specificity and F1 score have been used as performance metrics in this study. These statistics are calculated as follows: Here TP and FP are the number of correctly and incorrectly identified, TN and FN are the number of correctly rejected and incorrectly rejected cases, respectively.

Application of BUPA liver disorders
Dataset of BUPA liver disorders has 345 patients. The dataset has been split into two subsets as training and testing.      The training and testing dataset have 230 and 115 records respectively. The three-layer architecture has been used. The network has been trained using BP, LM and proposed algorithms.
The neurons in hidden layer have been used between 2-12. The results of application are shown in Table 3, Table 4 and Table 5.
As it seen from Table 3 the BP algorithm failed in classification of BUPA. From Table 4, it can be seen that LM is more successful than BP for BUPA classification. LM is more successful in accuracy and both sensitivity and specificity. What is more, the F1 score is increased by LM. However, PUNN trained using COA is more successful than both LM and BP for both training and testing sets. The PUNN trained with COA is also achieved a successful result for sensitivity, specificity and F1 score.

Application of Haberman's Survival Data
Haberman's Survival Data Dataset has 306 patients. The dataset has been split into two subsets as training and testing. The training and testing dataset have 204 and 102 records respectively. The three-layer architecture has been used. The network has been trained using BP, LM and proposed algorithms. The neurons in hidden layer have been used between 2 and 12. The results of application are shown in Table 6-8. As it seen from Table 6 and Table 7 the BP and LM algorithms both produced a successful result for Haberman Survival Data while BP is better than LM for training set. From Table 8, PUNN trained using COA is seen as more unsuccessful than LM, while it produced best results for training set, which is more important. Furthermore, sensitivity, specificity and F1 Score are better.

Conclusion
In this paper, product unit neural network has been trained using Cuckoo optimization algorithm. BUPA liver disorders and Haberman's Survival Data have been used for the application stage. For comparison aims, BP and LM algorithm have been used. The results show that PUNN trained with COA produced best accuracy for BUPA liver disorders. It achieved 71.30% of accuracy for testing set. Thus, the LM algorithm outperformed BP. For Haberman's Survival Data, the best was PUNN trained with COA. It achieved 78.43% of classification accuracy for testing set. BP and LM achieved equal accuracy as 75.49%. Therefore, the results showed that PUNN reaches better performance than BP and LM.