Document Type : Review Article
Authors
Abstract
Knowledge embedded within artificial neural networks (ANNs) is distributed over the connections and weights of neurons. So, the user considers ANN as a black box system. There are many researches investigating the area of rule extraction by ANNs. In this paper, a dynamic cell structure (DCS) neural network and a modified version of LERX algorithm are used for rule extraction. On the other hand, intrusion detection system (IDS) is known as a critical technology to secure computer networks. So, the proposed algorithm is used to develop an IDS and classify the patterns of intrusion. To compare the performance of the proposed system with other machine learning algorithms, a multi layer perceptron (MLP) and an Elman neural network are employed with selected inputs based on the results of a feature relevance analysis. Empirical results show the superior performance of the IDS based on rule extraction from DCS in recognizing hard-detectable attack categories, e.g. user-to-root (U2R). Although, MLP with 15 selected input features, instead of 41 standard features introduced by knowledge discovery and data mining group (KDD), has better classification rates for other attack categories. This network performs better in terms of detection rate (DR), false alarm rate (FAR), and cost per example (CPE) when compared with some other machine learning methods, as well.
Keywords
[1] R. Andrews, J. Diederich, and A.B. Tickle, “A survey and critique of techniques for extracting rules from trained artificial neural networks”, Knowledge-Based Systems, 8, pp. 373-389, (1995)
[2] F. Behloul, B.P.F. Lelieveldt, A. Boudraa, and J.H.C. Reiber, “Optimal design of radial basis function neural networks for fuzzy-rule extraction in high dimensional data”, Pattern Recognition, 35, pp. 659-675, (2002).
[3] C.J. Mantas, J.M. Puche, and J.M. Mantas, “Extraction of similarity based fuzzy rules from artificial neural networks”, International Journal of Approximate Reasoning, 43, pp. 202-221, (2006)
[4] G. Towell, and J. Shavlik, “The extraction of refined rules from knowledge based neural networks”, Machine Learning, 13, pp. 71-101, (1993)
[5] C.W. Omlin, and C.L. Giles, “Extraction of rules from discrete-time recurrent neural networks”, Neural Networks, 9, pp. 41-52, (1996)
[6] S.H. Huang, and H. Xing, “Extract intelligible and concise fuzzy rules from neural networks”, Fuzzy Sets and Systems, 132, pp. 233-243, (2002)
[7] G. Bologna, “Is it worth generating rules from neural network ensembles?”, Journal of Applied Logic, 2, pp. 325-348, (2004)
[8] G. Leng, T.M. McGinnity, and G. Prasad, “An approach for on-line extraction of fuzzy rules using a self-organizing fuzzy neural network”, Fuzzy Sets and Systems, 150, pp. 211-243, (2005)
[9] E.R. Hruschka, and N.F.F. Ebecken, “Extracting rules from multilayer perceptrons in classification problems: a clustering-based approach”, Neurocomputing, 70, pp. 384-397, (2006)
[10] K. Odajima, Y. Hayashi, G. Tianxia, and R. Setiono, “Greedy rule generation from discrete data and its use in neural network rule extraction”, Neural Networks, 21, pp. 1020-1028, (2008)
[11] L.E. Zarate, S.M. Dias, and M.A.J. Song, “FCANN: A new approach for extraction and representation of knowledge from ANN trained via formal concept analysis”, Neurocomputing, 71, pp. 2670-2684, (2008)
[12] R. Setiono, B. Baesens, and C. Mues, “A note on knowledge discovery using neural networks and its application to credit card screening”, European Journal of Operational Research, 192, pp. 326-332, (2009)
[13] E. Kolman, and M. Margaliot, “Extracting symbolic knowledge from recurrent neural networks-a fuzzy logic approach”, Fuzzy Sets and Systems, 160, pp. 145-161, (2009)
[14] H. Kahramanli, and N. Allahverdi, “Rule extraction from trained adaptive neural networks using artificial immune systems”, Expert Systems with Applications, 36, pp. 1513-1522, (2009)
[15] S. Yu, X. Guo, K. Zhu, and J. Du, “A neuro-fuzzy-GA-BP method of seismic reservoir fuzzy rules extraction”, Expert Systems with Applications, 37, pp. 2037-2042, (2010)
[16] Y. Hayashi, “A neural expert system with automated extraction of fuzzy if-then rules”, Advances in Neural Information Processing Systems, Vol. 3, pp. 1263-1268, (1991)
[17] C.L. Giles, and C.W. Omlin, “Extraction, insertion, and refinement of symbolic rules in dynamically driven recurrent networks”, Connection Science, 5, pp. 307-328, (1993)
[18] L.M. Fu, “Rule generation from neural networks”, IEEE Transactions on System, Man and Cybernetics, 28, pp. 1114-1124, (1994)
[19] D.W. Optiz, and J.W. Shavlik, “Dynamically adding symbolically meaningful nodes to knowledge-based neural networks”, Knowledge-Based Systems, 8, pp. 301-311, (1995)
[20] K. Saito, and P. Nakano, “Medical diagnosis expert system based on PDP model”, Proc. Int. Conf. Neural Networks, Vol. 1, pp. 255-262, (1988)
[21] E. Keedwell, A. Narayanan, and D. Savic, “Creating rules from trained neural networks using genetic algorithms”, International Journal of Computers, Systeming Signals, 1, pp. 30-42, (2000)
[22] R. Andrews, and S. Geva, Rule extraction from local cluster neural nets, Neurocomputing, 47, pp. 1-20, (2002)
[23] K. McGarry, J. Tait, S. Wermter, and J. McIntyre, “Rule-extraction from radial basis function networks”, Proc. Int. Conf. Artificial Neural Networks, Vol. 1, pp. 613-618, (1999)
[24] K. McGarry, S. Wermter, and J. McIntyre, The extraction and comparison of knowledge from local function networks, International Journal of Computational Intelligence and Applications, 1, pp. 369-382, (2001)
[25] P. Garcia-Teodoro, J. Diaz-Verdejo, G. Macia-Fernandez, and E. Vazquez, “Anomaly-base network intrusion detection: techniques, systems and challenges”, Computers & Security, 28, pp. 18-28, (2009)
[26] P. Kabiri, and A.A. Ghorbani, “Research in intrusion detection and response-a survey”, International Journal of Network Security, 1, pp. 84-102, (2005)
[27] T. Shon, and J. Moon, “A hybrid machine learning approach to network anomaly detection”, Information Sciences, 177, pp. 3799-3821, (2007)
[28] Z. Chen, H. Wang, B. Yang, L. Wang, and R. Sun, “A FDRS-based data classification method used for abnormal network intrusion detection”, Proc. IEEE 3rd Int. Conf. Natural Computation, Vol. 2, pp. 375-380, (2007)
[29] Y. Chen, A. Abraham, and B. Yang, “Hybrid flexible neural-tree-based intrusion detection systems”, International Journal of Intelligent Systems, 22, pp. 337-352, (2007)
[30] R. Chang, L. Lai, W. Su, J. Wang, and J. Kouh, “Intrusion detection by backpropagation neural networks with sample-query and attribute-query”, International Journal of Computational Intelligence Research, 3, pp. 6-10, (2007)
[31] N. Ye, S.M. Emran, Q. Chen, and S. Vilbert, “Multivariate statistical analysis of audit trials for host-based intrusion detection”, IEEE Transactions on Computers, 51, pp. 810-820, (2002)
[32] C. Kruegel, D. Mutz, W. Robertson, and F. Valeur, “Bayesian event classification for intrusion detection”, Proceedings 19th Ann. Computer Security Applications Conf., pp. 14-23, (2003)
[33] M.V. Mahoney, and P.K. Chan, “Learning nonstationary models of normal network traffic for detecting novel attacks”, Proc. 8th ACM SIGKDD, pp. 376-385, (2002)
[34] R. Beghdad, “Training all the KDD data set to classify and detect attacks”, Neural Network World, 17, pp. 81-91, (2007)
[35] J. Gomez, and D. Dasgupta, “Evolving fuzzy classifiers for intrusion detection”, Proc. IEEE Workshop Information Assurance, pp. 68-75, (2002)
[36] D. Song, M.I. Heywood, and A.N. Zincir-Heywood, “Training genetic programming on half a million patterns: an example from anomaly detection”, IEEE Transactions on Evolutionary Computation, 9, pp. 225-239, (2005).
[37] Y. Liao, and V.R. Vemuri, “Use of K-nearest neighbor classifier for intrusion detection”, Computers & Security, 21, pp. 439-448, (2002)
[38] D. Novikov, R.V. Yampolskiy, and L. Reznik, “Artificial intelligence approaches for intrusion detection”, Proc. IEEE Conf. Systems, Applications and Technology, pp. 1-8, (2006)
[39] M.V. Joshi, R.C. Agrawal, and V. Kumar, “Mining needless in a haystack: classifying rare classes via two-phase rule induction”, Proc. ACM SIGMOD Conf. Management of Data, pp. 91-102, (2001)
[40] V. Golovko, and L. Vaitsekhovich, “Neural network techniques for intrusion detection”, Proc. Int. Conf. Neural Networks and Artificial Intelligence, pp. 65-69, (2006)
[41] A. Herrero, E. Corchado, P. Gastaldo, F. Picasso, and R. Zunino, “Auto-association neural techniques for intrusion detection systems”, Proc. IEEE Int. Symp. Industrial Electronics, pp. 1905-1910, (2007)
[42] R. Beghdad, “Critical study of neural networks in detecting intrusions”, Computers & Security, 27, pp. 168-175, (2008)
[43] M. Sheikhan, Z. Jadidi, and M. Beheshti, “Effects of feature reduction on the performance of attack recognition by static and dynamic neural networks”, World Applied Sciences Journal, 8, pp. 302-308, (2010)
[44] J.E. Dickerson, J. Juslin, J. Koukousoula, and J.A. Dickerson, “Fuzzy intrusion detection”, Proc. IFSA World Congress and 20th North American Fuzzy Information Processing Society (NAFIPS) Int. Conf., Vol. 3, pp. 1506-1510, (2001)
[45] Y. Lin, K. Chen, and X. Liao, “A genetic clustering method for intrusion detection”, Pattern Recognition, 37, pp. 924-927, (2004)
[46] B. Pfahringer, “Winning the KDD 99 classification cup: bagged boosting”, SIGKDD Explorations, 1, pp. 65-66, (2000)
[47] K. Shah, N. Dave, S. Chavon, S. Mukherjee, A. Abraham, and S. Sanyal, “Adaptive neuro-fuzzy intrusion detection system”, Proc. IEEE Int. Conf. Information Technology: Coding and Computing, Vol. 1, pp. 70-74, (2004)
[48] M.S. Abadeh, J. Habibi, and C. Lucas, “Intrusion detection using a fuzzy genetic–based learning algorithm”, Journal of Network and Computer Applications, 30, pp. 414-428, (2005)
[49] J. Bruske, and G. Sommer, “Dynamic cell structures”, Proc. Neural Information Processing Systems, pp. 497-504, (1994)
[50] B. Fritzke, “Growing cell-structures-a self-organizing network for unsupervised and supervised learning”, Neural Networks, 7, pp. 1441-1460, (1994)
[51] T.M. Martinez, “Competitive Hebbian learning rule forms perfectly topology preserving maps”, Proc. Int. Conf. Artificial Neural Networks, pp. 427-434, (1993)
[52] M. Darrah, B. Taylor, and M. Webb, “A geometric rule extraction approach used for verification and validation of a safety critical application”, Proc. 18th Annual Florida Artificial Intelligence Research Society Conf., Vol. 3, pp. 624-627, (2005)
[53] MIT Lincoln Lab., Information Systems Technology Group, “The 1998 intrusion detection off-line evaluation plan” (http://www.11.mit.edu/IST/ideval/docs/1998/id98-eval-11.txt, Mar. 1998).
[54] “1999 KDD cup competition” (http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html, 2007).
[55] W. Lee, S.J. Stolfo, and K.W. Mok, “Mining in a data-flow environment: experience in network intrusion detection”, Proc. 5th ACM SIGKDD, pp. 114-124, (1999)
[56] M.A. Sartori, and P.J. Antsaklis, “A Simple method to derive bounds on the size and to train multilayer neural networks”, IEEE Transactions on Neural Networks, 2, pp. 467-471, (1991)
[57] K. Rohani, M.S. Chen, and M.T. Manry, “Neural subnet design by direct polynomial mapping”, IEEE Transactions on Neural Networks, 3, pp. 1024-1026, (1992)
[58] H.H. Chen, M.T. Manry, and H. Chandrasekaran, “A neural network training algorithm utilizing multiple sets of linear equations”, Conference Record of the 30th Asilomar Conference on Signals, Systems and Computers, pp. 1166-1170, (1996)
[59] P. Werbos, “Backpropagation: past and future”, Proc. Int. Conf. Neural Networks, pp. 343-353, (1988)
[60] R.S. Scalero, and N. Tepedelenlioglu, “A fast new algorithm for training feedforward neural networks”, IEEE Transactions on Signal Processing, 40, pp. 202–210, (1992)
[61] A. Tamilarasan, S. Mukkamala, A.H. Sung, and K. Yendrapalli, “Feature ranking and selection for intrusion detection using artificial neural networks and statistical methods”, Proc. Int. Joint Conf. Neural Networks, pp. 4754-4761, (2006)
[62] R. Agrawal, and M.V. Joshi, “PNrule: a new framework for learning classifier models in data mining (a case-study in network intrusion detection)”, IBM Research Division Report No. RC-21719, (2000)
[63] I. Levin, “KDD classifier learning contest: LLSoft's results overview”, SIGKDD Explorations, 1, pp. 67-75, (2000)
[64] A. Nadjaran Toosi, and M. Kahani, “A novel soft computing model using adaptive neuro-fuzzy inference system for intrusion detection”, Proc. IEEE Int. Conf. Networking, Sensing and Control, pp. 834-839, (2007)
[65] M. Sabhnani, and G. Serpen, “Why machine learning algorithms fail in misuse detection on KDD intrusion detection data set”, Journal of Intelligent Data Analysis, 6, pp. 1-13, (2004)
)