Extraction of fuzzy logic rules from data by means of artificial neural networks

Martin Holeňa

Kybernetika (2005)

  • Volume: 41, Issue: 3, page [297]-314
  • ISSN: 0023-5954

Abstract

top
The extraction of logical rules from data has been, for nearly fifteen years, a key application of artificial neural networks in data mining. Although Boolean rules have been extracted in the majority of cases, also methods for the extraction of fuzzy logic rules have been studied increasingly often. In the paper, those methods are discussed within a five-dimensional classification scheme for neural-networks based rule extraction, and it is pointed out that all of them share the feature of being based on some specialized neural network, constructed directly for the rule extraction task. As an important representative, a method for the extraction of rules in a general fuzzy disjunctive normal form is described in detail and illustrated on real-world applications. Finally, the paper proposes an algorithm demonstrating a principal possibility to extract fuzzy logic rules from multilayer perceptrons with continuous activation functions, i. e., from the kind of neural networks most universally used in applications. However, complexity analysis of the individual steps of that algorithm reveals that it involves computations with doubly-exponential complexity, due to which it can not without simplifications serve as a practically applicable alternative to methods based on specialized neural networks.

How to cite

top

Holeňa, Martin. "Extraction of fuzzy logic rules from data by means of artificial neural networks." Kybernetika 41.3 (2005): [297]-314. <http://eudml.org/doc/33755>.

@article{Holeňa2005,
abstract = {The extraction of logical rules from data has been, for nearly fifteen years, a key application of artificial neural networks in data mining. Although Boolean rules have been extracted in the majority of cases, also methods for the extraction of fuzzy logic rules have been studied increasingly often. In the paper, those methods are discussed within a five-dimensional classification scheme for neural-networks based rule extraction, and it is pointed out that all of them share the feature of being based on some specialized neural network, constructed directly for the rule extraction task. As an important representative, a method for the extraction of rules in a general fuzzy disjunctive normal form is described in detail and illustrated on real-world applications. Finally, the paper proposes an algorithm demonstrating a principal possibility to extract fuzzy logic rules from multilayer perceptrons with continuous activation functions, i. e., from the kind of neural networks most universally used in applications. However, complexity analysis of the individual steps of that algorithm reveals that it involves computations with doubly-exponential complexity, due to which it can not without simplifications serve as a practically applicable alternative to methods based on specialized neural networks.},
author = {Holeňa, Martin},
journal = {Kybernetika},
keywords = {knowledge extraction from data; artificial neural networks; fuzzy logic; Lukasiewicz logic; disjunctive normal form; knowledge extraction from data; artificial neural network; fuzzy logic; Łukasiewicz logic; disjunctive normal form},
language = {eng},
number = {3},
pages = {[297]-314},
publisher = {Institute of Information Theory and Automation AS CR},
title = {Extraction of fuzzy logic rules from data by means of artificial neural networks},
url = {http://eudml.org/doc/33755},
volume = {41},
year = {2005},
}

TY - JOUR
AU - Holeňa, Martin
TI - Extraction of fuzzy logic rules from data by means of artificial neural networks
JO - Kybernetika
PY - 2005
PB - Institute of Information Theory and Automation AS CR
VL - 41
IS - 3
SP - [297]
EP - 314
AB - The extraction of logical rules from data has been, for nearly fifteen years, a key application of artificial neural networks in data mining. Although Boolean rules have been extracted in the majority of cases, also methods for the extraction of fuzzy logic rules have been studied increasingly often. In the paper, those methods are discussed within a five-dimensional classification scheme for neural-networks based rule extraction, and it is pointed out that all of them share the feature of being based on some specialized neural network, constructed directly for the rule extraction task. As an important representative, a method for the extraction of rules in a general fuzzy disjunctive normal form is described in detail and illustrated on real-world applications. Finally, the paper proposes an algorithm demonstrating a principal possibility to extract fuzzy logic rules from multilayer perceptrons with continuous activation functions, i. e., from the kind of neural networks most universally used in applications. However, complexity analysis of the individual steps of that algorithm reveals that it involves computations with doubly-exponential complexity, due to which it can not without simplifications serve as a practically applicable alternative to methods based on specialized neural networks.
LA - eng
KW - knowledge extraction from data; artificial neural networks; fuzzy logic; Lukasiewicz logic; disjunctive normal form; knowledge extraction from data; artificial neural network; fuzzy logic; Łukasiewicz logic; disjunctive normal form
UR - http://eudml.org/doc/33755
ER -

References

top
  1. Adamo J. M., Data Mining for Association Rules and Sequential Patterns: Sequential and Parallel Algorithms, Springer–Verlag, Berlin 2001 Zbl0998.68228
  2. Aguzzoli S., Mundici D., Weierstrass approximations by Lukasiewicz formulas with one quantified variable, In: 31st IEEE Internat. Symposium on Multiple-Valued Logic, 2001 
  3. Alexander J. A., Mozer M. C., 10.1016/S0893-6080(99)00009-X, Neural Networks 12 (1999), 479–498 (1999) DOI10.1016/S0893-6080(99)00009-X
  4. Amato P., Nola, A. Di, Gerla B., Neural networks and rational Lukasiewicz logic, J. Multiple-Valued Logic and Soft Computing (accepted for publication) 
  5. Amato P., Porto M., An algorithm for the automatic generation of logical formula representing a control law, Neural Network World 10 (2000), 777–786 
  6. Andrews R., Diederich, J., Tickle A. B., 10.1016/0950-7051(96)81920-4, Knowledge-based Systems 8 (1995), 378–389 (1995) DOI10.1016/0950-7051(96)81920-4
  7. Bern M., Chew L. P., Eppstein, D., Ruppert J., Dihedral bounds for mesh generation in high dimensions, In: Proc. Sixth ACM-SIAM Symposium on Discrete Algorithms, ACM, San Francisco 1995, pp. 189–196 (1995) Zbl0849.68116MR1321850
  8. Berthold M., Hand D., editors, 10.1007/978-3-662-03969-4, An Introduction. Springer–Verlag, Berlin 1999 MR1723394DOI10.1007/978-3-662-03969-4
  9. Chen J., Liu J., 10.1021/ie9905613, Indust. Engrg. Chemistry Research 39 (2000), 2355–2367 DOI10.1021/ie9905613
  10. Cignoli L. O., D’Ottaviano I. M. L., Mundici D., Algebraic Foundations of Many-valued Reasoning, Kluwer Academic Publishers, Dordrecht 2000 Zbl0937.06009MR1786097
  11. Garcez A. S. d’Avila, Broda, K., Gabbay D. M., 10.1016/S0004-3702(00)00077-1, Artificial Intelligence 125 (2001), 155–207 MR1805645DOI10.1016/S0004-3702(00)00077-1
  12. Daňková M., Perfilieva I., 10.1007/s00500-002-0209-3, Soft Computing 7 (2003), 228–233 Zbl1029.03503DOI10.1007/s00500-002-0209-3
  13. Raedt L. De, Interactive Theory Revision: An Inductive Logic Programming Approach, Academic Press, London 1992 
  14. Duch W., Adamczak, R., Grabczewski K., 10.1023/A:1009670302979, Neural Processing Lett. 7 (1998), 211–219 (1998) DOI10.1023/A:1009670302979
  15. Duch W., Adamczak, R., Grabczewski K., A new methodology of extraction, optimization and application of crisp and fuzzy logical rules, IEEE Trans. Neural Networks 11 (2000), 277–306 
  16. Dzeroski S., Lavrac N., Relational Data Mining, Springer–Verlag, Berlin 2001 Zbl1003.68039
  17. Edelsbrunner H., Algorithms in Combinatorial Geometry, Springer–Verlag, Heidelberg 1987 Zbl0634.52001MR0904271
  18. Esteva F., Godo, L., Montagna F., 10.1007/s001530050173, Arch. Math. Logic 40 (2001), 39–67 MR1816606DOI10.1007/s001530050173
  19. Faber J., Novák M., Svoboda, P., Tatarinov V., Electrical brain wave analysis during hypnagogium, Neural Network World 13 (2003), 41–54 
  20. Finn G. D., 10.1007/s005210050003, Neural Computing Appl. 8 (1999), 9–24 (1999) DOI10.1007/s005210050003
  21. Freitas A. A., Data Mining and Knowledge Discovery with Evolutionary Algorithms, Springer–Verlag, Berlin 2002 Zbl1013.68075
  22. Gehrke M., Walker C. L., Walker E. A., Normal forms and truth tables for fuzzy logics, Fuzzy Sets and Systems 138 (2003), 25–51 Zbl1027.03023MR2012239
  23. Hájek P., Metamathematics of Fuzzy Logic, Kluwer Academic Publishers, Dordrecht 1998 Zbl1007.03022MR1900263
  24. Hájek P., Havránek T., Mechanizing Hypothesis Formation, Springer–Verlag, Berlin 1978 Zbl0371.02002MR0501342
  25. Healy M. J., Caudell T. P., 10.1109/72.572088, IEEE Trans. Neural Networks 8 (1997), 461–474 (1997) DOI10.1109/72.572088
  26. Holeňa M., Extraction of logical rules from data by means of piecewise-linear neural networks, In: Proc. 5th Internat. Conference on Discovery Science, Springer–Verlag, Berlin 2002, pp. 192–205 Zbl1024.68560
  27. Holeňa M., Baerns M., Artificial neural networks in catalyst development, In: Experimental Design for Combinatorial and High Throughput Materials Development (J. N. Cawse, ed.), Wiley, Hoboken 2003, pp. 163–202 
  28. Holeňa M., Baerns M., Feedforward neural networks in catalysis, A tool for the approximation of the dependency of yield on catalyst composition, and for knowledge extraction. Catalysis Today 81 (2003), 485–494 
  29. Horčík R., Cintula P., 10.1007/s00153-004-0214-6, Arch. Math. Logic 43 (2004), 477–503 Zbl1059.03011MR2060396DOI10.1007/s00153-004-0214-6
  30. Ishikawa M., 10.1016/S0893-6080(00)00072-1, Neural Networks 13 (2000), 1171–1183 DOI10.1016/S0893-6080(00)00072-1
  31. Lu H., Setiono, R., Liu H., 10.1109/69.553163, IEEE Trans. Knowledge and Data Engrg. 8 (1996), 957–961 (1996) DOI10.1109/69.553163
  32. Maire F., 10.1016/S0893-6080(99)00013-1, Neural Networks 12 (1999), 717–725 (1999) DOI10.1016/S0893-6080(99)00013-1
  33. McNaughton R., 10.2307/2268660, J. Symbolic Logic 16 (1951), 1–13 (1951) Zbl0043.00901MR0041799DOI10.2307/2268660
  34. Mitra S., De R. K., Pal S. K., 10.1109/72.641457, IEEE Trans. Neural Networks 8 (1997), 1338–1350 (1997) DOI10.1109/72.641457
  35. Mitra S., Hayashi Y., 10.1109/72.846746, IEEE Trans. Neural Networks 11 (2000), 748–768 DOI10.1109/72.846746
  36. Muggleton S., Inductive Logic Programming, Academic Press, London 1992 Zbl1132.68007
  37. Mundici D., 10.2307/2275410, J. Symbolic Logic 59 (1994), 596–602 (1994) Zbl0807.03012MR1276635DOI10.2307/2275410
  38. Narazaki H., Watanabe, T., Yamamoto M., 10.1109/3477.484442, IEEE Trans. Systems Man Cybernet. 26 (1996), 107–117 (1996) DOI10.1109/3477.484442
  39. Nauck D., Nauck, U., Kruse R., Generating classification rules with the neuro-fuzzy system NEFCLASS, In: Proc. Biennial Conference of the North American Fuzzy Information Processing Society NAFIPS’96, 1996, pp. 466–470 (1996) 
  40. Novák V., Perfilieva I., Some consequences of herbrand and McNaughton theorems in fuzzy logic, In: Discovering World with Fuzzy Logic: Perspectives and Approaches to Formalization of Human-Consistent Logical Systems (V. Novák and I. Perfilieva, eds.), Springer–Verlag, Heidelberg 1999, pp. 271–295 (1999) MR1858104
  41. Novák V., Perfilieva, I., Močkoř J., Mathematical Principles of Fuzzy Logic, Kluwer Academic Publishers, Dordrecht 1999 Zbl0940.03028MR1733839
  42. Perfilieva I., Neural nets and normal forms from fuzzy logic point of view, Technical Report, Institute for Research and Applications of Fuzzy Modelling, 2001 
  43. Perfilieva I., Normal forms for fuzzy logic functions and their approximation ability, Fuzzy Sets and Systems 124 (2001), 371–384 Zbl0994.03019MR1860857
  44. Perfilieva I., 10.1007/s00500-002-0173-y, Soft Computing 7 (2003), 73–78 DOI10.1007/s00500-002-0173-y
  45. Perfilieva I., 10.1016/j.fss.2003.06.009, Fuzzy Sets and Systems 143 (2004), 111–127 MR2060276DOI10.1016/j.fss.2003.06.009
  46. Perfilieva I., 10.1007/s00500-003-0274-2, Soft Computing 8 (2004), 291–298 Zbl1077.03047DOI10.1007/s00500-003-0274-2
  47. Perfilieva I., Kreinovich V., 10.1002/int.10063, Internat. J. Intelligent Systems 17 (2002), 1121–1130 Zbl1028.68169DOI10.1002/int.10063
  48. Polkowski L., Rough Sets, Mathematical Foundations. Physica–Verlag, Heidelberg 2002 Zbl1166.68307MR1967129
  49. Quinlan J., C4, 5: Programs for Machine Learning. Morgan Kaufmann Publishers, San Francisco 1992 Zbl0900.68112
  50. Setiono R., 10.1162/neco.1997.9.1.205, Neural Computation 9 (1997), 05–225 (1997) Zbl0872.68155DOI10.1162/neco.1997.9.1.205
  51. Tickle A. B., Andrews R., Golea, M., Diederich J., 10.1109/72.728352, IEEE Trans. Neural Networks 9 (1998), 1057–1068 (1998) DOI10.1109/72.728352
  52. Towell G. G., Shavlik J. W., 10.1007/BF00993103, Mach. Learning 13 (1993), 71–101 (1993) DOI10.1007/BF00993103
  53. Triantaphyllou E., (eds.) G. Felici, Data Mining and Knowledge Discovery Approaches Based on Rule Induction Techniques, Kluwer Academic Publishers, Dordrecht 2003 Zbl1117.68028
  54. Tsukimoto H., 10.1109/72.839008, IEEE Trans. Neural Networks 11 (2000), 333–389 DOI10.1109/72.839008
  55. Wong M. L., Leung K. S., Data Mining Using Grammar Based Genetic Programming and Applications, Kluwer Academic Publishers, Dordrecht 2000 Zbl0944.68172
  56. Zhang C., Zhang, S., Heymer B. E., Association Rule Mining: Models and Algoritms, Springer–Verlag, Berlin 2002 

NotesEmbed ?

top

You must be logged in to post comments.

To embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.

Only the controls for the widget will be shown in your chosen language. Notes will be shown in their authored language.

Tells the widget how many notes to show per page. You can cycle through additional notes using the next and previous controls.

    
                

                
Note: Best practice suggests putting the JavaScript code just before the closing </body> tag.