Fuzzy modeling

Constrained (grey-box) identification of operating regime based models enable the effective use of prior knowledge and results in a robust modeling approach Fuzzy model identification is an effective tool for the approximation of uncertain nonlinear systems on the basis of measured data. Among the different fuzzy modeling techniques, the model introduced by Takagi and Sugeno has attracted most attention. The Takagi-Sugeno (TS) model formed by logical rules; that consists of a fuzzy antecedent and a mathematical function as consequent part. The construction of a TS model is usually done in two steps. In the first step, the fuzzy sets in the rule antecedents are determined that partition the input space into a number of fuzzy regions. In the second step, the rule consequents are determined which means identification of (usually linear) models. TS fuzzy model identification is a complex task; there are non-trivial problems as follows: (1) how to automatically partition the input space, (2) how many fuzzy rules are really needed for properly approximating an unknown nonlinear system, and (3) how to construct a fuzzy system from data examples automatically. These problems can be partially solved by the recent developments of fuzzy systems.

A novel approach to data-driven identification of Takagi-Sugeno fuzzy models have been worked out. It allows to translate prior knowledge about the process (including stability, minimal or maximal static gain and settling time) into constraints on the model parameters. This approach has been successfully applied in (adaptive) model predictive control. The advantage of the algorithm is that by constraining the parameters of the local linear models, it is possible to speed up the adaptation and avoid unrealistic model parameters that could result bad control performance. This grey box fuzzy model approach allows the development of TS models also in cases where little experimental data are available. It has been shown that fuzzy models built on the basis of data combined with prior knowledge perform better in control than models obtained from data only.

J. Abonyi, Fuzzy model identification for control, Birkhauser Boston, 2003, 310 pages
J. Abonyi, R. Babuska, H. B. Verbruggen, F. Szeifert, Incorporating prior knowledge in fuzzy model identification, Int. Journal of Systems Science, 31(5), 657-667, 2000, IF 0.268
J. Abonyi, R. Babuska, F. Szeifert, Fuzzy modeling with multivariate membership functions: Gray-box identification and control design, IEEE Systems, Man and Cybernetics, Part B: Cybernetics, 31 (5), pp. 755-767 IF 0.789, 2001
J. Abonyi, R. Babuska, M. Ayala Botto, F. Szeifert, L. Nagy, Identification and control of nonlinear systems using fuzzy Hammerstein models, Industrial and Engineering Chemistry Research, 39, 4302-4314, 2000., IF 1.294

Compact TS-Fuzzy Models through Clustering and OLS plus FIS Model Reduction
Modified Gath--Geva Fuzzy Clustering for Identification of Takagi-Sugeno Fuzzy Models

The construction of interpretable Takagi--Sugeno (TS) fuzzy models by means of clustering is addressed. First, it is shown how the antecedent fuzzy sets and the corresponding consequent parameters of the TS model can be derived from clusters obtained by the Gath--Geva algorithm. To preserve the partitioning of the antecedent space, linearly transformed input variables can be used in the model. This may, however, complicate the interpretation of the rules. To form an easily interpretable model that does not use the transformed input variables, a new clustering algorithm is proposed, based on the Expectation Maximization (EM) identification of Gaussian mixture models. The most relevant consequent variables of the TS model are selected by an orthogonal least squares method based on the obtained clusters. For the selection of the relevant antecedent (scheduling) variables a new method has been developed based on Fisher's interclass separability criteria. This new technique is applied to two well-known benchmark problems: the MPG (miles per gallon) prediction and a simulated second-order nonlinear process. The obtained results are compared with results from the literature.

J. Abonyi, R. Babuska, F. Szeifert, Modified gath-geva fuzzy clustering for identification of Takagi-Sugeno fuzzy models, IEEE Trans. on Systems, Man and Cybernetics, Part B,612-621, Oct, 2002
J. Abonyi, J.A. Roubos, M. Oosterom, F. Szeifert, Compact TS-Fuzzy models through clustering and OLS plus FIS model reduction, FUZZ-IEEE'01 Conference, Sydney, Australia, 1420-1423, 2001, (MATLAB implementation)

Visualization and Complexity Reduction of Neural Networks

The identification of the proper structure of nonlinear neural networks (NNs) is a difficult problem, since these black-box models are not interpretable. The aim of the paper is to propose a new approach that can be used for the analysis and the reduction of these models. It is shown that NNs with sigmoid transfer function can be transformed into fuzzy systems. Hence, with the use of this transformation NNs can be analyzed by human experts based on the extracted linguistic rules. Moreover, based on the similarity of the resulted membership functions the hidden neurons of the NNs can be mapped into a two dimensional space. The resulted map provides an easily interpretable  figure about the redundancy of the neurons. Furthermore, the contribution of these neurons can be measured by orthogonal least squares technique that can be used for the ordering of the extracted fuzzy rules based on their importance. A practical example related to the dynamic modeling of a chemical process system is used to prove that synergistic combination of model transformation, visualization and reduction of NNs is an effective technique, that can be used for the structural and parametrical analysis of NNs.

Kenesei, Tamas, Balazs Feil, Janos Abonyi, Visualization and Complexity Reduction of Neural Networks, Applications of Soft Computing. Springer Berlin Heidelberg, 2009. 43-52.

Hinging hyperplane based regression tree identified by fuzzy clustering and its application

Hierarchical fuzzy modeling techniques have great advantage since model accuracy and complexity can be easily controlled thanks to the transparent model structures. A novel tool for regression tree identification is proposed based on the synergistic combination of fuzzy c-regression clustering and the concept of hierarchical modeling. In a special case (c = 2), fuzzy c-regression clustering can be used for identification of hinging hyperplane models. The proposed method recursively identifies a hinging hyperplane model that contains two linear submodels by partitioning operating region of one local linear model resulting a binary regression tree. Novel measures of model performance and complexity are developed to support the analysis and building of the proposed special model structure. Effectiveness of proposed model is demonstrated by benchmark regression datasets. Examples also demonstrate that the proposed model can effectively represent nonlinear dynamical systems. Thanks to the piecewise linear model structure the resulted regression tree can be easily utilized in model predictive control. A detailed application example related to the model predictive control of a water heater demonstrate that the proposed framework can be effectively used in modeling and control of dynamical systems.

Tamás Kenesei, János Abonyi, Hinging hyperplane based regression tree identified by fuzzyclustering and itsapplication, Applied Soft Computing, 13, 782-792, 2013, (MATLAB implementation)

A priori knowledge based spline smoohing is useful for the data-driven identification of kinetic parameters

In many practical situations, the involvement of laboratory and industrial experiments are expensive and time consuming and accurate measurements cannot be made. This problem results in a small number of data points that are often noisy and obtained at irregular time intervals. Hence, data smoothing and re-sampling are often required to reduce the effect of measurement noise and irregular time intervals. Typically, an interpolation method is used for this purpose, e.g. cubic spline interpolation, but the disadvantage of the common interpolation methods is that they can not utilize any a priori information. Hence, we developed a new cubic spline interpolation approach which utilizes a priori knowledge, e.g. material balance, or prior information about the measured properties. The methodology has been demonstrated through the investigation of a simulated and an industrial chemical reactor that the new method improves the accuracy of the data-driven estimation of kinetic parameters.

J. Madár, J. Abonyi, H. Roubos, F. Szeifert, Incorporating prior knowledge in cubic spline approximation - Application to the Identification of Reaction Kinetic Models, Industrial and Engineering Chemistry Research, 42, 4043-4049, 2003, IF: 1.252

Support Vector Regression

This project deals with transforming Support vector regression (SVR) models into fuzzy systems (FIS). It is highlighted that trained support vector based models can be used for the construction of fuzzy rule-based regression models. However, the transformed support vector model does not automatically result in an interpretable fuzzy model. Training of a support vector model results a complex rule base, where the number of rules are approximately 40-60% of the number of the training data, therefore reduction of the support vector model initialized fuzzy model is an essential task. For this purpose, a three-step reduction algorithm is used based on the combination of previously published model reduction techniques, namely the reduced set method to decrease number of kernel functions, then after the reduced support vector model is transformed into fuzzy rule base similarity measure based merging and orthogonal least-squares methods are utilized. The proposed approach is applied for nonlinear system identification, the identification of a Hammerstein system is used to demonstrate accuracy of the technique with fulfilling the criteria of interpretability.

Tamas Kenesei, Janos Abonyi, Interpretable Support Vector Regression, Artificial Intelligence Research, 1, 11-21, 2012.
Tamas Kenesei, Janos Abonyi, Interpretable Support Vector Machines In Regression, And Classification – Application In Process Engineering, Hungarian Journal of Industrial Chemistry, 35, 101-108, 2007.