Download Fuzzy Logic and Intelligent Systems by M. Mizumoto (auth.) PDF

Show description

T h e o r e m 2 (Stone-Weierstrass Theorem II, Ray, [28]) Let S be a compact set with n dimensions, and il D C[S] be a set of continuous real-valued functions on S satisfying the conditions: (a) Identity Function: The constant function / ( x ) = 1 is in il; (b) Separability: For any two points Xi, Xa e S and Xi ^ X2, there exists a f £ il such that / ( x i ) ^ f(x2); (c) Algebraic Closure: For any f,gSil and a, P G R, the functions fg and af •{• fig are in H. Then il is dense in C\S\. Although the Stone-Weierstrass theorem has a potential application to continuous function approximation, many interesting functions, including step functions, are discontinuous.

This issue has been investigated by many a u t h o r s including Carroll and Dickinson [6], Cybenko [5], Funahashi [7], Gallant and W h i t e [8], Heclit-Nielsen [10], and Hornik [15]. For function approximation, b o t h the series expansion approach and the Stone-Weierstrass theorem are very effective analytic tools. However, Hedit-Nielsen [10,11] found the relationship between the Kolmogorov's theorem and the approximation principle of the feedforward networks. Indeed, functional analytic m e t h o d s have been successfully used to show t h a t feedforward neural structures with at-Ieast one hidden layer are capable of simultaneously approximating continuous functions in several variables and their derivatives if the neural activation functions of the hidden neural imits are differentiable [15].

This leads a natural extension that the sigmoidal functions often used in the conventional neural networks may be replaced with a more general class of the squashing functions in terms of the universal approximation. An interesting consequence is that the approximation using a two-layered network, as shown in Figure 4 with the hidden units of McCulloch-Pitts, called Mc-P units, is easily implied. " the network is capable of approximating any continuous function to a desired degree of accuracy. This conclusion is also summarized in the following corollary.