## Yi Yiakyik Ybiuil527

fc=i i=i where the coefficients ak and bi are to be determined from the observations, typically using a least-squares or an information-theoretic criterion. If we take the z-transform of the Eq (5.27), we obtain the transfer function

U{-z> i-£jt=i akzh that defines the process dynamics.

Availability of a reliable model gives us the unprecedent power of predicting the outcome of hypothetical operations of the system, thus in many cases enable us to device methods of controlling it. In the case of discrete linear systems described, the choice of coefficients should be consistent with any prior knowledge of spectral peaks the system has. The denominator of the transfer function (after possible pole-zero cancellations) holds all that information, in terms of poles in the z-plane. Much of the linear signal processing literature is devoted to efficient and effective ways of choosing the coefficients in this kind of linear modeling cases of varying complexity. From the dynamical systems point of view, this kind of modeling consists of autocorrelating the signal, and time averaging the forcing function.

When chaotic dynamics is concerned, such models will not be of much use, since they cannot evolve on a strange attractor, that is, they cannot have any positive Lyapunov exponents, or equivalently, they will always have zero Kolmogorov-Sinai entropy. Nonlinear modeling of chaotic processes is based on the idea of a compact geometric attractor on which our observations evolve. The attractor of a chaotic system is a fractal object called a strange attractor. Due to its fractal nature, the orbit of a particular trajectory is folded back on itself by the nonlinear dynamics. Thus, in the neighborhood of any orbit x(i), other orbit points x(r) (i) with r = 1,..., Nb, arrive in the neighborhood at quite different times than i. One can then build various forms of interpolation functions, which account for whole neighborhoods of state space, and how they evolve from near x(i) to the whole set of points near x(i + 1). The use of state space information in the modeling of the temporal evolution of the process is the key innovation in modeling chaotic systems. The general procedure would work for non-chaotic systems as well, but is likely to be less successful, because the neighborhoods are underpopulated to make reliable statistical inferences.

The implementation of this idea is to build parameterized nonlinear functions that take x(i) into x(i + 1) as x(i + l) = f(x(t);a) (5.29)

and then use various criteria to determine the parameters a. Thus building an understanding of local neighborhoods, one can build up a global nonlinear model by piecing the local models to capture much of the attractor structure.

The main departure from linear modeling techniques is to use the state space and the attractor structure dictated by the data itself, rather than to resort to some predefined algorithmic approach. It is likely that there is no algorithmic solution [511] to how to choose a model structure for chaotic systems, as the data from the dynamics dictate properties that are characteristic for the underlying structure.

If we are going to build a continuous model, we need the time derivatives of the measured quantities, which are generally not available. However, one should avoid numerical differentiation whenever possible, as it amplifies measurement noise. One remedy is to smooth the data before taking the time derivatives. The smoothing techniques usually involve least-squares fit of the data using some known functional form, e.g., a polynomial. Instead of approximating the time series data by a single (thus of high order) polynomial over the entire range of the data, it is often desirable to replace each data point by the value taken on by a (low order) least-squares polynomial relevant to a subrange of 2M ■+-1 points, centered, where possible, at the point for which the entry is to be modified. Thus, each smoothed value replaces a tabulated value. For example, if we consider a first order least squares fit with three points, the smoothed values, in terms of the original values, j/,, would be computed as follows:

If the system is sampled with a fixed frequency, I/At, an interpolation formula, such as Newton's, may be used, and the resulting formula is differentiated analytically. If the sampling is not done homogeneously, then Lagrange's formulae must be used. The following differentiation formulae are obtained for uniformly sampled data points by differentiating a three-point Lagrange interpolation formula

Next, we can consider the reconstructed state space, and seek which set of coordinates give enough information about the time derivative of each coordinate. This is illustrated with an example.

Example 9 Functional dependencies in the reconstructed state-space

In the four dimensional reconstructed state space of the blood oxygen concentration signal (Table 5.1), we would like to investigate the mutual information contents of Table 5.2.

Prom an information theoretic point of view, the more state components we compare with a given time derivative, the more information we would gather. For example, the mutual information between i\ and 21,2:2 would be greater than or equal to the mutual information between x\ and x\ . Therefore, for each xk, the last line of the Table 5.2 would be the largest entry. Of course, this would depend on the choice of time delay r we use to reconstruct the state-space. If we plot this dependence (Figure 5.17), the mutual information contents of all time derivatives behave similarly, making a peak around r = 5 sec, and a dip around r = 22 sec. For modeling purposes, this plot suggests the use of a time delay of r = 5 sec.

For this choice of time delay, we can investigate how information are gathered about the time derivatives by filling out the mutual information table (Table 5.2) and observing the information provided by various subsets of coordinates. At this point, we again resort to our judgement about the system, and tailor the functional dependencies of the coordinates guided by our knowledge about the system and educated guess. We are interested in

Vi « {-yi-i+yi+i)/2At Vi+i ~ {Vi-1 - % + 3yi+1)/2At .

Table 5.1. The mutual information contents to be computed for the four dimensional reconstructed state space of the blood oxygen concentration signal, with k = 1,2,3,4. The mutual information content between the time derivative of each xk and the entries in the right-hand column are computed.

 Xk X\