Basic Remarks about the Choice of Embedding Dimension and Sampling Time

The first idea one might have about the choice of the sampling time $\tau$ is to make it as small as possible, such that one can reconstruct a ``smooth'' trajectory in the embedding space. However, this approach is rather short-sighted [5]: If we choose $\tau$ too small then consecutive measurements of $v$ will give nearly the same results,

$$\quad v_i\approx v_{i+1}\approx v_{i+2}\approx\ldots\approx v_{i+n-1} \quad.$$ (18)

This means that the vectors

$$\quad x_i=\left( v_i,v_{i+1},v_{i+2},\ldots,v_{i+n-1} \right)^T \quad,$$ (19)

constructed via the method of delays, will be stretched along the diagonal in the $n$-dimensional embedding space and thus the analysis of the picture of the attractor will be very difficult. To get an intuitive picture of what is happening in this case one can think of the phase space picture being artificially compressed towards the diagonal and this decreases the dimensionality of the attractor although there is no physical reason for this. In fact, numerical experiments show that for small sampling times one gets spuriously low results of dimension calculations, for example the correlation dimension tends to zero as $\tau$ approaches zero [8]. On the other hand $\tau$ must not be chosen too large, because in this case the $v_i$ become totally uncorrelated (since one of the features of a ``chaotic'' system is the exponential separation of nearby trajectories, and thus the noise which is present in every real system gives rise to total non-correlation of measurements which are made in sufficiently large time intervals). This means that the vectors $x_i$ fill (the relevant part of) the embedding space more or less homogeneously and extraction of any information from this phase space picture becomes impossible [5]. It is possible to regard this problem of $\tau$ being too large from another point of view [8]: We have seen in section 2.2 that one can also use (instead of the method of delays) a vector the components of which are the first $n$ derivatives of the observable to construct a phase space picture. Now, since we cannot measure the derivatives themselves we compute approximations to them using the time series. It is typical for such approximation formulae that the error of the approximant is ${\cal O}(\tau^k)$ with some integer $k$. (See for example footnote 2 in section 2.2.) This means that to get a sensible result for this approximation, and thus for $x_i$, we want to have $\tau$ small! We illustrate the effect of a ``bad'' choice of $\tau$ with a numerical experiment: We consider a derivate of the Lorenz system, the Rössler system (see e.g. [14], chapter 5.3), which is defined by

$$\frac{dx}{dt} = -z-y$$

$$\frac{dy}{dt} = x+ay$$ (20)

$$\frac{dz}{dt} = b+z(x-c) \quad. \vspace{-0.1cm}$$

For the parameter values $a=0.15$, $b=0.20$, $c=10$ the flow of this system becomes attracted to the Rössler attractor, the projections of which onto the $xy$- and to the $xz$-planes are shown in Fig. 5.a and Fig. 5.b, respectively. Numerical integration (with some step size $\Delta$) of eq. (22) provides us, for example, with the time series $\left\{x(i\Delta)\right\}_{i=0}^\infty$, depicted in Fig. 5.c. Using this data and several different delay times $\tau$, we can reconstruct (in 2-dimensional embedding space) the phase space pictures of the Rössler attractor which are shown in Fig. 5.d, ..., Fig. 5.h⁹. The effects we described in the above

$\parbox{12cm}{ Figure 5. (a,b) Projections of the R\uml ossler att... ...d of delays, with $\tau=10\Delta$, $50\Delta$, $140\Delta$, respectively. }$

$\parbox{12cm}{ Figure 5. (g,h) Results of the method of delays, with $\tau=235\Delta$, $2000\Delta$, respectively. }$

paragraph for $\tau$ too small (stretching along the diagonal) and $\tau$ too large (increasing non-correlation of the data) are clearly visible in Fig. 5.d and Fig. 5.h, respectively. So it is clear that a proper choice for the sampling time is a necessary requirement for the method of delays to give sensible results. A similar statement holds for the embedding dimension $n$: Obviously $n$ must be large enough to allow $M$ to be embedded within ${\bf R}^n$. The more complex the attractor is the higher $n$ must be (see footnote 7 in section 2.2). What is more, to make sure that the choice of $n$ allows Takens' theory to be applied one might be tempted to make $n$ very large in order to guarantee that $n\geq2m+1$. If $n$ is chosen too small then we change the distribution of the points on the attractor artificially and the point densitiy

$$p\left(x\right) = \lim_{N\to\infty} \frac{1}{N} \sum_{i=1}^{N} \delta\left(x_i-x\right)$$ (21)

develops singularities which are absent in an embedding space with $n$ large enough [5]. (As a simple example consider the surface of a sphere in 3 dimensions which is ``embedded'' in 2 dimensions. Even if the point density on the sphere in 3-space is constant the ``embedded'' sphere, which is essentially a projection of the sphere into the plane, will have singularities on the boundary.) If it is known a priori that the attractor is visited uniformly then one can use this observation to determine the embedding dimension: Take the smallest $n$ such that the densitity on the reconstructed attractor does not have any singularities. However, this method requires some pre-knowledge about the system and cannot be taken as a general approach to the choice of $n$. There are good arguments to keep $n$ small as well: It is easier to work with a low-dimensional embedding space, especially if one aims at building up some geometrical intuition about the attractor's shape. Also, working in higher dimensions results in larger numerical errors when one wants to use the reconstructed attractor to calculate the usual quantities which characterize the system, such as the Hausdorff or correlation dimension [5]. But the best argument not to make $n$ too large is perhabs the problem which we pointed to in section 2.2: in contrast to Takens' statement that systems with periods smaller than or equal to $(2m+1)\tau$ can be excluded on generic grounds we saw that these periods actually can appear, generally speaking. So we have to take additional precautions to prevent such periodic flows with small periods to cause trouble, since otherwise Takens' theory can not be applied! An easy solution for this problem is to make $n\tau$ small [7].

Footnotes

... 5.h⁹

Fig. 5.d, ..., Fig. 5.h consist of clouds of points, in contrast to the continuous orbit of the system shown in Fig. 5.a and Fig. 5.b. This is not an inherent problem of the method of delays, but due to the limited computing power we had available for this little experiment: By choosing a smaller step size $\Delta$ while keeping $\tau$ constant, we would have got pictures similar to Fig. 5.d, ..., Fig. 5.h, but consisting of ``quasi-continuous'' orbits.