Choice of a Proper Embedding Dimension for a Noise-Free Time Series of Finite Length

As a first step we choose some $n$. We want to embed $M$ in ${\bf R}^n$. In general $m$ will be unknown so that it is not obvious how to meet Takens' condition $n=2m+1$. But if we choose $n$ large enough to ensure that $n\geq2m+1$ then we can embed $M$ in ${\bf R}^n$, according to Takens' theorem 2, because it is a well known fact that every Euclidean space ${\bf R}^{2m+1}$ can again be embedded in any higher-dimensional Euclidean space ${\bf R}^n$ without any problems. So we start by making an ansatz for $n$ and our concern for the next few paragraphs will be to find a more appropriate $n'\leq n$ such that ${\bf R}^{n'}$ is a space containing Takens' embedding space. Thus $n'$ will be a better embedding dimension than the $n$ we started with, since it gives a lower-dimensional embedding space. We want to use the method of delays to construct vectors in ${\bf R}^n$ from the $v_i$ of the time series. Doing this one realizes that there is even one more quantity which is not yet specified: One could, for example, take the series of $n$-vectors

$$\quad x_i = \left(v_i,v_{i+J},\ldots,v_{i+(n-1)J}\right)^T, \quad i=0,1,2,\ldots \quad,$$ (27)

which we are obviously allowed to use in accordance with Takens' statements. So we also have to choose the ``lag time'' $\tau_L=J\tau$, where $J\in{\bf N}_+$. We will see in section 3.4.2 that we do not have to spend much effort on choosing $J$: when we use the singular systems technique the influence of the lag time becomes insignificant, hence we will choose it from now on to equal $\tau$ (i.e. $J=1$). Consider a sequence of $N$ vectors $x_i\in{\bf R}^n$, $i=1,2,\ldots,N$ (i.e. we take a time series containing $N+n-1$ data points)¹². There seems to be no analytical way to compute the proper (i.e. minimal) embedding dimension from the time series. However, it is possible to determine a reasonable estimate for it: For some given $n$ the $n$-vectors $x_i$ usually do not explore the whole space ${\bf R}^n$. Rather than that they are restricted to some subspace $T$ of ${\bf R}^n$; T contains the embedded manifold $M$ which contains the picture of the attractor:

$$\quad T = \mbox{span}\{x_i\left\vert\right.i=1,2,\ldots,N\} \quad.$$ (28)

When we assume that the $x_i$ really visit the whole attractor in the embedding space (more or less) uniformly and we bear in mind that $N$ usually is much larger than $n$ then $n'\equiv \dim(T)$ is a sensible upper bound for the minimal embedding dimension. In order to determine $n'$ we compute the maximum number of linearly independent vectors that can be constructed as linear combinations of the $x_i$. To do this, we define the $(N,n)$-trajectory matrix $X$:

$$\quad X \equiv \frac{1}{\sqrt{N}} \left( \begin{array}{c} ... ...T \\ x_2^T \\ \vdots \\ x_N^T \end{array} \right) \quad,$$ (29)

which is built out of all vectors we want to use to reconstruct the attractor. Notice that when operating with $X^T$ on some $N$-vector we get an $n$-vector:

$$\quad X^T s = c, \quad \forall s\in{\bf R}^N: c\in{\bf R}^n \quad.$$ (30)

Since we are interested in linearly independent $n$-vectors, we choose a set of vectors $\tilde{s}_i\in{\bf R}^N$ such that the $n$-vectors

$$X^T \tilde{s}_i = c_i$$ (31)

are orthonormal. We introduce some real constants $\sigma_i\in{\bf R}$ into this equation, in order to normalize the $\tilde{s}_i$:

$$\quad X^T s_i = \sigma_i c_i \quad.$$ (32)

The important point about this equation is that, after transposing, it can be rewritten as

$$\quad s_{i1}x_1 + s_{i2}x_2 + \cdots + s_{iN}x_N = s_i^T X = \sigma_i c_i^T \quad,$$ (33)

i.e. as a linear combination of the reconstructed trajectory vectors. This tells us, when we keep in mind the definition of $n'$, that we can get $n'$ linearly independent vectors $c_i$, using eq. (34); so we have $n'$ vectors $s_i$ and $n'$ numbers $\sigma_i\neq 0$, too. The $c_1,c_2,\ldots,c_{n'}$ are elements of an orthonormal basis $\{c_i\}_{i=1}^n$ of ${\bf R}^n$. Thus we are left to determine $n'$ as the number of those $\sigma_i$ which are non-zero. Define the structure matrix $\Theta \equiv XX^T$; then it follows from eq. (34) that the $\sigma_i^2$ are the eigenvalues of this matrix:

$$\quad \Theta s_i = \sigma_i^2 s_i, \ \ \ i=1,2,\ldots,n' \quad.$$ (34)

We could determine $n'$ as the number of the non-zero eigenvalues of $\Theta$. But $\Theta$ is a huge $(N,N)$-matrix, and singular, and its diagonalization is in practice impossible. Instead, we notice that the covariance matrix $\Xi\equiv X^TX$ has the same non-zero eigenvalues as $\Theta$, and $\Xi$ is much easier to diagonalize, because it is only an $(n,n)$-matrix. So all one has to do in order to calculate $n'$ which, cum grano salis, estimates the minimal embedding dimension $(2m+1)$, is to determine the number of the non-zero eigenvalues of

$$\quad \Xi = \frac{1}{N} \left( \begin{array}{cccc} \s... ...mits_{i=1}^{N}v_{i+n-1}v_{i+n-1} \end{array} \right) \quad.$$ (35)

Now we know that the trajectory is confined to an $n'$-dimensional subspace of ${\bf R}^n$, and we can use ${\bf R}^{n'}$ as the embedding space. However, this treatment only makes sense in the case that we have noise- free data.

Footnotes

... points)¹²

We will generally assume $N\gg n$, which is obviously true in most cases.