diff --git a/text/thesis/02MaterialsAndMethods.tex b/text/thesis/02MaterialsAndMethods.tex index 33b2c78..cc549d7 100644 --- a/text/thesis/02MaterialsAndMethods.tex +++ b/text/thesis/02MaterialsAndMethods.tex @@ -36,7 +36,7 @@ \subsubsection{Burg's method} \label{mat:burg} Burg's method (\cite{Burg75}) is a special case of parametric PSD estimation. It interprets the Yule-Walker-Equations as least squares problem and iteratively estimates solutions.\\ - According to \cite{Huang14} Burg's method fits well in cases with the need of high resolution. %TODO? + According to \cite{Huang14} Burg's method fits well in cases with the need of high resolution. %TODO: Autoregressive model vs. fft \subsection{Low Frequencies} In the 2000s there began a movement using new techniques to record ultrafast and infraslow brainwaves (above 50Hz and below 1Hz). These were found to have some importance (cf. \cite{Vanhatalo04}).\\ Also in predicting movements there was found some significance in low frequency as was done by Liu et al. (\cite{Liu11}) and Antelis et al. (\cite{Antelis13}) for example. Antelis et al. found correlations between hand movement and low frequency signal of $(0.29,0.15,0.37)$ in the dimensions respectively.\\ @@ -49,45 +49,51 @@ \subsection{EMG} When using muscles they are contracted after an signal via an efferent nerve activates them. Contraction of muscles also releases measurable energy which is used for Electromyography (EMG). There are intramuscular applications of EMG but we only used surface EMG.\\ From surface EMG activity of muscles can be estimated however not very precisely without repetition. Since the muscles used for arm movements are quite large in our setting EMG allows relatively precise estimations of underlying muscle activity. - %TODO Use in science? + + EMG is mainly developed for diagnostic tasks. However it is also applicable in science to track muscle activity. %TODO? \subsection{Synergies} Movement of the arm (and other parts of the body) are under-determined meaning with given trajectory there are different muscle contractions possible. One idea how this problem could be solved by our nervous system are synergies. Proposed by Bernstein in 1967 (\cite{Bernstein67}) they describe the goal of the movement (e.g. the trajectory) instead of controlling single muscles. This would mean however that predicting the activity of single muscles from EEG is harder than predicting a synergy which in turn determines the contraction of muscles.\\ Evidence for the use of synergies in the nervous system was found e.g. by Bizzi et al. (\cite{Bizzi08}) and Byadarhaly et al. (\cite{Byadarhaly12}). They also showed that synergies meet the necessary requirement to be able to build predictable trajectories.\\ Synergies are usually gotten from EMG signal through a principal component analysis (PCA, cf. \ref{mat:pca}), non-negative matrix factorisation (NMF, cf. \ref{mat:nmf}) or autoencoders (a form of neuronal network, cf. \ref{mat:autoenc}). + \subsection{PCA} + \label{mat:pca} + Principal Component Analysis (PCA) is probably the most common technique for dimensionality reduction. The idea is to use those dimensions with the highest variance to keep as much information as possible in the lower dimensional room.\\ + \begin{figure} + \centering + \includegraphics[width=0.6\textwidth]{GaussianScatterPCA.jpg} + \caption{Eigenvectors of Gaussian scatter} + \label{fig:pca} + \end{figure} + \subsection{NMF} + \label{mat:nmf} + In some applications Non-negative Matrix Factorization (NMF) is preferred over PCA (cf. \cite{Lee99}). This is because it does not learn eigenvectors but decomposes the input into parts which are all possibly used in the input. When seen as matrix factorisation PCA yields matrices of arbitrary sign where one represents the eigenvectors the other the specific mixture of them. Because an entry may be negative cancellation is possible. This leads to unintuitive representation in the first matrix.\\ + NMF in contrast only allows positive entries. This leads to \qq{what is in, is in} meaning no cancellation which in turn yields more intuitive matrices. The first contains possible parts of the data, the second how strongly they are represented in the current input.\\ + The formula for NMF is + $$Input\approx \mathbf{WH}$$ + where Input is $n\times m$, $W$ is $n\times r$ and $H$ is $r\times m$ with $r<<\min\{m,n\}$. So $\mathbf{WH}$ is only an approximation of the input however with significant lower dimension - the number of Synergies used.\\ %TODO: formulation + The factorisation is learnt with an update rule that my be chosen. We used the \matlab{} default, an alternating least squares (ALS) algorithm. It can be described as in algorithm \ref{alg:als} (cf. \cite{Berry07}).\\ + \begin{algorithm} + \begin{algorithmic} + \State initialize $\mathbf{W}$ randomly + \State $i\gets0$ + \While{$i <$ numberOfIterations} + \State Solve $\mathbf{W^TWH=W^TA}$ for $\mathbf{H}$ + \State set all negative values in $H$ to zero + \State Solve $\mathbf{HH^TW^T=HA^T}$ for $\mathbf{W}$ + \State set all negative values in $\mathbf{W}$ to zero + \State $i\gets i+1$ + \EndWhile + \Return $\mathbf{W}$ + \end{algorithmic} + \caption{Alternating Least Squares in NMF} + \label{alg:als} + \end{algorithm} + This version uses some simplifications (as setting to zero to be non-negative) and an slightly improved form is used in \matlab{}.\\ + ALS usually converges faster and with an better result than multiplicative update algorithms which would be the alternative in \matlab{}. \subsection{Autoencoders} \label{mat:autoenc} Autoencoders are a specific type of artificial neural networks (ANN). They work like typical ANNs by adjusting weights between the layers however they don't predict an unknown output but they predict their own input. What is interesting now is manipulating the size of the hidden layer where the size of the hidden layer has to be smaller than the input size. Now in the hidden layer the information of the input can be found in a condensed form (e.g. synergies instead of single muscle activity).\\ Autoencoders have been successfully used by Spüler et al. to extract synergies from EMG (\cite{Spueler16}). Especially with a lower number of synergies autoencoders perform better than PCA or NMF because linear models fail to discover the agonist-antagonist relations that are typical for muscle movements. These however can be detected by autoencoders which allows for good estimations with half the synergies. - \subsection{PCA} - \label{mat:pca} - \subsection{NMF} - \label{mat:nmf} - In some applications Non-negative Matrix Factorization (NMF) is preferred over PCA (cf. \cite{Lee99}). This is because it does not learn eigenvectors but decomposes the input into parts which are all possibly used in the input. When seen as matrix factorisation PCA yields matrices of arbitrary sign where one represents the eigenvectors the other the specific mixture of them. Because an entry may be negative cancellation is possible. This leads to unintuitive representation in the first matrix.\\ - NMF in contrast only allows positive entries. This leads to \qq{what is in, is in} meaning no cancellation which in turn yields more intuitive matrices. The first contains possible parts of the data, the second how strongly they are represented in the current input.\\ - The formula for NMF is - $$Input\approx \mathbf{WH}$$ - where Input is $n\times m$, $W$ is $n\times r$ and $H$ is $r\times m$ with $r<<\min\{m,n\}$. So $\mathbf{WH}$ is only an approximation of the input however with significant lower dimension - the number of Synergies used.\\ %TODO: formulation - The factorisation is learnt with an update rule that my be chosen. We used the \matlab{} default, an alternating least squares (ALS) algorithm. It can be described as in algorithm \ref{alg:als} (cf. \cite{Berry07}).\\ - \begin{algorithm} - \begin{algorithmic} - \State initialize $\mathbf{W}$ randomly - \State $i\gets0$ - \While{$i <$ numberOfIterations} - \State Solve $\mathbf{W^TWH=W^TA}$ for $\mathbf{H}$ - \State set all negative values in $H$ to zero - \State Solve $\mathbf{HH^TW^T=HA^T}$ for $\mathbf{W}$ - \State set all negative values in $\mathbf{W}$ to zero - \State $i\gets i+1$ - \EndWhile - \Return $\mathbf{W}$ - \end{algorithmic} - \caption{Alternating Least Squares in NMF} - \label{alg:als} - \end{algorithm} - This version uses some simplifications (as setting to zero to be non-negative) and an slightly improved form is used in \matlab{}.\\ - ALS usually converges faster and with an better result than multiplicative update algorithms which would be the alternative in \matlab{}. - - \section{Experimental design} The data used for this work were mainly recorded by Farid Shiman, Nerea Irastorza-Landa, and Andrea Sarasola-Sanz for their work (\cite{Shiman15},\cite{Sarasola15}). We were allowed to use them for further analysis.\\ There were 9 right-handed subjects%TODO diff --git a/text/thesis/GaussianScatterPCA.jpg b/text/thesis/GaussianScatterPCA.jpg new file mode 100644 index 0000000..d0455eb --- /dev/null +++ b/text/thesis/GaussianScatterPCA.jpg Binary files differ diff --git a/text/thesis/pca.py b/text/thesis/pca.py new file mode 100644 index 0000000..7ab0948 --- /dev/null +++ b/text/thesis/pca.py @@ -0,0 +1,32 @@ +import numpy as np +import matplotlib.pyplot as plt +from matplotlib import rcParams + +rcParams['font.family'] = 'serif' +rcParams['font.size'] = 16 + + +ang = np.pi/6 +mean = np.array([1, 3]) +cov = np.array([[9, 0], + [0, 1]]) +rot = np.array([[np.cos(ang), -np.sin(ang)], + [np.sin(ang), np.cos(ang)]]) +cov = np.dot(rot, np.dot(cov, rot.T)) + +np.random.seed(seed=1) +data = np.random.multivariate_normal(mean, cov, size=(5000)) +x = data[:, 0] +y = data[:, 1] + +plt.figure(figsize=(8, 8)) +plt.scatter(x, y, c="k", alpha=0.5, lw=0) +plt.arrow(1, 3, 3*np.cos(ang), 3*np.sin(ang), width=0.02, color="r", lw=2, + overhang=0.1) +plt.arrow(1, 3, -np.sin(ang), np.cos(ang), width=0.02, color="r", lw=2, + overhang=0.1) +plt.grid(True) +plt.axis("image") +plt.axis([-8, 10, -6, 12]) +plt.savefig("GaussianScatterPCA.jpg") +plt.show()