\chapter{Results}
\label{chp:results}
\section{EMG}
\subsection{Classification}
In figure~\ref{fig:overviewEMG} the different settings for classification based on EMG-data are shown. Default has values as in \ref{mat:default}. The runs with pause leave out the data 1 second before the movement begins (cf. \ref{mat:pause}).
\begin{figure}[p]
\centering
\includegraphics[width=0.9\textwidth]{pictures/results/overviewEMGclass.png}
\caption{Classification with EMG-data in different configurations: with a 1s pause (top left) and in default configuration (top right) classifying into 5 classes and in default (bottom left) and pause (bottom right) configuration classifying Move and Rest only}
\label{fig:overviewEMG}
\end{figure}
When calculating an Analysis of Variance (ANOVA) on the data with and without pause we get $p<0.001$.
\subsubsection{Confusion Matrix}
A confusion matrix shows whether there is a systematic error in the classification.\\
Figure \ref{fig:cmEMG} shows the confusion matrix for EMG data. Since EMG works well for classifying Move/Rest there is also one where only the decision which movement is present is shown. In the second plot we see that many movements, especially those belonging to class 2, are classified as class 3.
\begin{figure}[p]
\centering
\includegraphics[width=\textwidth]{pictures/results/cmEMGfull.png}
\includegraphics[width=\textwidth]{pictures/results/cmEMGmovements.png}
\caption{Confusion Matrices in default configuration for EMG showing all classes (top) and only the movement classes (bottom). We see a strong bias in direction of classification as class 3}
\label{fig:cmEMG}
\end{figure}
\subsection{Regression}
Whether an offset is used or not does not make any difference since the offset is only applied to EEG-data (cf. \ref{mat:offset}).\\
Predicting synergies from EMG does not make sense since they are computed from EMG (cf. \ref{mat:synergies}).\\
Predictions of velocities and positions are quite bad from EMG.
The prediction of the $y$-dimension is slightly better than $x$ ($p<0.05$) for velocities. For positions there is no significant difference ($p>0.1$). Predicting $\theta$ is worse significantly ($p<0.001$) for positions and velocities (also see tables \ref{tab:corrKin} and \ref{tab:corrPos}).
There is no significant effect of the use of a pause when predicting velocities from EMG ($p>0.1$).
\section{EEG}
\subsection{Classification}
Figure~\ref{fig:overviewEEG} shows the different settings for classification based on EEG-data. Default has values as in \ref{mat:default}. The runs with pause leave out the data 1 second before the movement begins (cf. \ref{mat:pause}). Runs with offset have an offset of 1 or 2 (cf. \ref{mat:offset}).
\begin{figure}[p]
\centering
\includegraphics[width=\textwidth]{pictures/results/overviewEEGclass.png}
\caption{Classification with EEG-data in different configurations: default (top left), 1s pause (top, 2nd from left), pause and offset of 1 (top, 3rd from left), offset of 1 only (top right), offset of 2 (bottom left), default but classifying Move and Rest only (bottom mid) and classifying Move and Rest with 1s Pause (bottom right)}
\label{fig:overviewEEG}
\end{figure}
\subsubsection{Confusion Matrix}
Figure \ref{fig:cmEEGFull} shows the confusion matrix for EEG. It shows a main diagonal with relatively high values, the right class is chosen more often than other classes.
\begin{figure}[p]
\centering
\includegraphics[width=\textwidth]{pictures/results/cmEEGfull.png}
\caption{Confusion Matrix in default configuration for EEG, right classification is more likely than any other class}
\label{fig:cmEEGFull}
\end{figure}
\subsection{Regression}
When predicting velocities from EEG we get mean correlations of $(0.18,0.20,0.01)$, for positions we get $(0.57,0.56,0.50)$.
\subsubsection{Offset}
\label{res:offsetEEG}
Offset makes no significant difference when predicting Synergies (Autoencoder: $p>0.1$, PCA: $p>0.1$, NMF: $p>0.1$) or velocities ($p>0.1$) or positions ($p>0.1$).
\subsubsection{Pause}
Whether there is a pause of 1s or only 0.5s makes no significant difference for Autoencoder ($p>0.1$), PCA ($p>0.1$), NMF ($p>0.1$) or Velocities ($p>0.1$).
\subsubsection{EMG}
For comparison also EMG was predicted from EEG. The results are shown in figure \ref{fig:EEGemg}. There are no significant differences between the channels ($p>0.1$).
\begin{figure}[p]
\centering
\includegraphics[width=\textwidth]{pictures/results/EEGemg.png}
\caption{Prediction of the 6 EMG-channels from 32 EEG-channels; the means of the channels are not significantly different from each other}
\label{fig:EEGemg}
\end{figure}
\section{Low Frequencies}
\subsection{Classification}
In figure~\ref{fig:overviewLF} the different settings for classification based on LowFrequency(LF)-data are shown. Default has values as in \ref{mat:default}. The runs with pause leave out the data 1 second before the movement begins (cf. \ref{mat:pause}). Runs with offset have an offset of 1 or 2 (cf. \ref{mat:offset}).
\begin{figure}[p]
\centering
\includegraphics[width=\textwidth]{pictures/results/overviewLFclass.png}
\caption{Classification with LF-data in different configurations: 1s pause (top left), default (top, 2nd from left), pause and offset of 1 (top, 3rd from left), offset of 1 only (top right), offset of 2 (bottom left), default but classifying Move and Rest only (bottom mid) and classifying Move and Rest with 1s Pause (bottom right)}
\label{fig:overviewLF}
\end{figure}
\subsubsection{Confusion Matrix}
Figure \ref{fig:cmLFFull} shows the confusion matrix for LF. It shows a main diagonal with relatively high values, the right class is chosen more often than other classes. However there are also quite high values for Rest as class.
\begin{figure}[p]
\centering
\includegraphics[width=\textwidth]{pictures/results/cmLFfull.png}
\caption{Confusion Matrix in default configuration for Low Frequencies; discrimination between move and rest is problematic, which movement is done is classified quite good}
\label{fig:cmLFFull}
\end{figure}
\subsection{Regression}
When predicting velocities from Low Frequencies we get mean correlations of $(0.04,0.07,-0.01)$, for positions we get $(0.27,0.26,0.20)$.
\subsubsection{Offset}
\label{res:offsetLF}
Offset makes no significant difference for predicting Autoencoder ($p>0.1$), PCA ($p>0.1$), NMF ($p>0.1$), velocities ($p>0.1$) or position ($p>0.1$).
\subsubsection{Pause}
There is no effect of pause for velocities from low frequencies ($p>0.1$).\\
However there is an effect for Autoencoder ($p<0.001$), PCA ($p<0.001$) and NMF ($p<0.001$).
The plot shows a better performance with a shorter pause and more data taken in (see figure~\ref{fig:lfToAutoencPause})
\begin{figure}[p]
\centering
\includegraphics[width=\textwidth]{pictures/results/lfToAutoencPause.png}
\caption{Autoencoder data predicted from Low Frequencies without (left) and with pause (right)}
\label{fig:lfToAutoencPause}
\end{figure}
\section{Comparison of methods of recording}
\subsection{Classification}
The different methods of recording (EEG, EMG and Low frequencies) also differ in the results. An ANOVA gives $p<0.001$ for all classifications done on 4 different movements and rest.
\begin{figure}[p]
\centering
\includegraphics[width=\textwidth]{pictures/results/classEEGemgLF.png}
\caption{EEG, EMG and LF compared based on classification accuracy with 5 classes}
\label{fig:classEEGemgLF}
\end{figure}
The mean classification accuracies for the default run are are given in Table~\ref{tab:accs}.
\begin{table}
\centering
\begin{math}
\begin{array}
{r||c|c|c|c}
&\text{EMG}&\text{EEG}&\text{LF}&\text{chance}\\\hline
mean&60.4&40.4&32.7&20\\
std&7.97&2.27&3.35\\
max&71.9&46.7&43.4\\
min&35.7&37.2&26.2
\end{array}
\end{math}
\caption{Accuracies in \% for the different methods of recording in default configuration}
\label{tab:accs}
\end{table}
\subsection{Regression}
\subsubsection{Velocities}
Predicting velocities from EEG, EMG and Low Frequencies is significantly($p<0.001$) pairwise different (cf. figure~\ref{fig:corrEEGemgLF}). The corresponding $p$-Values of the ANOVA are given in table~\ref{tab:pCorr}.\\
The over all performance is given in table \ref{tab:corrKin}.
\begin{figure}
\centering
\includegraphics[width=\textwidth]{pictures/results/corrEEGemgLF.png}
\caption{Correlations of EEG, EMG and LF compared based on prediction of velocities}
\label{fig:corrEEGemgLF}
\end{figure}
\begin{table}
\centering
\begin{math}
\begin{array}
{r||c|c|c}
&EEG&EMG&LF\\\hline
EEG&-&<0.001&<0.001\\
EMG&<0.001&-&<0.001\\
LF&<0.001&<0.001&-
\end{array}
\end{math}
\caption{$p$-Values for prediction of velocities from EEG, EMG or LF respectively}
\label{tab:pCorr}
\end{table}
\begin{table}
\centering
\begin{math}
\begin{array}
{r||c|c|c|c}
&\text{EMG}&\text{EEG}&\text{LF}\\\hline
mean&(0.06,0.08,0.02)&(0.18,0.20,0.01)&(0.04,0.07,-0.01)\\
std&(0.05,0.05,0.02)&(0.15,0.13,0.09)&(0.05,0.05,0.04)\\
max&(0.19,0.17,0.11)&(0.49,0.49,0.21)&(0.16,0.17,0.10)\\
min&(-0.007,0.003,-0.01)&(-0.06,-0.03,-0.18)&(-0.07,-0.02,-0.08)
\end{array}
\end{math}
\caption{Correlations for the different methods of recording in default configuration predicting velocities}
\label{tab:corrKin}
\end{table}
\subsubsection{Positions}
Predicting positions from EEG, EMG and Low Frequencies is significantly($p<0.001$) different, however not pairwise (cf. figure~\ref{fig:corrEEGemgLFpos}). Positions predicted from EMG and LF are not significantly different. The corresponding $p$-Values of the ANOVA are given in table~\ref{tab:pCorrPos}.\\
The over all performance is given in table \ref{tab:corrPos}.
\begin{figure}
\centering
\includegraphics[width=\textwidth]{pictures/results/corrEEGemgLFpos.png}
\caption{Correlations of EEG, EMG and LF compared based on prediction of positions}
\label{fig:corrEEGemgLFpos}
\end{figure}
\begin{table}
\centering
\begin{math}
\begin{array}
{r||c|c|c}
&EEG&EMG&LF\\\hline
EEG&-&<0.001&<0.001\\
EMG&<0.001&-&0.34\\
LF&<0.001&0.34&-
\end{array}
\end{math}
\caption{$p$-Values for prediction of positions from EEG, EMG or LF respectively}
\label{tab:pCorrPos}
\end{table}
\begin{table}
\centering
\begin{math}
\begin{array}
{r||c|c|c|c}
&\text{EMG}&\text{EEG}&\text{LF}\\\hline
mean&(0.27,0.30,0.15)&(0.57,0.56,0.50)&(0.27,0.26,0.20)\\
std&(0.14,0.16,0.09)&(0.13,0.13,0.13)&(0.09,0.07,0.08)\\
max&(0.52,0.56,0.42)&(0.79,0.75,0.84)&(0.52,0.43,0.44)\\
min&(0.07,0.05,0.04)&(0.19,0.28,0.24)&(0.13,0.14,0.10)
\end{array}
\end{math}
\caption{Correlations for the different methods of recording in default configuration predicting positions}
\label{tab:corrPos}
\end{table}
\section{Cross-validation Parameters}
\subsection{SVM}
\label{res:maxC}
With a cross validation the results for the soft-margin parameter are compared for $\lambda=0.1,1,10$. The results are shown in table~\ref{fig:svmCV}.\\
There are no clear preferences for EEG and LF, EMG however seems to prefer a lower parameter $c$. Since this is no clear preference the parameter $c$ of the SVM does not seem to have a great influence.
\begin{table}
\centering
\begin{math}
\begin{array}
{r||c|c|c}
c&0.1 & 1 & 10\\\hline
EEG & 158 & 165 & 187\\
EMG &200 &147 & 163\\
LF& 166 & 174 & 170\\
EEGmovements &168 & 159 & 183 \\
EMGmovements &213 & 164 & 133\\
LFmovements& 150 & 187 & 173
\end{array}
\end{math}
\caption{Number of sessions in which the according $c$ was chosen as best parameter when computing an SVM for EEG, EMG or LF data; either for classification into move/rest or into 5 classes}
\label{fig:svmCV}
\end{table}
\subsection{RIDGE-Regression}
In tables \ref{tab:ridgeParamEMGkin}, \ref{tab:ridgeParamHighKin} and \ref{tab:ridgeParamAO6Kin} we find the number of 'wins' for each parameter\footnote{\ref{tab:ridgeParamHighKin} and \ref{tab:ridgeParamAO6Kin} were calculated with a order for Burg's method of 50 instead of the later default of 250}. A 'win' refers to a run where this $\lambda$ scored the highest correlation.\\
For EMG there is no clear preference but it seems like 100 should work as parameter. For EEG we see a clear preference for $\lambda=100$. Low Frequencies seem to prefer a lower parameter of about 10 however this was only evaluated for one session.
For all other runs $\lambda = 100$ is used for all methods, better results might be possible with a better adapted parameter.
\begin{table}
\centering
\begin{math}
\begin{array}
{r||c|c|c|c|c}
\lambda&0.1 & 1 & 10 & 100 & 1000\\\hline
EMG&324 & 314 & 312 & 314 & 266
\end{array}
\end{math}
\caption{Number of sessions in which the according $\lambda$ was chosen as best parameter when doing ridge regression to predict velocities from EMG}
\label{tab:ridgeParamEMGkin}
\end{table}
\begin{table}
\centering
\begin{math}
\begin{array}
{r||c|c|c|c}
\lambda&100 & 1000 & 10000 & 100000\\\hline
EEG & 1334 & 192 & 4 &0\\
EMG & 794 & 468 & 159 & 109\\
LF& 1396 & 71 & 24 & 39
\end{array}
\end{math}
\caption{Number of sessions in which the according $\lambda$ was chosen as best parameter when doing ridge regression to predict velocities from EEG, EMG or LF}
\label{tab:ridgeParamHighKin}
\end{table}
\begin{table}
\centering
\begin{math}
\hfill\begin{array}
{r||c|c|c|c}
\lambda&0.001 & 0.01 & 0.1 & 1 \\\hline
EEG & 0&0&0& 30\\
\end{array}\hfill
\begin{array}
{r||c|c|c|c}
\lambda& 1 & 5 & 10 & 100\\\hline
EEG & 0 & 0 & 1 & 29\\
EMG & 7 & 9 & 10 & 4\\
LF& 1 & 13 & 14 & 2
\end{array}\hfill
\end{math}
\caption{Number of sessions in which the according $\lambda$ was chosen as best parameter when doing ridge regression to predict velocities from EEG, EMG or LF (run on one session with one subject (subject AO, session 6) only)\\
Low ($\lambda\le 1$) values were only tested for EEG in a separate run}
\label{tab:ridgeParamAO6Kin}
\end{table}
\section{Synergies}
\subsection{Number of Synergies}
\label{res:noSyn}
To determine the number of synergies to use, all EMG data is predicted with each technique and each number of synergies from itself. The result is the plot in figure~\ref{fig:noSyn}.\\
The plot shows that 2 and 4 synergies are good values for Autoencoders, for default, nevertheless, 3 synergies are used here because there are also 3 dimensions of kinematics and so it is more comparable. 3 is also the most efficient number of Synergies for PCA and NNMF (cf. Section \ref{dis:noSyn}).\\
\begin{figure}
\centering
\includegraphics[width=\textwidth]{pictures/results/noSyn.png}
\caption{Self prediction accuracy of EMG with 1 to 6 synergies. Each channel of EMG and the mean performance is shown. We see a lowering of the slope at 2 and 4 synergies for Autoencoders and at 3 synergies for PCA and NMF}
\label{fig:noSyn}
\end{figure}
When comparing the results of prediction via different numbers of synergies, 2 synergies perform significantly ($p<0.01$) worse than 3 and 4. Between 3 and 4 synergies there is no significant difference ($p>0.1$).\\
For each method of synergy generation alone, the performance of 2 synergies is not significantly ($p>0.05$) worse. Only the over-all performance with more data becomes significant.
\subsection{Autoencoder}
In table~\ref{tab:corrAutoenc} the correlations for velocities and positions predicted from Autoencoder are given. The data for the Autoencoder were calculated from recorded EMG data.
\begin{table}
\centering
\begin{math}
\begin{array}
{r||c|c|c|c}
&\text{velocities}&\text{positions}\\\hline
mean&(0.05,0.08,0.01)&(0.20,0.29,0.11)\\
std&(0.04,0.05,0.02)&(0.12,0.16,0.08)\\
max&(0.18,0.17,0.08)&(0.51,0.60,0.38)\\
min&(-0.02,-0.01,-0.01)&(0.03,0.03,0.02)
\end{array}
\end{math}
\caption{Correlations for predicting velocities and positions from Autoencoder data}
\label{tab:corrAutoenc}
\end{table}
\subsubsection{Comparison with EMG}
When compared to the original 6D EMG data as a predictor a 3D autoencoder is only significantly worse at predicting positions ($p<0.05$), not for velocities ($p>0.1$).
\begin{figure}[p]
\includegraphics[width=\textwidth]{pictures/results/EMGautoencPos.png}
\caption{Predicting positions from EMG (left) or Autoencoder (right); EMG with 6D data works slightly better}
\label{fig:EMGautoencPos}
\end{figure}
\subsection{Compare Prediction direct and via Synergies}
\label{res:differentSynergiesVia}
\subsubsection{Velocities}
There is a significant ($p<0.001$) difference between the predictions. The different synergies however have no significant difference ($p>0.1$). Also see figure~\ref{fig:directVia}.
\begin{figure}[p]
\centering
\includegraphics[width=\textwidth]{pictures/results/predictKinfromEEG.png}
\caption{Correlations for movement-velocities predicted from EEG directly or via Synergies; direct prediction works slightly better}
\label{fig:directVia}
\end{figure}
\subsubsection{Positions}
There is a significant ($p<0.001$) difference between the predictions. The different synergies however have no significant difference ($p>0.1$). Also see figure~\ref{fig:directViaPos}.
\begin{figure}[p]
\centering
\includegraphics[width=\textwidth]{pictures/results/predictPosfromEEG.png}
\caption{Correlations for positions predicted from EEG directly or via Synergies; direct prediction works better}
\label{fig:directViaPos}
\end{figure}
\subsubsection{EMG}
There is a significant difference between predicting EMG from EEG directly or via Autoencoders ($p<0.001$, see figure~\ref{fig:directViaEMG}). The prediction via Autoencoders performs slightly worse (mean correlation $0.23$ (EMG) vs. $0.20$ (Autoencoder)).
\begin{figure}[p]
\centering
\includegraphics[width=\textwidth]{pictures/results/predictEMGfromEEG.png}
\caption{Correlations for EMG predicted from EEG directly or via Autoencoder; direct prediction works slightly better}
\label{fig:directViaEMG}
\end{figure}
\subsection{Different Synergies}
\subsubsection{Prediction via Synergies}
When predicting via synergies there is no significant difference between Autoencoder, PCA and NMF data ($p>0.1$).
\subsubsection{Prediction from EEG}
Autoencoder data can be predicted better from EEG than EMG from EEG ($p<0.05$). PCA shows no significant difference ($p>0.05$). NMF data also can be predicted better ($p<0.01$).\\
An overview is shown in figure~\ref{fig:predictEMGSyn}.
\begin{figure}[p]
\includegraphics[width=\textwidth]{pictures/results/predictEMGSyn.png}
\caption{Predicting EMG or Synergies from EEG}
\label{fig:predictEMGSyn}
\end{figure}
\section{Topographical plots}
\label{res:topo}
In figure \ref{fig:topoAlpha} we see the difference between move and rest in the alpha band, in \ref{fig:topoBeta} beta band (13-20Hz) is displayed.\\
Values greater 0 stand for more activity when moving, negative values mean less activity. A value of e.g. 0.15 stands for $15\%$ higher activity when moving.
In figure \ref{fig:topoAlpha24} the difference between two reaching movements (class 2 and 4) is shown. Here positive values stand for higher activation for class 2 and vice versa.
\begin{figure}
\centering
\includegraphics[height=0.4\textheight]{pictures/results/topoAlpha.png}
\caption{Topographical plot of alpha band (7-13 Hz) of the difference between movement and rest for subject FS in the 3rd session}
\label{fig:topoAlpha}
\end{figure}
\begin{figure}
\centering
\includegraphics[height=0.4\textheight]{pictures/results/topoBeta.png}
\caption{Topographical plot of beta band (13-20 Hz) of the difference between movement and rest for subject FS in the 3rd session}
\label{fig:topoBeta}
\end{figure}
\begin{figure}
\centering
\includegraphics[height=0.4\textheight]{pictures/results/topoAlpha24.png}
\caption{Differences in activity comparing class 2 and 4 in alpha/mu band for subject FS in session 3. Positive values stand for higher activation in class 2.}
\label{fig:topoAlpha24}
\end{figure}
Jan-Peter Hohloch