diff --git a/text/thesis/01Introduction.tex b/text/thesis/01Introduction.tex index 7531e58..1ec573b 100644 --- a/text/thesis/01Introduction.tex +++ b/text/thesis/01Introduction.tex @@ -16,10 +16,10 @@ Assuming this it should be easier to predict synergies while we can also use them to move a robotic arm or a quadrocopter. This improvements shall be shown in this thesis. To do so different methods of the acquisition of synergies from EMG are compared with other data and paradigms like direct prediction from EEG, EMG and low frequencies. -\section{Overview}%TODO - After this Introduction in Materials and Methods (Chapter \ref{chp:mat}) we show the scientific background of the methods used in the work. These reach from Principal Component Analysis (PCA) and Autoencoders over Support Vector MAchines (SVMs) and regression to boxplots and topographical plots.\\ +\section{Overview} + After this Introduction the scientific background and context of this work will be stated (Chapter \ref{chp:background}). This reaches from Principal Component Analysis (PCA) and Autoencoders over Support Vector Machines (SVMs) and regression to boxplots and topographical plots.\\ + Material and Methods (Chapter \ref{chp:mat}) shows the work done for tis thesis, beginning with the experimental design followed by the methods for data acquisition and analysis.\\ In chapter \ref{chp:results} Results we show the numerical findings of our work separated into parts on synergies, classification, regression and a topographical analysis of the brain activity.\\ - This results and their meaning will be discussed in chapter \ref{chp:dis} Discussion.\\ - Finally we take a look in the possible future and discuss which further research could be done based on or related to our work (chapter \ref{chp:fut}). + This results and their meaning will be discussed in chapter \ref{chp:dis} Discussion, which is concluded with a look in the possible future. The appendix then contains a list of contents on the CD and in the repository (Appendix \ref{app:cd}) and a small documentation of the code used (Appendix \ref{app:docu}) diff --git a/text/thesis/02MaterialsAndMethods.tex b/text/thesis/02MaterialsAndMethods.tex index 3486eb0..9629819 100644 --- a/text/thesis/02MaterialsAndMethods.tex +++ b/text/thesis/02MaterialsAndMethods.tex @@ -1,8 +1,8 @@ %\chapter{Materials and Methods} %\label{chp:mat} \chapter{Scientific background} -\label{mat:background} -\section{Communication between Brain and Computer} +\label{chp:background} +\section{Communication between Neurons and Machines} \subsection{Brain-Computer-Interfaces} The idea of BCIs began to spread in the 1970s when Vidal published his paper (\cite{Vidal73}).\\ The connection between brain and computer allows to help the human in different ways. From implants to re-acquire hearing and sight in one direction to the commanding of machines by brainwaves or communication although having the Locked-In syndrome in the other direction a wide field of possibilities is given yet. However most applications require lots of training and are sometimes quite far from natural behavior. Binary decisions for example are usually made through an excited or relaxed mood, which can easily be detected in brain activity.\\ @@ -91,7 +91,7 @@ \begin{figure} \centering \includegraphics[width=0.7\textwidth]{GaussianScatterPCA.jpg} - \caption{Eigenvectors of Gaussian scatter} + \caption{Gaussian Scatter with both eigenvectors, the principal component (long arrow) explaining most, the other least variance} \label{fig:pca} \end{figure} In Figure~\ref{fig:pca} we see the eigenvectors of the data. The longer vector is the principal component the shorter one is orthogonal to it and explains the remaining variance. The second component here also is the component which explains least variance, since most variance is orthogonal to it. @@ -129,7 +129,7 @@ \begin{figure} \centering \input{autoencoder.tikz} - \caption{Autoencoder (6-3-6)} + \caption{Autoencoder with 6 input and 6 output neurons and a hidden layer of size 3} \label{fig:autoenc} \end{figure} \section{Machine Learning} @@ -148,7 +148,7 @@ \begin{figure} \centering \includegraphics[width=0.6\textwidth]{pictures/svm.png} - \caption{Margins and hyperplane (Figure by Cyc and Peter Buch)} + \caption{Margins and hyperplane for a SVM (Figure by Cyc and Peter Buch)} \label{fig:svm} \end{figure} This prototype of a SVM is only able to separate two classes of linear separable data. For other data some improvements are necessary. @@ -179,7 +179,7 @@ $k$-fold cross validation means splitting the data into $k$ equally sized parts, training the model on $k-1$ parts and validating on left one (see Figure~\ref{fig:crossValidation}). \begin{figure} \includegraphics[width=\textwidth]{pictures/K-fold_cross_validation_EN.jpg} - \caption{k-fold cross validation (picture by Joan.domenech91 and Fabian Flöck)} + \caption{Principle of k-fold cross validation (here $k=4$)(picture by Joan.domenech91 and Fabian Flöck)} \label{fig:crossValidation} \end{figure} This is done to achieve a measure for how good the fit is. When using cross validation all the data is used for prediction and is predicted. This eliminates effects of randomness.\\ @@ -228,10 +228,10 @@ \caption{2D confusion matrix} \label{tab:tptnftfn} \end{table} - In the higher dimensional case \matlab{} uses color coded maps as figure~\ref{fig:exampleCM}. In this thesis scaled confusion matrices where each row adds up to 1 are used. + In the higher dimensional case \matlab{} uses color coded maps as figure~\ref{fig:exampleCM}. In this thesis scaled confusion matrices are used where each row adds up to 1. \begin{figure} \includegraphics[width=\textwidth]{pictures/results/cmEEGfull.png} - \caption{Example for a confusion matrix} + \caption{Example for a color coded confusion matrix with 5 classes} \label{fig:exampleCM} \end{figure} \subsection{ANOVA} @@ -243,6 +243,7 @@ A boxplot contains information about the median (red line), 0.25 and 0.75 quantiles (ends of the box) and about the highest and lowest values that are not classified as outliers.\\ A data point $y$ is classified as outlier if $y > q_3+1.5\cdot(q_3-q_1)$ or $y < q_1-1.5\cdot(q_3-q_1)$, where $q_1,q_3$ are the first and third quartile (which are also defining the box). \chapter{Materials and Methods} +\label{chp:mat} \section{Experimental design} \label{mm:design} The data used for this work was mainly recorded by Farid Shiman, Nerea Irastorza-Landa, and Andrea Sarasola-Sanz for their work (\cite{Shiman15},\cite{Sarasola15}). I was allowed to use it for further analysis.\\ @@ -253,7 +254,7 @@ \begin{figure} \centering \includegraphics{experimentalDesign.jpg} - \caption{Center-out reaching task with four color-coded targets (picture from \citeauthor{Shiman15})} + \caption{Center-out reaching task with four color-coded targets (picture from \cite{Shiman15})} \label{fig:experimentalDesign} \end{figure} Of the kinematic information tracked only position ($x,y$) and angle ($\theta$, rotation around $z$-axis) of the hand are used.\\ @@ -269,7 +270,9 @@ The signal plus the according status data and the parameters is loaded as shown in Algorithm~\ref{alg:load_bcidat}). \begin{algorithm} \begin{algorithmic} - \State [signal, states, params] = load\_bcidat('dataFile.dat'); + \State [signal, states, params] = load\_bcidat('dataFile.dat','-calibrated'); + \State\Comment loading using microvolts + \State [signal, states, params] = load\_bcidat('dataFile.dat');\Comment integers \end{algorithmic} \caption{Usage of \texttt{load\_bcidat}} \label{alg:load_bcidat} @@ -391,42 +394,51 @@ The recording of kinematics was started after that of EEG. In synchronization channel\footnote{cf. Table~\ref{tab:channelNames}} there is a peak when kinematic recording is started. This was used to align movement with EEG and EMG data. In addition the kinematic data is adjusted to the EMG window and shift to be able to use corresponding data for the same time step. This was done by summing all differences (for movement) or by calculating the mean position in the time window.\\ Size of this data is same as EMG and Synergies in length but has only three features per time step since only 3D positioning ($x,y$ and $\theta$) of the hand and no information about the fingers are used. \section{Data Analysis} - Figure~\ref{fig:overview} shows the steps of this work. EEG, EMG and positions were recorded, Synergies and velocities were calculated from them. To check the performance of the methods the relations between them were predicted. + Figure~\ref{fig:overview} shows the regression steps of this work. EEG, EMG and positions were recorded, Synergies and velocities were calculated from them. To check the performance of the methods the relations between them were predicted. \begin{figure} \centering \input{pictures/overview.tikz} - \caption{Overview: What is predicted?} + \caption{Overview: What is predicted?\\Normal arrows indicate prediction, fat ones computation and dashed ones prediction with an intermediate step} \label{fig:overview} \end{figure} \subsection{Classification} + In addition to the regressions, classifications were done to have a benchmark and a possibility to compare with results from other work. Classification can be done in different ways. First approach is discriminating Movement from Rest. This is done by training an SVM and testing its results with 10-fold cross validation. Here this is done with EMG, EEG and LF data. EMG in this setting is trivial since it was the basis for the classification (cf. \ref{mm:newClass}).\\ - In a second step I try to discriminate movement in different directions also with an SVM trained on EMG, EEG or LF data respectively. The fit of the model is also checked with 10-fold cross validation.\\ + In a second step discrimination of movement in different directions is done, also with an SVM trained on EMG, EEG or LF data respectively. The fit of the model is also checked with 10-fold cross validation.\\ For unbiased classification it is necessary to train with equally sized classes. For that purpose and to lower computation time only 250 (as default) samples per class are taken in.\\ The parameter $c$ for the support vector machine is found with an additional step of cross validation or set to 1. (Results in Section~\ref{res:maxC}).\\ To learn about biased classifications and about similar brain activity for different movements the confusion matrix is created (cf. Section~\ref{mm:cm}).\\ The resulting accuracy is the mean of each of the 10 cross validation steps. - \subsection{Predicting Kinematics} - The prediction of kinematics is done with ridge regression. Since there are more data for kinematics than for EEG the mean position or movement are used and predicted.\\ - The regression is done in 10-fold cross validation for each dimension ($x,y,\theta$) and the parameter $\lambda$ (cf. ~\ref{mm:ridge}) is ascertained with an additional cross validation. The resulting correlation is the mean correlation of each of the 10 parts with the best parameter lambda each while the correlation for each dimension is calculated independently. - \subsection{Predicting Synergies} - Predicting synergies works similar as for the kinematics. Only change is that the synergies may have other dimensionality. Nevertheless each synergy is predicted from all EEG data as one output and correlation is calculated for each synergy. - \subsection{Predicting EMG} - When predicting EMG data the sum of the waveform length in the time corresponding to the EEG data is used. As the EMG data was summed to gain the data this is a reasonable approach.\\ - The remaining steps are the same as for kinematics and Synergies. - \subsection{EEG offset} - \label{mat:offset} - Since it takes some time for commands to go from brain to the muscles, a variable offset between EEG and other data is used. The offset has to be given in a number of shifts, so in default is a multiple of 200ms.\\ - Results are given in Sections~\ref{res:offsetEEG} and~\ref{res:offsetLF}. - \subsection{Pause} - \label{mat:pause} - A pause is used before movement onset. This pause means that 1 second before movement onset is not taken into account when analyzing the data. If there is no pause only 1s to 0.5 second before movement onset is left out and the last 0.5 seconds before movement are classified as belonging to the following task.\\ - This was necessary since the data about presentation of stimuli did not match the recordings and reclassification was necessary (cf. section \ref{mm:newClass}). - \subsection{Prediction with interim step} - All these analyses only show the accuracy of one step. To get a measure for the over-all performance synergies are predicted from EEG and used to predict EMG or kinematics respectively.\\ - The resulting correlation is the mean of the correlations of a 10-fold cross validation where the same unknown synergies are predicted from EEG and used to predict EMG or kinematics. So there is no correction step between the steps and EMG or kinematics are predicted from EEG via the Synergies. Here also different methods to determine Synergies are compared (see Section~\ref{res:differentSynergiesVia}). + \subsection{Regression} + \subsubsection{Predicting Kinematics} + The prediction of kinematics is done with ridge regression. Since there are more data for kinematics than for EEG the mean position or movement are used and predicted.\\ + The regression is done in 10-fold cross validation for each dimension ($x,y,\theta$) and the parameter $\lambda$ (cf. ~\ref{mm:ridge}) is ascertained with an additional cross validation. The resulting correlation is the mean correlation of each of the 10 parts with the best parameter lambda each while the correlation for each dimension is calculated independently. + \subsubsection{Predicting Synergies} + Predicting synergies works similar as for the kinematics. Only change is that the synergies may have other dimensionality. Nevertheless each synergy is predicted from all EEG data as one output and correlation is calculated for each synergy. + \subsubsection{Predicting EMG} + When predicting EMG data the sum of the waveform length in the time corresponding to the EEG data is used. As the EMG data was summed to gain the data this is a reasonable approach.\\ + The remaining steps are the same as for kinematics and Synergies. + \subsubsection{Prediction with interim step} + All these analyses only show the accuracy of one step. To get a measure for the over-all performance synergies are predicted from EEG and used to predict EMG or kinematics respectively.\\ + The resulting correlation is the mean of the correlations of a 10-fold cross validation where the same unknown synergies are predicted from EEG and used to predict EMG or kinematics. So there is no correction step between the steps and EMG or kinematics are predicted from EEG via the Synergies. Here also different methods to determine Synergies are compared (see Section~\ref{res:differentSynergiesVia}). + \subsection{Parameters} + Some structural parameters were introduced to check their influence on the predictions and classifications. + \subsubsection{EEG offset} + \label{mat:offset} + Since it takes some time for commands to go from brain to the muscles, a variable offset between EEG and other data is used. The offset has to be given in a number of shifts, so in default is a multiple of 200ms.\\ + Results are given in Sections~\ref{res:offsetEEG} and~\ref{res:offsetLF}. + \subsubsection{Pause} + \label{mat:pause} + A pause is used before movement onset. This pause means that 1 second before movement onset is not taken into account when analyzing the data. If there is no pause only 1s to 0.5 second before movement onset is left out and the last 0.5 seconds before movement are classified as belonging to the following task.\\ + This was necessary since the data about presentation of stimuli did not match the recordings and reclassification was necessary (cf. section \ref{mm:newClass}). \subsection{Multiple Sessions} Each session (cf. Section~\ref{mm:design}) is analyzed independently meaning there are 51 independent results for each analysis. These are used for the statistical evaluation in Chapter~\ref{chp:results}.\\ Some analyses are only done on one session - if so it will be clearly stated. + \subsection{Topographical Plots} + Sometimes the interpretation of EEG data is easier if plotted topographically, meaning visualized according to the corresponding positions on a modeled head. + + Usually differences between different classes (e.g. movement or rest) are plotted in a band and not in a single frequency. Examples are given in the Results section in figures \ref{fig:topoAlpha} and \ref{fig:topoBeta}.\\ + To have 0 centered data the relation can be calculated as $$\frac{\text{Move}}{\text{Rest}}-1,$$ where Move and Rest are the mean activity in the band. Dividing instead of subtracting provides a more intuitive measure for the strength of desynchronization. \subsection{Default values} \label{mat:default} The values of the variables used in \texttt{'Default'} are given in table~\ref{tab:default}. @@ -456,12 +468,6 @@ windowEEG & 1 & size of the EEG window \\ windowEMG & 0.2 & size of the EMG window \\ \end{tabular} - \caption{Values used for default} + \caption{Default values for all variables} \label{tab:default} \end{table} - \pagebreak - \subsection{Topographical Plots} - Sometimes the interpretation of EEG data is easier if plotted topographically, meaning visualized according to the corresponding positions on a modeled head. - - Usually differences between different classes (e.g. movement or rest) are plotted in a band and not in a single frequency. Examples are given in the Results section in figures \ref{fig:topoAlpha} and \ref{fig:topoBeta}.\\ - To have 0 centered data the relation can be calculated as $$\frac{\text{Move}}{\text{Rest}}-1,$$ where Move and Rest are the mean activity in the band. Dividing instead of subtracting provides a more intuitive measure for the strength of desynchronization. diff --git a/text/thesis/03Results.tex b/text/thesis/03Results.tex index 577776e..c00c3f9 100644 --- a/text/thesis/03Results.tex +++ b/text/thesis/03Results.tex @@ -2,15 +2,15 @@ \label{chp:results} \section{Number of Synergies} \label{res:noSyn} - To determine the number of synergies to use I predicted all EMG data with each technique and each number of synergies. The result is the plot in figure~\ref{fig:noSyn}.\\ + To determine the number of synergies to use, all EMG data is predicted with each technique and each number of synergies from itself. The result is the plot in figure~\ref{fig:noSyn}.\\ The plot tells that 2 and 4 synergies are good values for Autoencoders, for default nevertheless 3 synergies are used here because there are also 3 dimensions of kinematics and so it is more comparable. Three is also the most efficient number of Synergies for PCA and NNMF (cf. Section \ref{dis:noSyn}).\\ \begin{figure} \centering - \includegraphics[width=\textwidth,height=\textheight]{pictures/results/noSyn.png} - \caption{Self prediction accuracy with 1 to 6 synergies} + \includegraphics[width=\textwidth]{pictures/results/noSyn.png} + \caption{Self prediction accuracy of EMG with 1 to 6 synergies. Each channel of EMG and the mean performance is shown. We see a lowering of the slope at 2 and 4 synergies for Autoencoders and at 3 synergies for PCA and NMF} \label{fig:noSyn} - \end{figure}%TODO (last): check orientation of figure (bottom should be outer edge) - When comparing the results of prediction via different number of synergies, 2 synergies perform significantly ($p<0.01$) worse than 3 and 4. Between 3 and 4 synergies there is no significant difference ($p\approx0.1$).\\ + \end{figure} + When comparing the results of prediction via different number of synergies, 2 synergies perform significantly ($p<0.01$) worse than 3 and 4. Between 3 and 4 synergies there is no significant difference ($p>0.1$).\\ For each method of synergy generation alone the performance of 2 synergies is not significantly ($p>0.05$) worse. Only the over-all performance with more data becomes significant. \section{Classification} \subsection{Comparison of methods of recording} @@ -34,15 +34,15 @@ min&35.7&37.2&26.2 \end{array} \end{math} - \caption{Accuracys for the different methods of recording in default configuration} + \caption{Accuracys in \% for the different methods of recording in default configuration} \label{tab:accs} \end{table} \subsection{EMG} In figure~\ref{fig:overviewEMG} the different settings for classification based on EMG-data are shown. Default has values as in \ref{mat:default}. The runs with pause leave out the data 1 second before the movement begins (cf. \ref{mat:pause}). \begin{figure} \centering - \includegraphics[width=\textwidth]{pictures/results/overviewEMGclass.png} - \caption{Classification with EMG-data} + \includegraphics[width=0.9\textwidth]{pictures/results/overviewEMGclass.png} + \caption{Classification with EMG-data in different configurations: with a 1s pause (top left) and in default configuration (top right) classifying into 5 classes and in default (bottom left) and pause (bottom right) configuration classifying Move and Rest only} \label{fig:overviewEMG} \end{figure} When calculating an ANOVA on the data with and without pause we get $p<0.001$. @@ -51,7 +51,7 @@ \begin{figure} \centering \includegraphics[width=\textwidth]{pictures/results/overviewEEGclass.png} - \caption{Classification with EEG-data} + \caption{Classification with EEG-data in different configurations: default (top left), 1s pause (top, 2nd from left), pause and offset of 1 (top, 3rd from left), offset of 1 only (top right), offset of 2 (bottom left), default but classifying Move and Rest only (bottom mid) and classifying Move and Rest with 1s Pause (bottom right)} \label{fig:overviewEEG} \end{figure} \subsection{Low Frequencies} @@ -59,7 +59,7 @@ \begin{figure} \centering \includegraphics[width=\textwidth]{pictures/results/overviewLFclass.png} - \caption{Classification with LF-data} + \caption{Classification with LF-data in different configurations: 1s pause (top left), default (top, 2nd from left), pause and offset of 1 (top, 3rd from left), offset of 1 only (top right), offset of 2 (bottom left), default but classifying Move and Rest only (bottom mid) and classifying Move and Rest with 1s Pause (bottom right)} \label{fig:overviewLF} \end{figure} \subsection{Trade-off parameter} @@ -77,25 +77,25 @@ \centering \includegraphics[width=\textwidth]{pictures/results/cmEEGfull.png} \includegraphics[width=\textwidth]{pictures/results/cmLFfull.png} - \caption{Confusion Matrices in default configuration} + \caption{Confusion Matrices in default configuration for EEG (top) and Low Frequencies (bottom)} \label{fig:cmFull} \end{figure} \begin{figure}[p] \centering \includegraphics[width=\textwidth]{pictures/results/cmEMGfull.png} \includegraphics[width=\textwidth]{pictures/results/cmEMGmovements.png} - \caption{Confusion Matrices in default configuration} + \caption{Confusion Matrices in default configuration for EMG showing all classes (top) and only the movement classes (bottom)} \label{fig:cmEMG} \end{figure} \section{Regression} \subsection{Comparison of methods of recording} \subsubsection{Velocities} - Predicting velocities from EEG, EMG and Low Frequencies is significantly\footnote{$p<0.001$} pairwise different (cf. figure~\ref{fig:corrEEGemgLF}). The corresponding $p$-Values of the ANOVA are given in table~\ref{tab:pCorr}.\\ + Predicting velocities from EEG, EMG and Low Frequencies is significantly($p<0.001$) pairwise different (cf. figure~\ref{fig:corrEEGemgLF}). The corresponding $p$-Values of the ANOVA are given in table~\ref{tab:pCorr}.\\ The over all performance is given in table \ref{tab:corrKin}. \begin{figure} \centering \includegraphics[width=\textwidth]{pictures/results/corrEEGemgLF.png} - \caption{EEG, EMG and LF compared based on prediction of velocities} + \caption{Correlations of EEG, EMG and LF compared based on prediction of velocities} \label{fig:corrEEGemgLF} \end{figure} \begin{table} @@ -128,12 +128,12 @@ \label{tab:corrKin} \end{table} \subsubsection{Positions} - Predicting positions from EEG, EMG and Low Frequencies is significantly\footnote{$p<0.001$} different, however not pairwise (cf. figure~\ref{fig:corrEEGemgLFpos}). Positions predicted from EMG and LF are not significantly different. The corresponding $p$-Values of the ANOVA are given in table~\ref{tab:pCorrPos}.\\ + Predicting positions from EEG, EMG and Low Frequencies is significantly($p<0.001$) different, however not pairwise (cf. figure~\ref{fig:corrEEGemgLFpos}). Positions predicted from EMG and LF are not significantly different. The corresponding $p$-Values of the ANOVA are given in table~\ref{tab:pCorrPos}.\\ The over all performance is given in table \ref{tab:corrPos}. \begin{figure} \centering \includegraphics[width=\textwidth]{pictures/results/corrEEGemgLFpos.png} - \caption{EEG, EMG and LF compared based on prediction of positions} + \caption{Correlations of EEG, EMG and LF compared based on prediction of positions} \label{fig:corrEEGemgLFpos} \end{figure} \begin{table} @@ -168,71 +168,73 @@ \subsection{Compare Prediction direct and via Synergies} \label{res:differentSynergiesVia} \subsubsection{Velocities} - There is a significant\footnote{$p<0.001$} difference between the predictions. The different synergies however have no significant difference ($p\approx0.87$). Also see figure~\ref{fig:directVia}. + There is a significant($p<0.001$) difference between the predictions. The different synergies however have no significant difference ($p>0.1$). Also see figure~\ref{fig:directVia}. \begin{figure} \centering \includegraphics[width=\textwidth]{pictures/results/predictKinfromEEG.png} - \caption{Velocities predicted from EEG direct or via Synergies} + \caption{Correlations for movement-velocities predicted from EEG directly or via Synergies} \label{fig:directVia} \end{figure} \subsubsection{Positions} - There is a significant\footnote{$p<0.001$} difference between the predictions. The different synergies however have no significant difference ($p\approx0.85$). Also see figure~\ref{fig:directViaPos}. + There is a significant($p<0.001$) difference between the predictions. The different synergies however have no significant difference ($p>0.1$). Also see figure~\ref{fig:directViaPos}. \begin{figure} \centering \includegraphics[width=\textwidth]{pictures/results/predictPosfromEEG.png} - \caption{Positions predicted from EEG direct or via Synergies} + \caption{Correlations for positions predicted from EEG directly or via Synergies} \label{fig:directViaPos} \end{figure} \subsubsection{EMG} - There is a significant difference between predicting EMG from EEG directly or via Autoencoders ($p<0.001$, see figure~\ref{fig:directViaEMG}). The prediction via Autoencoders performs a bit worse (mean is about 0.03 lower). + There is a significant difference between predicting EMG from EEG directly or via Autoencoders ($p<0.001$, see figure~\ref{fig:directViaEMG}). The prediction via Autoencoders performs a bit worse (mean $r\sim 0.03$). \begin{figure} \centering \includegraphics[width=\textwidth]{pictures/results/predictEMGfromEEG.png} - \caption{EMG predicted from EEG direct or via Autoencoder} + \caption{Correlations for EMG predicted from EEG directly or via Autoencoder} \label{fig:directViaEMG} \end{figure} \subsubsection{Different Synergies} - When predicting via synergies there is no significant difference between Autoencoder, PCA and NMF data ($p>0.85$). + When predicting via synergies there is no significant difference between Autoencoder, PCA and NMF data ($p>0.1$). \subsection{EEG} \subsubsection{Offset} \label{res:offsetEEG} - Offset makes no significant difference when predicting Synergies\footnote{Autoencoder: $p\approx0.81$, PCA: $p\approx0.77$, NMF: $p\approx0.60$} or velocities ($p\approx0.99$) or positions ($p\approx0.98$). + Offset makes no significant difference when predicting Synergies (Autoencoder: $p>0.1$, PCA: $p>0.1$, NMF: $p>0.1$) or velocities ($p>0.1$) or positions ($p>0.1$). \subsubsection{Pause} - Whether there is a pause of 1s or only 0.5s doesn't make a significant difference for Autoencoder ($p\approx0.13$), PCA ($p\approx0.29$), NMF ($p\approx0.15$) or Velocities ($p\approx0.95$). + Whether there is a pause of 1s or only 0.5s doesn't make a significant difference for Autoencoder ($p>0.1$), PCA ($p>0.1$), NMF ($p>0.1$) or Velocities ($p>0.1$). \subsubsection{EMG} - For comparison also EMG was predicted from EEG. The results are shown in figure \ref{fig:EEGemg}. There are no significant differences between the channels ($p\approx0.29$). + For comparison also EMG was predicted from EEG. The results are shown in figure \ref{fig:EEGemg}. There are no significant differences between the channels ($p>0.1$). \begin{figure} \centering \includegraphics[width=\textwidth]{pictures/results/EEGemg.png} - \caption{Prediction of EMG from EEG} + \caption{Prediction of the 6 EMG-channels from 32 EEG-channels} \label{fig:EEGemg} \end{figure} \subsubsection{Synergies} - Autoencoder data can be predicted better from EEG than EMG ($p<0.05$). PCA shows no significant difference ($p\approx0.07$). NMF data also can be predicted better ($p<0.01$).\\ + Autoencoder data can be predicted better from EEG than EMG ($p<0.05$). PCA shows no significant difference ($p>0.05$). NMF data also can be predicted better ($p<0.01$).\\ An overview is shown in figure~\ref{fig:predictEMGSyn}. \begin{figure} \includegraphics[width=\textwidth]{pictures/results/predictEMGSyn.png} - \caption{Predicting EMG or Synergies from EEG} + \caption{Predicting EMG (mean over channels) or Synergies (mean) from EEG} \label{fig:predictEMGSyn} \end{figure} \subsection{EMG} Using a offset or not does not make any difference since the offset is only applied on EEG-data (cf. \ref{mat:offset}).\\ Predicting synergies from EMG does not make sense since they are computed from EMG (cf. \ref{mat:synergies}).\\ + Predictions of velocities and positions are quite bad from EMG. + The prediction of the $y$-dimension is a bit better than $x$ ($p<0.05$) for velocities. For positions there is no significant difference ($p\approx 0.31$). Predicting $\theta$ is worse significantly ($p<0.001$) for positions and velocities (also see tables \ref{tab:corrKin} and \ref{tab:corrPos}). \subsubsection{Pause} - There is no significant effect of the use of a pause when predicting velocities from EMG ($p\approx0.90$). + There is no significant effect of the use of a pause when predicting velocities from EMG ($p>0.1$). \subsection{Low Frequencies} \subsubsection{Offset} \label{res:offsetLF} - Offset makes no significant difference for predicting Autoencoder ($p\approx0.50$), PCA ($p\approx0.59$), NMF ($p\approx0.38$), velocities ($p\approx0.97$) or position ($p\approx1.0$). + Offset makes no significant difference for predicting Autoencoder ($p>0.1$), PCA ($p>0.1$), NMF ($p>0.1$), velocities ($p>0.1$) or position ($p>0.1$). \subsubsection{Pause} - There is no effect of pause for velocities from low frequencies ($p\approx0.73$).\\ + There is no effect of pause for velocities from low frequencies ($p>0.1$).\\ However there is an effect for Autoencoder ($p<0.001$), PCA ($p<0.001$) and NMF ($p<0.001$). The plot shows a better performance with a shorter pause and more data taken in (see figure~\ref{fig:lfToAutoencPause}) \begin{figure} \centering \includegraphics[width=\textwidth]{pictures/results/lfToAutoencPause.png} - \caption{Autoencoder data predicted from Low Frequencies} + \caption{Autoencoder data predicted from Low Frequencies without (left) and with pause (right)} \label{fig:lfToAutoencPause} \end{figure} \subsection{Autoencoder} @@ -253,15 +255,15 @@ \label{tab:corrAutoenc} \end{table} \subsubsection{Comparison with EMG} - When compared to the original 6D EMG data as a predictor a 3D autoencoder is only significantly worse when predicting positions ($p<0.05$), not for velocities ($p\approx0.23$). + When compared to the original 6D EMG data as a predictor a 3D autoencoder is only significantly worse when predicting positions ($p<0.05$), not for velocities ($p>0.1$). \begin{figure} \includegraphics[width=\textwidth]{pictures/results/EMGautoencPos.png} - \caption{Predicting positions from EMG or Autoencoder} + \caption{Predicting positions from EMG (left) or Autoencoder (right)} \label{fig:EMGautoencPos} \end{figure} - \subsection{Cross-validation of Ridge Parameter}%TODO + \subsection{Cross-validation of Ridge Parameter} In tables \ref{tab:ridgeParamEMGkin}, \ref{tab:ridgeParamHighKin} and \ref{tab:ridgeParamAO6Kin} we find the number of 'wins' for each parameter\footnote{\ref{tab:ridgeParamHighKin} and \ref{tab:ridgeParamAO6Kin} were calculated with a order for Burg's method of 50 instead of the later default of 250}. A 'win' refers to a run where this $\lambda$ scored the highest correlation.\\ - For EMG there is no clear preference but it seems like 100 should work as parameter. For EEG we see a clear preference for $\lambda=100$. Low Frequencies seem to prefer a lower parameter about 10 however this was only evaluated for one session. %TODO, if better results + For EMG there is no clear preference but it seems like 100 should work as parameter. For EEG we see a clear preference for $\lambda=100$. Low Frequencies seem to prefer a lower parameter about 10 however this was only evaluated for one session. For all other runs $\lambda = 100$ is used for all methods, better results might be possible with a parameter adapted better. \begin{table} \centering @@ -272,7 +274,7 @@ EMG&324 & 314 & 312 & 314 & 266 \end{array} \end{math} - \caption{Number of 'wins' for each parameter when doing ridge regression to predict velocities from EMG} + \caption{Number of sessions in which the according $\lambda$ was chosen as best parameter when doing ridge regression to predict velocities from EMG} \label{tab:ridgeParamEMGkin} \end{table} \begin{table} @@ -286,7 +288,7 @@ LF& 1396 & 71 & 24 & 39 \end{array} \end{math} - \caption{Number of 'wins' for each parameter when doing ridge regression to predict velocities from EEG, EMG or LF} + \caption{Number of sessions in which the according $\lambda$ was chosen as best parameter when doing ridge regression to predict velocities from EEG, EMG or LF} \label{tab:ridgeParamHighKin} \end{table} \begin{table} @@ -300,7 +302,8 @@ LF& & & & & 1 & 13 & 14 & 2 \end{array} \end{math} - \caption{Number of 'wins' for each parameter when doing ridge regression to predict velocities from EEG, EMG or LF (run on AO6 only)} + \caption{Number of sessions in which the according $\lambda$ was chosen as best parameter when doing ridge regression to predict velocities from EEG, EMG or LF (run on AO6 only)\\ + Low ($\lambda\le 1$) values were only tested for EEG in a separate run} \label{tab:ridgeParamAO6Kin} \end{table} \section{Topographical plots} @@ -319,3 +322,9 @@ \caption{Topographical plot of beta band (13-20 Hz) of the difference between movement and rest for subject FS in the 3rd session} \label{fig:topoBeta} \end{figure} + \begin{figure} + \centering + \includegraphics[height=0.4\textheight]{pictures/results/topoAlpha24.png} + \caption{Differences in activity comparing class 2 and 4 in alpha/mu band for subject FS in session 3. Positive values stand for higher activation in class 2.} + \label{fig:topoAlpha24} + \end{figure} diff --git a/text/thesis/04Discussion.tex b/text/thesis/04Discussion.tex index 2e1d4a1..dbfd508 100644 --- a/text/thesis/04Discussion.tex +++ b/text/thesis/04Discussion.tex @@ -2,11 +2,8 @@ \label{chp:dis} \section{EMG} \label{dis:emg} - Predictions of velocities and positions are quite bad from EMG. - The prediction of the $y$-dimension is a bit better than $x$ ($p<0.05$) for velocities. For positions there is no significant difference ($p\approx 0.31$). Predicting $\theta$ is worse significantly ($p<0.001$) for positions and velocities (also see tables \ref{tab:corrKin} and \ref{tab:corrPos}). %Results? - - A correlation about $0.2$ cannot be used for a BCI. There might be some way of improving the predictions and it might be done several times to higher the correlations, finding another approach however is more promising.\\ - Additionally in many cases a BCI is needed there are no EMG signals since the muscles do not work as they should (e.g. after stroke) and so do not generate the corresponding activity. + Predictions of velocities and positions are quite bad from EMG. A correlation about $0.2$ cannot be used for a BCI. There might be some way of improving the predictions and it might be predicted several times to heighten the correlations, finding another approach however is more promising.\\ + Additionally in many cases a BCI is needed, there are no EMG signals since the muscles do not work as they should (e.g. after stroke) and so do not generate the corresponding activity. Out of these reasons I only use EMG as benchmark for other approaches: If the muscles would work this correlation could be reached. \section{EEG} @@ -77,11 +74,5 @@ This information for example can be found in the alpha band (see figure \ref{fig:topoAlpha}). Here we see clear activation in the left hemisphere and no impact of movement artifacts since the right hemisphere shows no prominent differences in activation.\\ What is interesting in the alpha band is that main activation is measured in the occipital lobe usually associated with visual processing. Since the cue was presented auditory the findings support the idea of the dorsal pathway. This pathway is often called \qq{Where Path} or sometimes \qq{How Path} of visual processing opposing to the ventral \qq{What Path} (cf. \cite{Ungerleider82}). The dorsal pathway is said to be involved in reaching tasks. This is supported by the findings. - When comparing reaches to different targets there are also differences in other brain regions. For example when comparing classes 2 and 4 we find differences differences in anterior regions of the parietal lobe (see figure \ref{fig:topoAlpha24}). Positive values stand for higher activation in class 2.\\ + When comparing reaches to different targets there are also differences in other brain regions. For example when comparing classes 2 and 4 we find differences differences in anterior regions of the parietal lobe (see figure \ref{fig:topoAlpha24}).\\ Here the difference in activation is found in the expected area: the premotoric regions. When predicting movements a focus should be laid on this region, here different movements can be discriminated. The main difference between movement and rest are found in the occipital lobe, for a BCI also this region should be monitored. - \begin{figure} - \centering - \includegraphics[width=0.9\textwidth]{pictures/results/topoAlpha24.png} - \caption{Differences in activity comparing class 2 and 4 in alpha band for subject FS in session 3} - \label{fig:topoAlpha24} - \end{figure} diff --git a/text/thesis/Acd.tex b/text/thesis/Acd.tex index 7b53088..b62cbb8 100644 --- a/text/thesis/Acd.tex +++ b/text/thesis/Acd.tex @@ -116,6 +116,31 @@ [shiftingMean.m] [shiftingPburg.m] [shiftingPos.m] + ] +] +\end{forest} +\begin{forest} +for tree={ + font=\ttfamily, + grow'=0, + child anchor=west, + parent anchor=south, + anchor=west, + calign=first, + edge path={ + \noexpand\path [draw, \forestoption{edge}] + (!u.south west) +(7.5pt,0) |- node[fill,inner sep=1.25pt] {} (.child anchor)\forestoption{edge label}; + }, + before typesetting nodes={ + if n=1 + {insert before={[,phantom]}} + {} + }, + fit=band, + before computing xy={l=15pt}, + } +[CD + [matlabCode [shiftingSum.m] [svmEciton.m] [waveformLengthAll.m] @@ -130,4 +155,8 @@ [thesis.pdf] ] ] -\end{forest} +\end{forest}\\\vfill +\newcommand{\repolink}{https://gitbucket.getenv.net/jph/msccd} +The contents of the CD are also available for download at\\[1cm] +\qrcode{\repolink}\\[.5cm]\texttt{\repolink} + diff --git a/text/thesis/pictures/results/noSyn.png b/text/thesis/pictures/results/noSyn.png index e22258c..42b775b 100644 --- a/text/thesis/pictures/results/noSyn.png +++ b/text/thesis/pictures/results/noSyn.png Binary files differ diff --git a/text/thesis/thesis.tex b/text/thesis/thesis.tex index 91be396..91fb1f0 100644 --- a/text/thesis/thesis.tex +++ b/text/thesis/thesis.tex @@ -24,6 +24,7 @@ \usepackage{dsfont} \usepackage{algpseudocode} \usepackage{lscape} +\usepackage{qrcode} %\renewcommand{\familydefault}{\sfdefault} \usepackage{tikz} @@ -111,7 +112,6 @@ %%% Seite I: Zusammenfassug, Danksagung %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - \section*{Abstract} \addcontentsline{toc}{section}{Abstract} Synergies are patterns of muscle activation where muscles are used in a coordinated way and not each muscle has to be activated separately. Theory is that these patterns can be found in the brain and its activation.\\ @@ -135,13 +135,15 @@ - +\cleardoublepage %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%% Table of Contents %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \renewcommand{\baselinestretch}{1.3} \small\normalsize + +\phantomsection \addcontentsline{toc}{chapter}{Table of Contents} \tableofcontents @@ -151,6 +153,7 @@ +\cleardoublepage %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%% List of Figures %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -158,6 +161,7 @@ \renewcommand{\baselinestretch}{1.3} \small\normalsize +\phantomsection \addcontentsline{toc}{chapter}{List of Figures} \listoffigures @@ -166,6 +170,7 @@ +\cleardoublepage %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%% List of tables %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -173,6 +178,7 @@ \renewcommand{\baselinestretch}{1.3} \small\normalsize +\phantomsection \addcontentsline{toc}{chapter}{List of Tables} \listoftables @@ -181,6 +187,7 @@ % +\cleardoublepage %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%% List of algorithms %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -188,6 +195,7 @@ \renewcommand{\baselinestretch}{1.3} \small\normalsize +\phantomsection \addcontentsline{toc}{chapter}{List of Algorithms} \listofalgorithms @@ -196,6 +204,7 @@ % +%\cleardoublepage %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%% List of abbreviations %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -222,6 +231,7 @@ % +\cleardoublepage %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%% Der Haupttext, ab hier mit arabischer Numerierung %%% Mit \input{dateiname} wird die Datei `dateiname' eingebunden @@ -232,14 +242,17 @@ % Introduction \input{01Introduction} +\cleardoublepage % Materials & Methods \input{02MaterialsAndMethods} +\cleardoublepage % Results \input{03Results} +\cleardoublepage % Discussion @@ -248,14 +261,17 @@ % future work \input{05Future} +\cleardoublepage % %Appendix \appendix \input{Acd} +\cleardoublepage \input{Bfunctions} +\cleardoublepage