diff --git a/text/ideas.txt b/text/ideas.txt deleted file mode 100644 index d48cf2e..0000000 --- a/text/ideas.txt +++ /dev/null @@ -1,7 +0,0 @@ -explanation for bad position/movement estimation: -* not known whether up or down -> gravitational influence -* better to estimate EMG since this corresponds to muscle activity -explanation for bad LF -* no voluntary movement? -explanation for no difference with offset -* ??? diff --git a/text/thesis/01Introduction.tex b/text/thesis/01Introduction.tex index 0bb3218..3839c7d 100644 --- a/text/thesis/01Introduction.tex +++ b/text/thesis/01Introduction.tex @@ -15,7 +15,10 @@ Because there are different possibilities to calculate synergies from EMG we compare them and try to reconstruct movement from them. To be able to compare the results similar calculations were done with other data and paradigms like direct prediction from EEG. - %TODO: maybe some more \section{Overview} -%TODO + After this Introduction in Materials and Methods (Chapter \ref{chp:mat}) we show the scientific background based on the methods used in the work. These reach from PCA and Autoencoders over SVMs and regression to boxplots and topographical plots.\\ + In chapter \ref{chp:results} Results we show the numerical findings of our work separated into parts on synergies, classification, regression and a topographical analysis of the brain activity.\\ + This results and their meaning will be discussed in chapter \ref{chp:dis} Discussion.\\ + Finally we take a look in the possible future and discuss which further research could be done based on or related to our work (chapter \ref{chp:fut}). + The appendix then contains a list of contents on the CD and in the repository (Appendix \ref{app:cd}) and a small documentation of the code used (Appendix \ref{app:docu}) diff --git a/text/thesis/02MaterialsAndMethods.tex b/text/thesis/02MaterialsAndMethods.tex index cc22141..57b8032 100644 --- a/text/thesis/02MaterialsAndMethods.tex +++ b/text/thesis/02MaterialsAndMethods.tex @@ -1,5 +1,5 @@ \chapter{Materials and Methods} -\label{mat} +\label{chp:mat} \section{Scientific background} \label{mat:background} \subsection{BCIs} @@ -26,6 +26,15 @@ \caption{Full 10-20 system} \label{fig:10-20} \end{figure} + \subsubsection{Frequency bands} + EEG can be divided into several bands. According to \cite{Gerrard07} we can propose following bands: + \begin{itemize} + \item Delta: 1-4 Hz + \item Theta: 4-7 Hz + \item Alpha: 7.5-12.5 Hz (depending also on age) + \item Beta: 13-20Hz + \end{itemize} + They were named almost alphabetically concerning their discovery, why they seem assorted sometimes. %TODO: naja \subsection{Power estimation} \subsubsection{EEG} To use data from EEG one way is to analyze the occurring frequencies and their respective power.\\ @@ -244,7 +253,7 @@ Of the kinematic information tracked we only used position ($x,y$) and angle ($\theta$, rotation around $z$-axis) of the hand.\\ Only complete sessions were used in our analysis to ensure better comparability.\\ One session consists of 5 runs with 40 trials each. The trials were separated by resting phases of varying length (2-3s, randomly assigned). Each trial began with an auditory cue specifying the random but equally distributed target for this trial. This leads to 50 reaches to the same target each session. - After the cue the participants should \qq{perform the movement and return to the starting position at a comfortable pace but within 4 seconds}\footnote{\cite{Shiman15}}\\ + After the auditory cue the participants should \qq{perform the movement and return to the starting position at a comfortable pace but within 4 seconds}\footnote{\cite{Shiman15}}\\ For each subject there were 4 to 6 sessions, each recorded on a different day. All in all there were 255 runs in 51 sessions. Each session was analyzed independently as one continuous trial. \subsection{Environment for evaluation} The calculations were done on Ubuntu \texttt{14.04 / 3.19.0-39} with \matlab{} \texttt{R2016a (9.0.0.341360) 64-bit (glnxa64) February 11, 2016}. @@ -445,12 +454,15 @@ \caption{Values used for default} \label{tab:default} \end{table} + \pagebreak \subsection{Topographical Plots} Sometimes the interpretation of EEG data is easier if plotted topographically, meaning visualized according to the corresponding positions on a modeled head.\\ An example is shown in figure~\ref{fig:blink}. \begin{figure} \centering - \includegraphics[width=.9\textwidth]{pictures/topoplotMB1blink.png} + \includegraphics[width=.8\textwidth]{pictures/topoplotMB1blink.png} \caption{Topographical plot of MB1 blinking} \label{fig:blink} \end{figure} + Usually we plot differences between different classes (e.g. movement or rest) and those in a band and not in a single frequency. Examples are given in the Results section in figures \ref{fig:topoAlpha} and \ref{fig:topoBeta}.\\ + To have 0 centered data the relation can be calculated as $$\frac{\text{Move}}{\text{Rest}}-1,$$ where Move and Rest are the mean activity in the band. Dividing instead of subtracting provides a more intuitive measure for the strength of desynchronization. diff --git a/text/thesis/03Results.tex b/text/thesis/03Results.tex index 903b4e5..8dd9dd0 100644 --- a/text/thesis/03Results.tex +++ b/text/thesis/03Results.tex @@ -4,15 +4,13 @@ \label{res:noSyn} To determine the number of synergies to use we predicted all EMG data with each technique and each number of synergies. The result is the plot in figure~\ref{fig:noSyn}.\\ The plot tells that 2 and 4 synergies are good values for Autoencoders, for default nevertheless we use 3 synergies since we also use 3 dimensions of kinematics and so it is better to compare. Three is also the most efficient number of Synergies for PCA and NNMF (cf. Section \ref{dis:noSyn}). - %TODO - \begin{landscape} - \begin{figure} - \centering - \includegraphics[height=\textwidth]{pictures/results/noSyn.png} - \caption{Self prediction accuracy with 1 to 6 synergies} - \label{fig:noSyn} - \end{figure}%TODO: check orientation - \end{landscape} + TODO%TODO + \begin{figure} + \centering + \includegraphics[width=\textwidth,height=\textheight]{pictures/results/noSyn.png} + \caption{Self prediction accuracy with 1 to 6 synergies} + \label{fig:noSyn} + \end{figure}%TODO (last): check orientation of figure (bottom should be outer edge) \section{Classification} \subsection{Comparison of methods of recording} The different methods of recording (EEG, EMG and Low frequencies) also differ in the results. An ANOVA gives $p<0.001$ for all classifications done on 4 different movements and rest. @@ -64,7 +62,8 @@ \label{fig:overviewLF} \end{figure} \subsection{Trade-off parameter} - With a cross validation we compare the results for the soft-margin parameter for $\lambda=0.1,1,10$. The results are shown in figure~\ref{fig:svmCV}.%TODO + With a cross validation we compare the results for the soft-margin parameter for $\lambda=0.1,1,10$. The results are shown in figure~\ref{fig:svmCV}.\\ + TODO%TODO \subsection{Confusion Matrices} A confusion matrix shows whether there is systematic error in classification. In figure \ref{fig:cmFull} there are the confusion matrices for EEG and Low Frequency data, in figure \ref{fig:cmEMG} there is the confusion matrix for EMG data. Since EMG works well for classifying Move/Rest there is also one where only the decision is shown which movement is present. In the second plot we see that many movements are classified as class 3. Especially those belonging to class 2. \begin{figure}[p] @@ -244,7 +243,33 @@ \label{fig:EMGautoencPos} \end{figure} \subsection{Cross-validation of Ridge Parameter} - %TODO + TODO\\%TODO + For EMG we find no clear best parameter. When predicting velocities we get best parameters chosen as shown in table \ref{tab:ridgeParamEMGkin}. A 'win' refers to a run where this $\lambda$ scored the highest correlation. + \begin{table} + \centering + \begin{math} + \begin{array} + {r||c|c|c|c|c} + \lambda&0.1 & 1 & 10 & 100 & 1000\\\hline + \text{number of 'wins'} &324 & 314 & 312 & 314 & 266 + \end{array} + \end{math} + \caption{Number of 'wins' for each parameter when doing ridge regression to predict velocities from EMG} + \label{tab:ridgeParamEMGkin} + \end{table} \section{Topographical plots} - %Maybe in discussion - %TODO + \label{res:topo} + In figure \ref{fig:topoAlpha} we see the difference between move and rest in the alpha band, in \ref{fig:topoBeta} beta band (13-20Hz) is displayed.\\ + Values greater 0 stand for more activity when moving, negative values mean less activity. A value of e.g. 0.15 stands for $15\%$ higher activity when moving. + \begin{figure}[p] + \centering + \includegraphics[height=0.4\textheight]{pictures/results/topoAlpha.png} + \caption{Topographical plot of alpha band (7-13 Hz) of the difference between movement and rest for subject FS in the 3rd session} + \label{fig:topoAlpha} + \end{figure} + \begin{figure}[p] + \centering + \includegraphics[height=0.4\textheight]{pictures/results/topoBeta.png} + \caption{Topographical plot of beta band (13-20 Hz) of the difference between movement and rest for subject FS in the 3rd session} + \label{fig:topoBeta} + \end{figure} diff --git a/text/thesis/04Discussion.tex b/text/thesis/04Discussion.tex index f2e5bdf..75f38a1 100644 --- a/text/thesis/04Discussion.tex +++ b/text/thesis/04Discussion.tex @@ -1,4 +1,5 @@ \chapter{Discussion} +\label{chp:dis} \section{EMG} \label{dis:emg} Predictions of velocities and positions are quite bad from EMG. @@ -14,6 +15,8 @@ This might be because EMG has a hard time classifying the different movements due to massive activity while moving. This can be seen in the confusion matrix (\ref{fig:cmEMG}). Many data points belonging to class 2 or 4 are classified as 3 in error. The classification between movement and rest however works fine.\\ All in all few samples are classified as class 2 even though the training was done on a balanced set. This could mean that features of class 2 are found in other classes too and by that do not have strong predictive power. + The classification results we found for EEG are similar to them of \cite{Shiman15}. They found a classification accuracy of 39.5\%, we have a mean classification of 40.4\% and for targets only (no rest) even 43.6\%. However our findings for discrimination of movement and rest in EEG are a lot worse ($64.6\%$ vs. $38.5\%$). This is probably due to the training of our SVM since when only discriminating between movement and rest we also score $57.1\%$. Higher frequencies with more muscle artifacts (cf. \ref{res:topo} and \ref{dis:topo}) are more predictive for the difference between movement or rest however less predictive for the different classes. So the SVM when trained to distinguish the classes gives higher predictive power to the lower frequencies. + When predicting velocities or positions from EEG there is no significant difference between $x$ and $y$. The difference between $x$ and $y$ and the angle $\theta$ is larger for velocities than for absolute positions since the absolute angle prediction a lot better than the prediction of change.\\ This again is an indication that the actual position is more important for the activity in brain than the change of position as itself. \section{Low Frequencies} @@ -28,7 +31,8 @@ \label{dis:velPos} We expected better performance when predicting velocities instead of absolute positions. Our findings however show the opposite. The performance is quite a lot better when predicting positions directly.\\ This might mean that both EMG and EEG data carry more information about the actual configuration and not only about the change. - %TODO: reason? + + Our findings do not match them of \cite{Ofner12} who found similar results for position and movement. However the task used by \citeauthor{Ofner12} was different from ours as they used self-motivated movement. \section{Pause} \label{dis:pause} The use of a 1 second pause before movement onset only shows significant differences for classification of EMG and predicting Synergies from low frequencies. @@ -57,3 +61,17 @@ \subsection{Comparison with EMG} The results show that the dimensionality reduction from 6 dimensional EMG to 3 dimensional Synergies (here via autoencoder) does not cost much information when predicting velocities and positions.\\ For velocities there is no significant difference and even for positions the mean only differs about $0.03$ (EMG: $0.23$, Autoencoder: $0.20$). +\section{Topographical information} +\label{dis:topo} + In the beta channel (see figure \ref{fig:topoBeta}) we see high activity in the right hemisphere. This is probably an artifact of muscle movements since the commands to drive the right arm should be produced in the left hemisphere.\\ + However - as we see in prediction from EMG - the muscle activity is not very predictive for the direction of movement. EEG is even better than EMG meaning there has to be more information the decision is based on than movement artifacts.\\ + This information for example can be found in the alpha band (see figure \ref{fig:topoAlpha}). Here we see clear activation in the left hemisphere and no impact of movement artifacts since the right hemisphere shows no prominent activation.\\ + What is interesting in the alpha band is that main activation is measured in the occipital lobe usually associated with visual processing. Since the cue was presented auditory our findings support the idea of the dorsal pathway. This pathway is often called \qq{Where Path} or sometimes \qq{How Path} of visual processing opposing to the ventral \qq{What Path} (cf. \cite{Ungerleider82}). The dorsal pathway is said to be involved in reaching tasks. This is supported by our findings. + + When comparing reaches to different targets we also find differences in other brain regions. For example when comparing classes 2 and 4 we find differences differences in anterior regions of the parietal lobe (see figure \ref{fig:topoAlpha24}). Positive values stand for higher activation in class 2. + \begin{figure} + \centering + \includegraphics[width=0.9\textwidth]{pictures/results/topoAlpha24.png} + \caption{Differences in activity comparing class 2 and 4 in alpha band for subject FS in the 3rd session} + \label{fig:topoAlpha24} + \end{figure} diff --git a/text/thesis/05Future.tex b/text/thesis/05Future.tex index 26f3354..4561c93 100644 --- a/text/thesis/05Future.tex +++ b/text/thesis/05Future.tex @@ -1,11 +1,19 @@ \chapter{Future Work} - \section{Classification} - Our results in the topic of classification are not very reliable since we did the classification based on EMG (cf. section \ref{mm:newClass}). It would be interesting to analyze data where the stimulus is matched to the EEG signal and check for early detectability (e.g. with low frequencies as \cite{Lew14}).\\ - Additionally classification - which is enough for some tasks - could be compared to regression. If there is only a limited set of movements a robotic prosthesis has to perform, it could use classification. This should give a lower error rate since the different movements can be distinguished better. - \section{Measurement of error} - For comparison of regression and classification it could be interesting to introduce another measure for performance than just classified correctly or not. It could be interesting how much the predicted movement differs from the real even in the classification task. In that way one would get a measure to decide whether using classification instead of regression pays off.\\ - For this analysis also a variable number of classes would be interesting since 4 movements (as in our setting) is not enough to use an artificial arm. - \section{Offset} - There is no significant effect of an offset in our configuration. When using smaller EEG windows however there might be one. This could be tried in further analyses with small EEG windows.\\ - These small windows however will probably bring other problems as e.g. unstable transformation into Fourier space. So maybe it is necessary to use large windows, then an offset is unnecessary. - \section{Synergies} +\label{chp:fut} +\section{Classification} + Our results in the topic of classification are not very reliable since we did the classification based on EMG (cf. section \ref{mm:newClass}). It would be interesting to analyze data where the stimulus is matched to the EEG signal and check for early detectability (e.g. with low frequencies as \cite{Lew14}).\\ + Additionally classification - which is enough for some tasks - could be compared to regression. If there is only a limited set of movements a robotic prosthesis has to perform, it could use classification. This should give a lower error rate since the different movements can be distinguished better. +\section{Measurement of error} + For comparison of regression and classification it could be interesting to introduce another measure for performance than just classified correctly or not. It could be interesting how much the predicted movement differs from the real even in the classification task. In that way one would get a measure to decide whether using classification instead of regression pays off.\\ + For this analysis also a variable number of classes would be interesting since 4 movements (as in our setting) is not enough to use an artificial arm. +\section{Offset} + There is no significant effect of an offset in our configuration. When using smaller EEG windows however there might be one. This could be tried in further analyses with small EEG windows.\\ + These small windows however will probably bring other problems as e.g. unstable transformation into Fourier space. So if it is necessary to use large windows, an offset is unnecessary. +\section{Use of EEG channels} + To achieve higher performance it would be interesting to identify those EEG channels that contribute most in an good estimation of arm movements or position. There should be channels that do not carry much information for those differentiations, this however has to be explored better.\\ + In this context research could also be done to find out which frequencies allow for the best predictions. +\section{Self-chosen movement} + For a better use of low frequency features our work could be compared with data recorded when subjects move voluntarily. This might also influence the way synergies are predicted and could lead to an better prediction.\\ + Additionally this task matches the requirements for an BCI better, as movement in daily life is more voluntary than decided by a single auditory cue. +\section{Synergies} + TODO %TODO diff --git a/text/thesis/Acd.tex b/text/thesis/Acd.tex index 53ad7ee..a2801bb 100644 --- a/text/thesis/Acd.tex +++ b/text/thesis/Acd.tex @@ -1 +1,2 @@ \chapter{Contents on CD} +\label{app:cd} diff --git a/text/thesis/Bfunctions.tex b/text/thesis/Bfunctions.tex index bb848b7..21fbf23 100644 --- a/text/thesis/Bfunctions.tex +++ b/text/thesis/Bfunctions.tex @@ -1,4 +1,5 @@ \chapter{Documentation of the Code} +\label{app:docu} The documentation of the Code will be split into parts according to the usage. in this parts the order will be alphabetically in the name of the function. \section{\texttt{callAll.m}} diff --git a/text/thesis/mylit.bib b/text/thesis/mylit.bib index 355a059..c472bd8 100755 --- a/text/thesis/mylit.bib +++ b/text/thesis/mylit.bib @@ -218,6 +218,29 @@ pages = "1–4" } +@article{Gerrard07, + author = "Paul Gerrard and Robert Malcolm", + title = "Mechanisms of modafinil: A review of current research", + year = "2007", + journal = "Neuropsychiatric disease and treatment", + volume = "3(3)", + pages = "349–364" +} +@inproceedings{Ungerleider82, + author = "Leslie G. Ungerleider and Mortimer Mishkin", + title = "Two cortical visual systems", + year = "1982", + booktitle= "Analysis of Visual Behavior", + editor = "David J. Ingle and Melvin A. Goodale and Richard J. W. Mansfield", + pages = "549-586", + journal = "MIT press" +} +@article{Ofner12, + author = "Patrick Ofner and Gernot R. Müller-Putz", + title = "Decoding of velocities and positions of 3D arm movement from EEG", + year = "2012", + journal = "Conf Proc IEEE Eng Med Biol Soc." +} %not cited @article{Morasso92, diff --git a/text/thesis/pictures/results/EMGautoenc.png b/text/thesis/pictures/results/EMGautoenc.png new file mode 100644 index 0000000..7279f35 --- /dev/null +++ b/text/thesis/pictures/results/EMGautoenc.png Binary files differ diff --git a/text/thesis/pictures/results/noSyn.png b/text/thesis/pictures/results/noSyn.png index e43143f..e22258c 100644 --- a/text/thesis/pictures/results/noSyn.png +++ b/text/thesis/pictures/results/noSyn.png Binary files differ diff --git a/text/thesis/pictures/results/topoAlpha.png b/text/thesis/pictures/results/topoAlpha.png new file mode 100644 index 0000000..d622aa0 --- /dev/null +++ b/text/thesis/pictures/results/topoAlpha.png Binary files differ diff --git a/text/thesis/pictures/results/topoAlpha24.png b/text/thesis/pictures/results/topoAlpha24.png new file mode 100644 index 0000000..ffb5686 --- /dev/null +++ b/text/thesis/pictures/results/topoAlpha24.png Binary files differ diff --git a/text/thesis/pictures/results/topoBeta.png b/text/thesis/pictures/results/topoBeta.png new file mode 100644 index 0000000..872a97c --- /dev/null +++ b/text/thesis/pictures/results/topoBeta.png Binary files differ