diff --git a/text/thesis/01Introduction.tex b/text/thesis/01Introduction.tex index 9ecac2e..7531e58 100644 --- a/text/thesis/01Introduction.tex +++ b/text/thesis/01Introduction.tex @@ -3,7 +3,7 @@ \section{Motivation} \label{intro:motivation} \qq{Reading the mind} is something humanity is and always was exited about. Whatever one may think about the possibility of doing so as a human, computers have a chance to catch a glimpse of the (neuronal) activity in the human brain and interpret it.\\ - Here we use Electroencephalography (EEG) to record brain activity and try to predict arm movements from the data.\\ + Here Electroencephalography (EEG) is used to record brain activity and try to predict arm movements from the data.\\ Using this as a Brain-Computer-Interface (BCI) holds the possibility of restoring e.g. a lost arm. This arm could be used as before by commands constructed in the brain. In a perfect application there would be no need of relearning the usage. The lost arm could just be replaced.\\ Another opportunity this technique provides is support of retraining the usage of the natural arm after stroke. If it is possible to interpret the brainwaves the arm can be moved passively according to the commands formed in brain. This congruency can restore the bodies own ability to move the arm as \cite{Gomez11} show.\\ In a slightly different context it might become possible to handle a machine (e.g. an industrial robot or mobile robots like quadrocopters) with \qq{thoughts} (i.e. brain activity) like an additional limb. One could learn to use the possibilities of the robot like the possibilities of his arm and hand to modulate something.\\ diff --git a/text/thesis/02MaterialsAndMethods.tex b/text/thesis/02MaterialsAndMethods.tex index 97f15db..3486eb0 100644 --- a/text/thesis/02MaterialsAndMethods.tex +++ b/text/thesis/02MaterialsAndMethods.tex @@ -10,11 +10,11 @@ \subsubsection{Methods of recording} First approaches used invasive BCIs earlier in Animals (rodents and monkeys) later also in humans. Invasive BCIs in humans were mostly implanted when the human was under brain surgery for another reason like therapy of epilepsy. Problems of invasive BCIs are the need to cut through skull and dura mater. This can lead to infections and severe brain damage.\\ An improvement were less invasive BCIs with e.g. Electrocorticography (ECoG) which is placed inside the skull but outside the dura which decreased the risk for infections massively.\\ - Measuring outside the skull entails even less risk, the dura and skull however lower the quality of the signal massively. With some improvements EEG has a spatial resolution of 2-3 cm (cf. \cite{Babiloni01}). This is quite bad compared to the single neuron one can observe with invasive methods. However we are more interested in the activity of areas then single cells for our task, so EEG meets our requirements here.\\ + Measuring outside the skull entails even less risk, the dura and skull however lower the quality of the signal massively. With some improvements EEG has a spatial resolution of 2-3 cm (cf. \cite{Babiloni01}). This is quite bad compared to the single neuron one can observe with invasive methods. However since activity is usually higher in an area and not only a single neuron EEG meets the requirements here.\\ In addition EEG is much cheaper and easier to use than other techniques. There is no need for surgery (like for invasive methods) and the hardware can be bought for less than 100\euro{} while functional Magnetic Resonance Imaging (fMRI) hardware costs far above 100,000\euro{}. This is one of the reasons EEG is far more available than other techniques. There are some inventions of younger date but not as much work has been done with them why they are not as well known and as far distributed as EEG.\\ Another pro of EEG is that the device is head mounted. That means the user may move while measuring without high impact on the tracking of activity. This is highly necessary for any BCI used in daily life. \subsection{EEG} - When using Electroencephalography (EEG) one measures the electrical fields on the scalp that are generated by activity of neurons in the brain. These measurements allow some interpretation about what is happening inside the skull. In our application we use the recorded currents directly to train a SVM or as predictor for regression. + When using Electroencephalography (EEG) one measures the electrical fields on the scalp that are generated by activity of neurons in the brain. These measurements allow some interpretation about what is happening inside the skull. Here the recorded currents are used directly to train a SVM or as predictor for regression. The foundation stone for EEG was laid in 1875 when Richard Caton found electrical activity around the brain of monkeys. After testing the methods on animals in 1924 the first human EEG was recorded by Hans Berger in Jena. He also coined the term Electroencephalography and is seen as the inventor of EEG. @@ -31,14 +31,14 @@ \label{fig:10-20} \end{figure} \subsubsection{Frequency bands} - EEG can be divided into several bands according to the frequency of the waves. Usually this also corresponds to the form of waves and allows insights in the mental state from rest to highly aroused. According to \cite{Gerrard07} we can propose following bands: + EEG can be divided into several bands according to the frequency of the waves. Usually this also corresponds to the form of waves and allows insights in the mental state from rest to highly aroused. According to \cite{Gerrard07} following bands can be proposed: \begin{itemize} \item Delta: 1-4 Hz \item Theta: 4-7 Hz \item Alpha: 7.5-12.5 Hz (depending also on age) \item Beta: 13-20Hz \end{itemize} - There are different definitions of the limits of the bands, as we only use them for rough estimation we stick to these. For more exact results an analysis of wave patterns would be necessary. + There are different definitions of the limits of the bands, but for a rough estimation this limits are suitable. For more exact results an analysis of wave patterns would be necessary. In limits similar to them of the alpha wave also Mu-waves are measured. They are associated with mirror neurons in the motor cortex and their activity is suppressed while the subject is moving. %TODO \subsection{Low Frequencies} @@ -46,18 +46,18 @@ Also in predicting movements there was found some significance in low frequency as was done by \cite{Liu11} and \cite{Antelis13} for example. \citeauthor{Antelis13} found correlations between hand movement and low frequency signal of $(0.29,0.15,0.37)$ in the dimensions respectively.\\ \cite{Lew14} state low frequencies are mainly involved in spontaneous self-induced movement and can be found before the movement starts. By this they may be a great possibility to lower reaction time of neuroprostheses for example. \subsection{EMG} - When using muscles, they are contracted after a signal via an efferent nerve activates them. Contraction of muscles also releases measurable energy, which is used for Electromyography (EMG). There are intramuscular applications of EMG but we only used surface EMG.\\ - From surface EMG, activity of muscles can be estimated, however not very precisely without repetition. Since the muscles used for arm movements are quite large in our setting, EMG allows relatively precise estimations of underlying muscle activity. + When using muscles, they are contracted after a signal via an efferent nerve activates them. Contraction of muscles also releases measurable energy, which is used for Electromyography (EMG). There are intramuscular applications of EMG but here only used surface EMG is used.\\ + From surface EMG, activity of muscles can be estimated, however not very precisely without repetition. Since the muscles used for arm movements are quite large in this setting, EMG allows relatively precise estimations of underlying muscle activity. - EMG is mainly developed for diagnostic tasks. However it is also applicable in science to track muscle activity as we do here. + EMG is mainly developed for diagnostic tasks. However it is also applicable in science to track muscle activity as is done here. \section{Signal Processing} \subsection{Power estimation} \subsubsection{EEG} To use data from EEG one way is to analyze the occurring frequencies and their respective power.\\ - To gain these from the continuous we use windows in which the signal is finite and a Fourier transform can be estimated. For this we use power spectral density (PSD) estimation. + To gain these from the continuous signal windows can be used in which the signal is finite and a Fourier transform can be estimated. For the estimation power spectral density (PSD) is used. \subsubsection{Power spectral density estimation} The PSD is the power per frequency, where power refers to the square of the amplitude.\\ - If the Fourier transform is existing, PSD can be calculated from it e.g. as periodogram. If not it has to be estimated. One way to do so is fast Fourier transform (FFT), another - used here - is parametrized with an Autoregressive model (AR). For this one assumes that there is a relationship of the spectral density between $p$ consecutive samples and the following one. This leads to an equation with only $p$ parameters which can be estimated in different ways. We used Burg's method (\texttt{pburg} from \matlab{} library).\\ + If the Fourier transform is existing, PSD can be calculated from it e.g. as periodogram. If not it has to be estimated. One way to do so is fast Fourier transform (FFT), another - used here - is parametrized with an Autoregressive model (AR). For this one assumes that there is a relationship of the spectral density between $p$ consecutive samples and the following one. This leads to an equation with only $p$ parameters which can be estimated in different ways. Here Burg's method (\texttt{pburg} from \matlab{} library) is used.\\ In Figure~\ref{fig:psd} we see the difference between autoregressive \texttt{pburg} and periodogram \texttt{pwelch} PSD estimation. \begin{figure} \includegraphics[width=\textwidth]{psd.png} @@ -68,8 +68,8 @@ \label{mat:burg} Burg's method (\cite{Burg75}) is a special case of parametric PSD estimation. It interprets the Yule-Walker-Equations as least squares problem and iteratively estimates solutions.\\ According to \cite{Huang14} Burg's method fits well in cases with the need of high resolution.\\ - Burg and Levinson-Durbin algorithms are examples for PSD estimation which use an autoregressive model instead of Fast Fourier Transformation. The approach is described well by \cite{Spyers98}. The idea is to lower the number of parameters determining the production of the signal. The number of parameters used is called \textit{model order} (250 in our example, lower in most cases). These parameters are estimated from the original data. For PSD estimation the modeled values are used which allows easier transformation since the data is generated by an known process.\\ - Often the Rational Transfer Function Modeling is used having the general form of $$x_n=-\sum\limits_{k=1}^p a_kx_{n-k}+ \sum\limits_{k=0}^q b_ku_{n-k},$$ where $x_n$ is the output, $u_n$ the input. $a,b$ are the system parameters which have to be estimated from original data. As we have unknown input in our application the output can only be estimated which simplifies the formula as follows $$\hat{x}_n=-\sum\limits_{k=1}^p a_k\hat{x}_{n-k}.$$ + Burg and Levinson-Durbin algorithms are examples for PSD estimation which use an autoregressive model instead of Fast Fourier Transformation. The approach is described well by \cite{Spyers98}. The idea is to lower the number of parameters determining the production of the signal. The number of parameters used is called \textit{model order} (250 for this thesis, lower in most cases). These parameters are estimated from the original data. For PSD estimation the modeled values are used which allows easier transformation since the data is generated by an known process.\\ + Often the Rational Transfer Function Modeling is used having the general form of $$x_n=-\sum\limits_{k=1}^p a_kx_{n-k}+ \sum\limits_{k=0}^q b_ku_{n-k},$$ where $x_n$ is the output, $u_n$ the input. $a,b$ are the system parameters which have to be estimated from original data. If there is unknown input the output can only be estimated which simplifies the formula as follows $$\hat{x}_n=-\sum\limits_{k=1}^p a_k\hat{x}_{n-k}.$$ Estimating the parameters is done by minimizing the forward prediction error $E$: $$E=\frac{1}{N}\sum\limits_{i=1}^N \left(x_i-\hat{x}_i\right)^2$$ The minimum has zero slope and can be found by setting the derivative to zero:$$\frac{\partial E}{\partial a_k}=0,\text{ for } 1\le k\le p$$ This yields a set of equations called \emph{Yule-Walker-Equations} (cf. \cite{Yule27},\cite{Walker31}).\\ @@ -102,7 +102,7 @@ The formula for NMF is $$Input\approx \mathbf{WH}$$ where Input is $n\times m$, $W$ is $n\times r$ and $H$ is $r\times m$ with $r<<\min\{m,n\}$. So $\mathbf{WH}$ is only an approximation of the input however with significant lower dimension - the number of Synergies used.\\ - The factorization is learned with an update rule that my be chosen. We used the \matlab{} default, an alternating least squares (ALS) algorithm. It can be described as in algorithm \ref{alg:als} (cf. \cite{Berry07}).\\ + The factorization is learned with an update rule that my be chosen.The \matlab{} default, an alternating least squares (ALS) algorithm, is used here. It can be described as in algorithm \ref{alg:als} (cf. \cite{Berry07}).\\ \begin{algorithm} \begin{algorithmic} \State initialize $\mathbf{W}$ randomly @@ -125,7 +125,7 @@ \label{mat:autoenc} Autoencoders are a specific type of artificial neural networks (ANN). They work like typical ANNs by adjusting weights between the layers however they do not predict an unknown output but they predict their own input. What is interesting now is manipulating the size of the hidden layer where the hidden layer has to be smaller than the input layer. Now in the hidden layer the information of the input can be found in a condensed form (e.g. synergies instead of single muscle activity).\\ Autoencoders have been successfully used by Spüler et al. to extract synergies from EMG (\cite{Spueler16}). Especially with a lower number of synergies autoencoders should perform better than PCA or NMF because linear models fail to discover the agonist-antagonist relations that are typical for muscle movements. These however can be detected by autoencoders which allows for good estimations with half the synergies.\\ - An autoencoder's input layer has as many neurons as there are input dimensions (e.g. one for each EMG channel). The number of hidden layer neurons may be varied. We mostly used 3. The output layer is of the same size as the input layer. This autoencoder is shown in Figure~\ref{fig:autoenc}. + An autoencoder's input layer has as many neurons as there are input dimensions (e.g. one for each EMG channel). The number of hidden layer neurons may be varied, here usually 3 are used. The output layer is of the same size as the input layer. This autoencoder is shown in Figure~\ref{fig:autoenc}. \begin{figure} \centering \input{autoencoder.tikz} @@ -228,7 +228,7 @@ \caption{2D confusion matrix} \label{tab:tptnftfn} \end{table} - In the higher dimensional case \matlab{} uses color coded maps as figure~\ref{fig:exampleCM}. In our application we use scaled confusion matrices where each row adds up to 1. + In the higher dimensional case \matlab{} uses color coded maps as figure~\ref{fig:exampleCM}. In this thesis scaled confusion matrices where each row adds up to 1 are used. \begin{figure} \includegraphics[width=\textwidth]{pictures/results/cmEEGfull.png} \caption{Example for a confusion matrix} @@ -239,12 +239,13 @@ The Hypotheses tested are that all group means are equal ($H_0$) or they are not ($H_1$). To check on those ANOVA compares the deviation from the over-all mean and compares it to the deviation within the groups. If a lot of variance in the data can be explained by the groups (meaning in-group variance is lower than variance between groups) it is quite likely that the proposed groups have different means.\\ Whether this is significant is decided based on the $p$-Value representing the probability that the difference between in-group and between-group variance is even higher. $H_0$ is rejected if $p$ is lower than a defined threshold (often $0.05$, $0.01$ or $0.001$). \subsection{Boxplot} - To plot data and show their distribution we use boxplots. + To plot data and show their distribution boxplots are used. A boxplot contains information about the median (red line), 0.25 and 0.75 quantiles (ends of the box) and about the highest and lowest values that are not classified as outliers.\\ A data point $y$ is classified as outlier if $y > q_3+1.5\cdot(q_3-q_1)$ or $y < q_1-1.5\cdot(q_3-q_1)$, where $q_1,q_3$ are the first and third quartile (which are also defining the box). -\chapter{Experimental design} +\chapter{Materials and Methods} +\section{Experimental design} \label{mm:design} - The data used for this work was mainly recorded by Farid Shiman, Nerea Irastorza-Landa, and Andrea Sarasola-Sanz for their work (\cite{Shiman15},\cite{Sarasola15}). We were allowed to use it for further analysis.\\ + The data used for this work was mainly recorded by Farid Shiman, Nerea Irastorza-Landa, and Andrea Sarasola-Sanz for their work (\cite{Shiman15},\cite{Sarasola15}). I was allowed to use it for further analysis.\\ There were 9 right-handed subjects with an average age of 25 (variance 6.67, minimum 20, maximum 28). Three female and 6 male subjects were tested. All the tasks were performed with the dominant right hand.\\ To perform was a center-out reaching task to one of four targets (see figure \ref{fig:experimentalDesign}) while 32 channel EEG, 6 channel% \footnote{\texttt{'AbdPolLo', 'Biceps', 'Triceps', 'FrontDelt', 'MidDelt'} and \texttt{'BackDelt'} were recorded for every subject, others only in some. Only the 6 channels tracked in every session were used} % @@ -255,17 +256,17 @@ \caption{Center-out reaching task with four color-coded targets (picture from \citeauthor{Shiman15})} \label{fig:experimentalDesign} \end{figure} - Of the kinematic information tracked we only used position ($x,y$) and angle ($\theta$, rotation around $z$-axis) of the hand.\\ - Only complete sessions were used in our analysis to ensure better comparability.\\ + Of the kinematic information tracked only position ($x,y$) and angle ($\theta$, rotation around $z$-axis) of the hand are used.\\ + Only complete sessions were used for this analyses to ensure better comparability.\\ One session consists of 5 runs with 40 trials each. The trials were separated by resting phases of varying length (2-3s, randomly assigned). Each trial began with an auditory cue specifying the random but equally distributed target for this trial. This leads to 50 reaches to the same target each session. After the auditory cue the participants should \qq{perform the movement and return to the starting position at a comfortable pace but within 4 seconds} (\cite{Shiman15})\\ For each subject there were 4 to 6 sessions, each recorded on a different day. All in all there were 255 runs in 51 sessions. Each session was analyzed independently as one continuous trial. - \section{Environment for evaluation} + \subsection{Environment for evaluation} The calculations were done on Ubuntu \texttt{14.04 / 3.19.0-39} with \matlab{} \texttt{R2016a (9.0.0.341360) 64-bit (glnxa64) February 11, 2016}. -\chapter{Data Acquisition} +\section{Data Acquisition} \subsection{Loading of data} The data recorded with BCI2000 (\cite{Schalk04}) can be loaded into \matlab{} with a specific \texttt{.mex} file. The according \texttt{.mex}-Files for some platforms (Windows, MAC, Linux) are available from BCI2000 precompiled.\\ - We load the signal plus the according status data and the parameters (see Algorithm~\ref{alg:load_bcidat}). + The signal plus the according status data and the parameters is loaded as shown in Algorithm~\ref{alg:load_bcidat}). \begin{algorithm} \begin{algorithmic} \State [signal, states, params] = load\_bcidat('dataFile.dat'); @@ -274,7 +275,7 @@ \label{alg:load_bcidat} \end{algorithm} \subsubsection{Signal} - The signal is loaded as matrix of 41 channels (see Table~\ref{tab:channelNames}). All the values are integers corresponding to the voltage and also can be loaded as floating point values representing microvolts. Since the representation should not make any difference when analyzing the spectrum we use the smaller representation as integers. + The signal is loaded as matrix of 41 channels (see Table~\ref{tab:channelNames}). All the values are integers corresponding to the voltage and also can be loaded as floating point values representing microvolts. Since the representation should not make any difference when analyzing the spectrum the smaller representation as integers is used here. \begin{table} \centering \begin{tabular}{c|c|l} @@ -332,65 +333,65 @@ All the settings from the BCI2000 recording are saved in and loaded from the \texttt{parameters}.\\ Examples are the names of the channels, the random seed for BCI2000 and the sounds, meaning and duration for different stimuli. \subsection{Filtering} - For filtering we use second order Butterworth filters. - For normal EEG we used: + For filtering I use second order Butterworth filters. + For normal EEG: \begin{enumerate} \item Highpass at \texttt{minEEGFreq} (usually 2Hz) \item Lowpass at \texttt{maxEEGFreq} (usually 49Hz) \item bandstop at 48-52Hz and 148-152Hz to eliminate (possibly relevant) power line noise \end{enumerate} - For low frequencies we use: + For low frequencies: \begin{enumerate} \item Highpass at 0.01Hz \item Lowpass at 1Hz \end{enumerate} - For EMG data we use: + For EMG data: \begin{enumerate} \item Highpass at 2Hz to eliminate movement artifacts \item bandstop at 48-52Hz and 148-152Hz to eliminate (possibly relevant) power line noise \end{enumerate} - For filtering we use \matlab{}s \texttt{filtfilt} function to reduce shift due to multiple filtering steps. + For filtering \matlab{}s \texttt{filtfilt} function is used to reduce shift due to multiple filtering steps. \subsection{Windowing} To process continuous EEG or EMG data it is necessary to define windows. These windows are shifted over the data to get discrete values for the further steps.\\ - We choose as default for EEG + Defaults for EEG are: \begin{itemize} \item window $1 s$ \item shift $200ms$ \end{itemize} - and for EMG + Defaults for EMG are: \begin{itemize} \item window $0.2s$ \item shift $50ms$. \end{itemize} - For re-synchronization we pass the shift to every function that needs to align EMG and EEG data and operate on \emph{time-from-start} there. + For re-synchronization the shift is passed to every function that needs to align EMG and EEG data and operate on \emph{time-from-start} there. \subsection{Waveform length} - For the representation of EMG data we used waveform length as described by \cite{Phinyomark12} which gives a measurement for the change in the window. It is calculated as the sum over the absolute difference of consecutive measurements in the time window: + For the representation of EMG data waveform length as described by \cite{Phinyomark12} is used, which gives a measurement for the change in the window. It is calculated as the sum over the absolute difference of consecutive measurements in the time window: $$\sum\limits_{i=1}^N \left| x_i-x_{i-1}\right|,$$ where $N$ is the number of measurements in the time window and $x_i$ are the measurements themselves.\\ It describes the length of the way of a needle plotting the EMG on a self-moving band. \subsection{Classification} \label{mm:newClass} - Very bad results when classifying EMG into Move/Rest made us further inspect the data. The actual movement came quite a while after the stimulus.\\ - To address this problem we did a re-classification of the data according to actual movements (cf. Appendix~\ref{code:classify}). To decide whether the subject is moving or not we compare the mean EMG activity (from Waveform Length) to a threshold (10,000 by default).\\ - If there is movement we define the class occurring most in the second before as the current task. If there is movement but the stimulus tells to rest, we assign the last active stimulus.\\ - In addition we take the second before movement onset out of the data (classified as -1) and (optionally) half a second before movement onset as belonging to the following stimulus (cf. \ref{mat:pause}).\\ - Finally we do some smoothening by taking the most occurring class one second before to one second after the current time step as its class.\\ - As last step we adjust the length of the stimulus-vector to the length of the EEG data.\\ - According to this classification we take only data points in the further analysis which are classified different than -1 meaning they are either clear rest or clear movement. + Very bad results when classifying EMG into Move/Rest made me further inspect the data. The actual movement came quite a while after the stimulus.\\ + To address this problem I did a re-classification of the data according to actual movements (cf. Appendix~\ref{code:classify}). To decide whether the subject is moving or not the mean EMG activity (from Waveform Length) is compared to a threshold (10,000 by default).\\ + If there is movement the class occurring most in the second before is defined as the current task. If there is movement but the stimulus tells to rest, the last active stimulus is assigned.\\ + In addition the second before movement onset is taken out of the data (classified as -1) and (optionally) half a second before movement onset as belonging to the following stimulus (cf. \ref{mat:pause}).\\ + Finally some smoothening is done by taking the most occurring class one second before to one second after the current time step as its class.\\ + As last step the length of the stimulus-vector is adjusted to the length of the EEG data.\\ + According to this classification only data points are taken in the further analysis that are classified different than -1 meaning they are either clear rest or clear movement. \subsection{Synergies} \label{mat:synergies} - Synergies we generate based on different options for dimensionality reduction (cf. \ref{back:synergies}).\\ + Synergies are generated based on different options for dimensionality reduction (cf. \ref{back:synergies}).\\ EMG data (as waveform length) is reduced to $n$ dimensions, where $n$ is the desired number of Synergies.\\ Using PCA this is done by taking the first $n$ components. Then the EMG data is transformed into the $n$-dimensional space spanned by the components.\\ NMF is done with $n$ as inner dimension. Then EMG data is multiplied with the resulting matrix to transform it to $n$-dimensional data.\\ Autoencoders eventually are trained with a hidden layer of size $n$ and afterwards EMG data is encoded with the learned weights. This is equivalent to taking the hidden layer activity for the corresponding time step.\\ Since synergies are generated from EMG they have the same dimensionality in the first dimension\footnote{only depending on window size and shift for EMG data and the recoding duration} and $n$ in the second. \subsection{Kinematics} - Kinematic data we used either as movement or as position. The position was directly recorded, the movement is the first derivative of the position in time.\\ - The recording of kinematics was started after that of EEG. In synchronization channel\footnote{cf. Table~\ref{tab:channelNames}} there is a peak when kinematic recording is started. This was used to align movement with EEG and EMG data. In addition we adjusted the kinematic data to the EMG window and shift to be able to use corresponding data for the same time step. This was done by summing all differences (for movement) or by calculating the mean position in the time window.\\ - Size of this data is same as EMG and Synergies in length but has only three features per time step since we used only 3D positioning ($x,y$ and $\theta$) of the hand and no information about the fingers. -\chapter{Data Analysis} - Figure~\ref{fig:overview} shows the steps of our work. EEG, EMG and positions were recorded, Synergies and velocities were calculated from them. To check the performance of our methods the relations between them were predicted. + Kinematic data is used either as movement or as position. The position was directly recorded, the movement is the first derivative of the position in time.\\ + The recording of kinematics was started after that of EEG. In synchronization channel\footnote{cf. Table~\ref{tab:channelNames}} there is a peak when kinematic recording is started. This was used to align movement with EEG and EMG data. In addition the kinematic data is adjusted to the EMG window and shift to be able to use corresponding data for the same time step. This was done by summing all differences (for movement) or by calculating the mean position in the time window.\\ + Size of this data is same as EMG and Synergies in length but has only three features per time step since only 3D positioning ($x,y$ and $\theta$) of the hand and no information about the fingers are used. +\section{Data Analysis} + Figure~\ref{fig:overview} shows the steps of this work. EEG, EMG and positions were recorded, Synergies and velocities were calculated from them. To check the performance of the methods the relations between them were predicted. \begin{figure} \centering \input{pictures/overview.tikz} @@ -398,38 +399,37 @@ \label{fig:overview} \end{figure} \subsection{Classification} - Classification can be done in different ways. First approach is discriminating Movement from Rest. This is done by training an SVM and testing its results with 10-fold cross validation. We do this with EMG, EEG and LF data. EMG in this setting is trivial since it was the basis for the classification (cf. \ref{mm:newClass}).\\ - In a second step we try to discriminate movement in different directions also with an SVM trained on EMG, EEG or LF data respectively. The fit of the model is also checked with 10-fold cross validation.\\ - For unbiased classification it is necessary to train with equally sized classes. For that purpose and to lower computation time we only take in 250 (as default) samples per class%TODO?: (cf. Comparison of results in Section~\ref{res:maxPerClass}) - .\\ + Classification can be done in different ways. First approach is discriminating Movement from Rest. This is done by training an SVM and testing its results with 10-fold cross validation. Here this is done with EMG, EEG and LF data. EMG in this setting is trivial since it was the basis for the classification (cf. \ref{mm:newClass}).\\ + In a second step I try to discriminate movement in different directions also with an SVM trained on EMG, EEG or LF data respectively. The fit of the model is also checked with 10-fold cross validation.\\ + For unbiased classification it is necessary to train with equally sized classes. For that purpose and to lower computation time only 250 (as default) samples per class are taken in.\\ The parameter $c$ for the support vector machine is found with an additional step of cross validation or set to 1. (Results in Section~\ref{res:maxC}).\\ To learn about biased classifications and about similar brain activity for different movements the confusion matrix is created (cf. Section~\ref{mm:cm}).\\ The resulting accuracy is the mean of each of the 10 cross validation steps. \subsection{Predicting Kinematics} - The prediction of kinematics is done with ridge regression. Since there are more data for kinematics than for EEG we use the mean position or movement and predict these.\\ + The prediction of kinematics is done with ridge regression. Since there are more data for kinematics than for EEG the mean position or movement are used and predicted.\\ The regression is done in 10-fold cross validation for each dimension ($x,y,\theta$) and the parameter $\lambda$ (cf. ~\ref{mm:ridge}) is ascertained with an additional cross validation. The resulting correlation is the mean correlation of each of the 10 parts with the best parameter lambda each while the correlation for each dimension is calculated independently. \subsection{Predicting Synergies} Predicting synergies works similar as for the kinematics. Only change is that the synergies may have other dimensionality. Nevertheless each synergy is predicted from all EEG data as one output and correlation is calculated for each synergy. \subsection{Predicting EMG} - When predicting EMG data we use the sum of the waveform length in the time corresponding to the EEG data. As the EMG data was summed to gain the data for our use this is a reasonable approach.\\ + When predicting EMG data the sum of the waveform length in the time corresponding to the EEG data is used. As the EMG data was summed to gain the data this is a reasonable approach.\\ The remaining steps are the same as for kinematics and Synergies. \subsection{EEG offset} \label{mat:offset} - Since it takes some time for commands to go from brain to the muscles, we introduced an variable offset between EEG and other data. The offset has to be given in a number of shifts, so in default is a multiple of 200ms.\\ + Since it takes some time for commands to go from brain to the muscles, a variable offset between EEG and other data is used. The offset has to be given in a number of shifts, so in default is a multiple of 200ms.\\ Results are given in Sections~\ref{res:offsetEEG} and~\ref{res:offsetLF}. \subsection{Pause} \label{mat:pause} - We introduce a pause before movement onset. This pause means that 1 second before movement onset is not taken into account when analyzing the data. If there is no pause we only take 1s to 0.5 second before movement onset out and classify the last 0.5 seconds before movement as belonging to the following task.\\ - This was necessary since the data about presentation of stimuli did not match the recordings and we had to reclassify (cf. section \ref{mm:newClass}). + A pause is used before movement onset. This pause means that 1 second before movement onset is not taken into account when analyzing the data. If there is no pause only 1s to 0.5 second before movement onset is left out and the last 0.5 seconds before movement are classified as belonging to the following task.\\ + This was necessary since the data about presentation of stimuli did not match the recordings and reclassification was necessary (cf. section \ref{mm:newClass}). \subsection{Prediction with interim step} - All these analyses only show the accuracy of one step. To get a measure for the over-all performance we predict synergies from EEG and use them to predict EMG or kinematics respectively.\\ + All these analyses only show the accuracy of one step. To get a measure for the over-all performance synergies are predicted from EEG and used to predict EMG or kinematics respectively.\\ The resulting correlation is the mean of the correlations of a 10-fold cross validation where the same unknown synergies are predicted from EEG and used to predict EMG or kinematics. So there is no correction step between the steps and EMG or kinematics are predicted from EEG via the Synergies. Here also different methods to determine Synergies are compared (see Section~\ref{res:differentSynergiesVia}). \subsection{Multiple Sessions} - We analyze each session (cf. Section~\ref{mm:design}) independently meaning there are 51 independent results for each analysis. These are used for the statistical evaluation in Chapter~\ref{chp:results}.\\ + Each session (cf. Section~\ref{mm:design}) is analyzed independently meaning there are 51 independent results for each analysis. These are used for the statistical evaluation in Chapter~\ref{chp:results}.\\ Some analyses are only done on one session - if so it will be clearly stated. \subsection{Default values} \label{mat:default} - The values of our variables used in \texttt{'Default'} are given in table~\ref{tab:default}. + The values of the variables used in \texttt{'Default'} are given in table~\ref{tab:default}. \begin{table} \centering \begin{tabular}{r|c|l} @@ -461,13 +461,7 @@ \end{table} \pagebreak \subsection{Topographical Plots} - Sometimes the interpretation of EEG data is easier if plotted topographically, meaning visualized according to the corresponding positions on a modeled head.\\ - An example is shown in figure~\ref{fig:blink}. - \begin{figure} - \centering - \includegraphics[width=.8\textwidth]{pictures/topoplotMB1blink.png} - \caption{Topographical plot of MB1 blinking} - \label{fig:blink} - \end{figure} - Usually we plot differences between different classes (e.g. movement or rest) and those in a band and not in a single frequency. Examples are given in the Results section in figures \ref{fig:topoAlpha} and \ref{fig:topoBeta}.\\ + Sometimes the interpretation of EEG data is easier if plotted topographically, meaning visualized according to the corresponding positions on a modeled head. + + Usually differences between different classes (e.g. movement or rest) are plotted in a band and not in a single frequency. Examples are given in the Results section in figures \ref{fig:topoAlpha} and \ref{fig:topoBeta}.\\ To have 0 centered data the relation can be calculated as $$\frac{\text{Move}}{\text{Rest}}-1,$$ where Move and Rest are the mean activity in the band. Dividing instead of subtracting provides a more intuitive measure for the strength of desynchronization. diff --git a/topoplot/plotOneSubjectOneDay.m b/topoplot/plotOneSubjectOneDay.m index d6b5f68..38e6289 100644 --- a/topoplot/plotOneSubjectOneDay.m +++ b/topoplot/plotOneSubjectOneDay.m @@ -20,8 +20,8 @@ % subject=subjectsForNumbers{j}; % fprintf('%s%i',subject,number) - subject='FS'; - number=3; + subject='MB'; + number=1; maxFile=5; threshold=10000; %EMG is classified as movement