diff --git a/text/TODO.txt b/text/TODO.txt index 6160808..9d5e2b6 100644 --- a/text/TODO.txt +++ b/text/TODO.txt @@ -66,17 +66,16 @@ Calculations done: - offset (0,1,2) - pause (0,1) -- Pos (offset 0-2) Calculations todo: +- Pos (offset 0-2) - c - ridge - maxFreq - maxPerclass? -- noLFsample höher, Frequenzen testen? - +- noLFsample höher(10), Frequenzen testen? Aus EEG EMG vorhersagen (WL) (regression, dann corr) (running) Aus Synergien kin (running) -Aus EEG (über Synergien) kinematik (testing) +Aus EEG (über Synergien) kinematik Intro nur Motivation, Rest in II diff --git a/text/thesis/01Introduction.tex b/text/thesis/01Introduction.tex index ca5cc01..3032b41 100644 --- a/text/thesis/01Introduction.tex +++ b/text/thesis/01Introduction.tex @@ -1,7 +1,7 @@ \chapter{Introduction} \label{introduction} \section{Motivation} -\label{intro:matovation} +\label{intro:motivation} %TODO: More explanation, maybe better link to the topic \qq{Reading the mind} is something humanity is and always was exited about. Whatever one may think about the possibility of doing so as a human, computers have a chance to catch a glimpse of the (neuronal) activity in the human brain and interpret it.\\ Here we use electroencephalography (EEG) to record brain activity and try to predict arm movements from the data.\\ @@ -13,28 +13,6 @@ Using non-invasive methods like EEG makes it harder to get a good signal and determine its origin. However it lowers the risk of injuries and infections which makes it the method of choice for wide spread application (cf. \cite{Collinger13}). Modern versions of these caps even use dry electrodes which allow for more comfort without loosing predictive strength (cf. \cite{Yeung15}). So everybody may put on and off an EEG-cap without high costs (e.g. for surgery). Predicting synergies instead of positions or movement is nearer to the concept the nervous system uses. This should make them easier to predict while we can also use them to move an robotic arm or an quadrocopter. -\section{Scientific background} -\label{intro:background} - \subsection{BCIs} - The idea of BCIs began to spread in the 1970s when Vidal published his paper (\cite{Vidal73}).\\ - First approaches used invasive BCIs earlier in Animals (rodents and monkeys) later also in humans. Invasive BCIs in humans were mostly implanted when the human was under brain surgery for another reason like epilepsy. Problems of invasive BCIs are the need to cut through skull and dura mater. This can lead to infections and severe brain damage.\\ - An improvement were less invasive BCIs with e.g. ECoG which is placed below the skull but outside the dura which decreased the risk for infections massively.\\ - Measuring outside the skull entails even less risk, the dura and skull however lower the quality of the signal massively. With some improvements EEG has a spatial resolution of 2-3 cm (cf. \cite{Babiloni01}). This is quite bad compared to the single neuron one can observe with invasive methods. However we are more interested in the activity of areas then single cells for our task, so EEG meets our requirements here. %TODO: price, availability - \subsection{EEG} - When using EEG one measures the electrical fields on the scalp that are generated by activity of neurons in the brain. These measurements allow some interpretation about what is happening inside the skull. In our application we use the recorded currents directly to train a SVM or as predictor for regression.\\ - The frequencies typically used for movement prediction in EEG are about 8-24 Hz (\cite{Blokland15},\cite{Ahmadian13},\cite{Wang09}). - \subsection{Low Frequencies} - Another approach is looking at the low frequency features (below 2Hz) in the signal. This was done by Liu et al. (\cite{Liu11}) and Antelis et al. (\cite{Antelis13}) for example.\\ - Antelis et al. found correlations between hand movement and low frequency signal of $(0.29,0.15,0.37)$ in the dimensions respectively.\\ - We used this method here for comparison but did not find significant correlation with the kinematics. %TODO: significant? - \subsection{Synergies} - Movement of the arm (and other parts of the body) are under-determined meaning with given trajectory there are different muscle contractions possible. One idea how this problem could be solved by our nervous system are synergies. Proposed by Bernstein in 1967 (\cite{Bernstein67}) they describe the goal of the movement (e.g. the trajectory) instead of controlling single muscles. This would mean however that predicting the activity of single muscles from EEG is harder than predicting a synergy which in turn determines the contraction of muscles.\\ - Evidence for the use of synergies in the nervous system was found e.g. by Bizzi et al. (\cite{Bizzi08}) and Byadarhaly et al. (\cite{Byadarhaly12}). They also showed that synergies meet the necessary requirement to be able to build predictable trajectories.\\ - Synergies are usually gotten from EMG signal through a principal component analysis (PCA, cf. \ref{tool:pca}), non-negative matrix factorisation (NMF, cf. \ref{tool:nmf}) or autoencoders (a form of neuronal network, cf. \ref{intro:autoenc},\ref{tool:autoenc}). - \subsection{Autoencoders} - \label{intro:autoenc} - Autoencoders are a specific type of artificial neural networks (ANN). They work like typical ANNs by adjusting weights between the layers however they don't predict an unknown output but they predict their own input. What is interesting now is manipulating the size of the hidden layer where the size of the hidden layer has to be smaller than the input size. Now in the hidden layer the information of the input can be found in a condensed form (e.g. synergies instead of single muscle activity).\\ - Autoencoders have been successfully used by Spüler et al. to extract synergies from EMG (\cite{Spuler16}). Especially with a lower number of synergies autoencoders perform better than PCA or NMF because linear models fail to discover the agonist-antagonist relations that are typical for muscle movements. These however can be detected by autoencoders which allows for good estimations with half the synergies. \section{Overview} %TODO diff --git a/text/thesis/02MaterialsAndMethods.tex b/text/thesis/02MaterialsAndMethods.tex index 093b5a5..22351cc 100644 --- a/text/thesis/02MaterialsAndMethods.tex +++ b/text/thesis/02MaterialsAndMethods.tex @@ -1,7 +1,16 @@ \chapter{Materials and Methods} \label{mat} -\section{Tools} +\section{Scientific background} +\label{intro:background} + \subsection{BCIs} + The idea of BCIs began to spread in the 1970s when Vidal published his paper (\cite{Vidal73}).\\ + First approaches used invasive BCIs earlier in Animals (rodents and monkeys) later also in humans. Invasive BCIs in humans were mostly implanted when the human was under brain surgery for another reason like epilepsy. Problems of invasive BCIs are the need to cut through skull and dura mater. This can lead to infections and severe brain damage.\\ + An improvement were less invasive BCIs with e.g. ECoG which is placed below the skull but outside the dura which decreased the risk for infections massively.\\ + Measuring outside the skull entails even less risk, the dura and skull however lower the quality of the signal massively. With some improvements EEG has a spatial resolution of 2-3 cm (cf. \cite{Babiloni01}). This is quite bad compared to the single neuron one can observe with invasive methods. However we are more interested in the activity of areas then single cells for our task, so EEG meets our requirements here. %TODO: price, availability \subsection{EEG} + When using EEG one measures the electrical fields on the scalp that are generated by activity of neurons in the brain. These measurements allow some interpretation about what is happening inside the skull. In our application we use the recorded currents directly to train a SVM or as predictor for regression.\\ + The frequencies typically used for movement prediction in EEG are about 8-24 Hz (\cite{Blokland15},\cite{Ahmadian13},\cite{Wang09}).%TODO: merge + Electroencephalography (EEG) is often used for non-invasive BCIs because it's cheap and easier to use than e.g. fMRI. The electrodes have to be spread over the scalp. To allow for comparability there are standardized methods for this. These methods also bring a naming convention with them. \subsubsection{10-20 system} In this standard adjacent electrodes are placed either 10\% or 20\% of the total front-back or left-right distance apart. This standardization also makes it possible to name each electrode or rather here place. This is done with capital letters for lobes (Frontal, \qq{Central}, Parietal, Occipital and Temporal) and numbers for the specific place on the lobe. Even numbers are on the right side of the head, odd on the left; larger numbers are closer to the ears, lower numbers closer to the other hemisphere. The exact number now refers to the exact position: $$\left\lceil\frac{x}{2}\right\rceil\cdot \frac{d}{10}$$ where $x$ is the number and $d$ the diameter of the scalp. Electrodes in the centre are named with a lower case $z$ e.g. $Cz$.\\ %TODO: figure @@ -17,8 +26,20 @@ \label{mat:pburg} For power spectral density estimation (PSD) of the EEG signal we used MATLAB's pburg method. This uses Burg's method, a parametrized method of PSD.\\ %... see bookmarks + \subsection{Low Frequencies} + Another approach is looking at the low frequency features (below 2Hz) in the signal. This was done by Liu et al. (\cite{Liu11}) and Antelis et al. (\cite{Antelis13}) for example.\\ + Antelis et al. found correlations between hand movement and low frequency signal of $(0.29,0.15,0.37)$ in the dimensions respectively.\\ + We used this method here for comparison but did not find significant correlation with the kinematics. %TODO: significant? \subsection{EMG} - Electromyography (EMG) is used to track muscle activity. This is done by measuring the electrical fields on the skin generated by muscle contraction. + Electromyography (EMG) is used to track muscle activity. This is done by measuring the electrical fields on the skin generated by muscle contraction. %TODO + \subsection{Synergies} + Movement of the arm (and other parts of the body) are under-determined meaning with given trajectory there are different muscle contractions possible. One idea how this problem could be solved by our nervous system are synergies. Proposed by Bernstein in 1967 (\cite{Bernstein67}) they describe the goal of the movement (e.g. the trajectory) instead of controlling single muscles. This would mean however that predicting the activity of single muscles from EEG is harder than predicting a synergy which in turn determines the contraction of muscles.\\ + Evidence for the use of synergies in the nervous system was found e.g. by Bizzi et al. (\cite{Bizzi08}) and Byadarhaly et al. (\cite{Byadarhaly12}). They also showed that synergies meet the necessary requirement to be able to build predictable trajectories.\\ + Synergies are usually gotten from EMG signal through a principal component analysis (PCA, cf. \ref{tool:pca}), non-negative matrix factorisation (NMF, cf. \ref{tool:nmf}) or autoencoders (a form of neuronal network, cf. \ref{intro:autoenc},\ref{tool:autoenc}). + \subsection{Autoencoders} + \label{intro:autoenc} + Autoencoders are a specific type of artificial neural networks (ANN). They work like typical ANNs by adjusting weights between the layers however they don't predict an unknown output but they predict their own input. What is interesting now is manipulating the size of the hidden layer where the size of the hidden layer has to be smaller than the input size. Now in the hidden layer the information of the input can be found in a condensed form (e.g. synergies instead of single muscle activity).\\ + Autoencoders have been successfully used by Spüler et al. to extract synergies from EMG (\cite{Spuler16}). Especially with a lower number of synergies autoencoders perform better than PCA or NMF because linear models fail to discover the agonist-antagonist relations that are typical for muscle movements. These however can be detected by autoencoders which allows for good estimations with half the synergies. \subsection{PCA} \label{tool:pca} \subsection{NMF} diff --git a/text/thesis/mylit.bib b/text/thesis/mylit.bib index 811b5e7..9078a81 100755 --- a/text/thesis/mylit.bib +++ b/text/thesis/mylit.bib @@ -110,6 +110,16 @@ } +@inproceedings{Sarasola15, + author = {A. Sarasola-Sanz, N. Irastorza-Landa, F. Shiman, E. Lopez-Larraz, M. Spüler, N. Birbaumer, A. Ramos-Murguialday}, + title = {EMG-based multi-joint kinematics decoding for robot-aided rehabilitation therapies}, + booktitle = {IEEE International Conference on Rehabilitation Robotics (ICORR)}, + year = {2015}, + month = {08}, + pages = {229-234}, +} + + @article{Ting07, author = "Lena H. Ting and J Lucas McKay", diff --git a/text/thesis/outline.txt b/text/thesis/outline.txt index 5c281d5..5191ee1 100644 --- a/text/thesis/outline.txt +++ b/text/thesis/outline.txt @@ -26,7 +26,7 @@ - Neuronal Networks - PCA - NNMF -- Experiment data came from +- Experiment data came from cooperation with Medical Psychology ( Andrea Sarasola-Sanz ; Nerea Irastorza-Landa ; Farid Shiman; 2015) - My history - load data - Where to cut? diff --git a/usedMcode/callAll.m b/usedMcode/callAll.m index 4b8f10a..38e90ad 100644 --- a/usedMcode/callAll.m +++ b/usedMcode/callAll.m @@ -77,9 +77,12 @@ correlationEMGemg=zeros([j,noSynergies]); maxRidgeParamIndexAutoencKin=zeros([j,noSynergies,k]); correlationAutoencKin=zeros([j,noSynergies]); - correlationViaAutoenc=zeros([j,noSynergies]); - correlationViaPCA=zeros([j,noSynergies]); - correlationViaNNMF=zeros([j,noSynergies]); + correlationViaAutoenc=zeros(j); + correlationViaPCA=zeros(j); + correlationViaNNMF=zeros(j); + viaCorrelationAutoenc=zeros(j,noSynergies); + viaCorrelationPCA=zeros(j,noSynergies); + viaCorrelationNNMF=zeros(j,noSynergies); parfor j=1:size(numbersMat,2) number=numbersMat(j); @@ -102,9 +105,9 @@ % [correlationLFnnmf(j,:),maxRidgeParamIndexLFnnmf(j,:,:)]=ridgeCV(savePath,'LF','NNMF',k,ridgeParams,eegOffset); [correlationEEGemg(j,:),maxRidgeParamIndexEEGemg(j,:,:)]=ridgeCV(savePath,'EEG','EMG',k,ridgeParams,eegOffset); [correlationAutoencKin(j,:),maxRidgeParamIndexAutoencKin(j,:,:)]=ridgeCV(savePath,'Autoenc','kin',k,ridgeParams,eegOffset); - correlationViaAutoenc(j)=ridgeCVvia(savePath,'EEG','Autoenc','kin',k,ridgeParams,eegOffset); - correlationViaPCA(j)=ridgeCVvia(savePath,'EEG','PCA','kin',k,ridgeParams,eegOffset); - correlationViaNNMF(j)=ridgeCVvia(savePath,'EEG','NNMF','kin',k,ridgeParams,eegOffset); + [correlationViaAutoenc(j),viaCorrelationAutoenc(j,:)]=ridgeCVvia(savePath,'EEG','Autoenc','kin',k,ridgeParams,eegOffset); + [correlationViaPCA(j),viaCorrelationPCA(j,:)]=ridgeCVvia(savePath,'EEG','PCA','kin',k,ridgeParams,eegOffset); + [correlationViaNNMF(j),viaCorrelationNNMF(j,:)]=ridgeCVvia(savePath,'EEG','NNMF','kin',k,ridgeParams,eegOffset); fprintf('%s%i finished %s\n',subject,number,datestr(datetime('now'))) end @@ -131,11 +134,12 @@ % [correlationEMGnnmf,maxRidgeParamIndexEMGnnmf(:,:)]=ridgeCV(savePath,'EMG','NNMF',k,ridgeParams,eegOffset); % [correlationEEGnnmf,maxRidgeParamIndexEEGnnmf(:,:)]=ridgeCV(savePath,'EEG','NNMF',k,ridgeParams,eegOffset); % [correlationLFnnmf,maxRidgeParamIndexLFnnmf(:,:)]=ridgeCV(savePath,'LF','NNMF',k,ridgeParams,eegOffset); -% [correlationEEGemg,maxRidgeParamIndexEEGemg(:,:)]=ridgeCV(savePath,'EEG','EMG',k,ridgeParams,eegOffset); -% [correlationAutoencKin,maxRidgeParamIndexAutoencKin(:,:)]=ridgeCV(savePath,'Autoenc','kin',k,ridgeParams,eegOffset); - correlationViaAutoenc=ridgeCVvia(savePath,'EEG','Autoenc','kin',k,[100],eegOffset); - correlationViaPCA=ridgeCVvia(savePath,'EEG','PCA','kin',k,[100],eegOffset); - correlationViaNNMF=ridgeCVvia(savePath,'EEG','NNMF','kin',k,[100],eegOffset); + [correlationEEGemg,maxRidgeParamIndexEEGemg(:,:)]=ridgeCV(savePath,'EEG','EMG',k,ridgeParams,eegOffset); + [correlationAutoencKin,maxRidgeParamIndexAutoencKin(:,:)]=ridgeCV(savePath,'Autoenc','kin',k,ridgeParams,eegOffset); + disp('begin') + [correlationViaAutoenc,viaCorrelationAutoenc]=ridgeCVvia(savePath,'EEG','Autoenc','kin',k,[100],eegOffset); + [correlationViaPCA,viaCorrelationPCA]=ridgeCVvia(savePath,'EEG','PCA','kin',k,[100],eegOffset); + [correlationViaNNMF,viaCorrelationNNMF]=ridgeCVvia(savePath,'EEG','NNMF','kin',k,[100],eegOffset); fprintf('%s%i finished %s\n',subject,number,datestr(datetime('now'))) save(strcat(pathToFile,sprintf('../matlabData/%s_call%s%i-%s.mat',datestr(datetime('now')),subject,number,name))); diff --git a/usedMcode/ridgeCVvia.m b/usedMcode/ridgeCVvia.m index 25e5d7d..af0ff60 100644 --- a/usedMcode/ridgeCVvia.m +++ b/usedMcode/ridgeCVvia.m @@ -1,4 +1,4 @@ -function [correlation]=ridgeCVvia(savePath,data,via,goal,k,ridgeParams,eegOffset) +function [correlation,viaCorrelation]=ridgeCVvia(savePath,data,via,goal,k,ridgeParams,eegOffset) load(savePath); clear classification; eeg=false; @@ -49,6 +49,7 @@ viaDataPredicted=zeros(size(viaData)); correlation=zeros(size(predicted,2)); + viaCorrelation=zeros(size(predicted,2)); parfor j=1:size(viaData,2) pred=viaData(:,j); @@ -56,7 +57,7 @@ coeffs=ridge(pred,trainingData,ridgeParams(1),0); viaDataPredicted(:,j)=coeffs(1)+trainingData*coeffs(2:end); - + viaCorrelation(:,j)=corr(viaData(:,j),viaDataPredicted(:,j)); end parfor j=1:size(predicted,2) diff --git a/usedMcode/shiftingMean.m b/usedMcode/shiftingMean.m index 7c417ae..edc2517 100644 --- a/usedMcode/shiftingMean.m +++ b/usedMcode/shiftingMean.m @@ -2,7 +2,7 @@ %shiftingMean downsampling by taking the mean % taking the mean of $factor data as new datum dataSize=size(data); - output=zeros([fix(ceil(dataSize(1)/factor)),dataSize(2:end)]); + output=zeros([fix(dataSize(1)/factor),dataSize(2:end)]); for i=1:size(output,1) output(i,:)=mean(data(fix(round((i-1)*factor))+1:min(fix(round(i*factor)),dataSize(1)),:),1); end