diff --git a/07_final_assignment/paper/main.tex b/07_final_assignment/paper/main.tex index d3d959c..eb0c454 100644 --- a/07_final_assignment/paper/main.tex +++ b/07_final_assignment/paper/main.tex @@ -91,7 +91,7 @@ \subsection{Choice of Parameters} \subsubsection{Number of Trials} The six monkeys in the original experiment participated in a different number of trials (min: 43.041, max: 61.142, mean: 52.812). For the sake of simplicity, we presented exactly 50.000 trials in each of our experiments. -\subsubsection{Random Parameter} The random parameter $ r $ was set to 0.65, which proved to be reasonable value in preliminary experiments. That means, in 65\% of the cases the monkey would guess for either word or nonword with equal probabilities. Therefore, the maximum possible performance $ p_{max} $ is: +\subsubsection{Random Parameter}\label{sec:randparam} The random parameter $ r $ was set to 0.65, which proved to be reasonable value in preliminary experiments. That means, in 65\% of the cases the monkey would guess for either word or nonword with equal probabilities. Therefore, the maximum possible performance $ p_{max} $ is: $$ p_{max} = 1 - \frac{r}{2} = 0.675$$ In other words, the maximum possible performance is no longer 1.0 (for a very intelligent monkey) but rather restricted by $ r $. If a monkey's performance is slightly better than $ p_{max} $, this is assured to be due to chance. \subsubsection{Alpha and Beta} How fast a Cue-Outcome connection is learned or unlearned depends on a learn rate which determines which fraction of the activation difference will be added or removed per learn event. The learn rate in a event is the multiplication of the learn rate of the cue $\alpha$ and the learn rate of the outcome $\beta$. @@ -106,11 +106,19 @@ Running an experiment with a single combination of $ \alpha $ and $ \beta $ on a normal desktop computer took about 75 minutes. Therefore, the parameter space one could explore within a reasonable amount of time was quite restricted. We decided to write a parallelized version of the code to reduce the overall runtime. Using the R packages foreach \parencite{Rforeach}, parallel \parencite{Rparallel} and doParallel \parencite{RdoParallel}, restructured the experiment. Since conflicts can easily occur when more than one core is trying to access a shared data structure at the same time, we implemented a parallelized version that is able to run without even containing critical sections. Instead, each thread has its own data structure, a .txt file, and in the end the results are harvested and combined. This version of the experiment ran on a cluster with 15 cores, each performing a total amount of eight experiments. Altogether, 120 combinations of $ \alpha $ and $ \beta $ were explored overnight, which would have taken about 150 hours (!) in a non-parallelized version. \section{Results} -The number of words learned by the actual monkeys ranged between 87 and 308. With the chosen range for $\alpha$ and $\beta$, we obtained between 275 and 307 learned words, however, it is important to note that we only presented 307 words, so the model reached maximum learning potential. The general accuracy for the real monkeys lay between 71.14\% and 79.81\%, while our accuracies moved between 0.60 and 0.68. Accuracies for word and non-word decisions are similar in both cases. The complete result data is attached in the appendix of this paper. +The number of words learned by the actual monkeys ranged between 87 and 308. With the chosen range for $\alpha$ and $\beta$, we obtained between 275 and 307 learned words, however, it is important to note that we only presented 307 different words, so the model reached maximum learning potential even for small learn rates (see \ref{fig:numwords}). +The general accuracy for the real monkeys lay between 71.14\% and 79.81\%, while our accuracies moved between 60\% and 68\% with random parameter $r=0.65$ depending on used learn rates. +Because the absolute accuracy depends heavily on the random parameter (see \ref{sec:randparam}), we could easily match increase the accuracy with modifying it. A more interesting property is the range of word accuracy which is $.0867$ for the monkeys and $.08$ for the simulation. + +Using non-liner regression models (GAM) we find non linear ($df>1$) main effects for learn rates predicting the word or nonword accuracy with fixed random parameter without an interactive effect. +Because of the multiplicatory commutativity one of $\alpha$, $\beta$ is enough for explanation as learn rates here. In \ref{fig:accuracy}, second row we see the accuracies growing fast and converging to a plateau in one dimension with almost no effect of the other learn rate dimension. + +The complete result data is attached in the appendix of this paper. %TODO we need a section explaining the results of the plots. What does that mean? -> small influence of parameters as a major finding, however there ARE effects -> perhaps explain that along with the GAM, where we definitely have to explain what the predictors included in the model are: Otherwise it's black magic. I think it is crucial that we explain our findings (= our contribution): We explored the whole parameter space (which others probably couldn't), and we found this and that influence. \begin{figure*}[ht] + \onecolumn \centering \includegraphics[width=0.9\textwidth]{../plots/plot_accuracy} \caption{ @@ -121,9 +129,11 @@ This indicates that the results might probably be approximated with just one nonlinear parameter. } \label{fig:accuracy} + \twocolumn \end{figure*} \begin{figure*}[ht] + \onecolumn \centering \includegraphics[width=0.9\textwidth]{../plots/plot_numwords} \caption{ @@ -131,6 +141,7 @@ This is not necessarily a consequence of a wrong hypothesis but may rather be caused by the weak data with one very high frequency of around 305 learned words and many very low frequencies (right plot). } \label{fig:numwords} + \twocolumn \end{figure*}