\documentclass[a4paper, 12pt]{apa6}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage[american]{babel}
\usepackage[style=apa,sortcites=true,sorting=nyt,backend=biber]{biblatex}
\DeclareLanguageMapping{american}{american-apa}
\addbibresource{references.bib}
\usepackage{listings}
\title{Simulation of Grainger et al. (2012) with Rescorla Wagner equations}
\shorttitle{Grainger et al. (2012) simulation with RW equations}
\author{Robert Geirhos (3827808), Klara Grethen (3899962), \\David-Elias Künstle (3822829), Felicia Saar (3818590)}
\affiliation{Linguistics for Cognitive Science Course, University of Tübingen}
\abstract{our abstract goes here!}
\begin{document}
\maketitle
%\cite{}
\section{Introduction}
statement of the problem
\section{Simulations}
\subsection{Experimental Code}
why we didn't use the given code, what we improved, how the result is structured - Goal: modular and comprehensive experiment. Problems with paper and given code. What's a block in our experiment.\\
Since preliminary experiments showed that the monkeys performed with very high accuracies (>90\%), we decided to introduce a random parameter $ r $ in the experiment, defined as the fraction of times the monkey would make a random guess instead of an experience-based prediction.
\subsection{Choice of Parameters}
\subsubsection{Number of Trials}
The six monkeys in the original experiment participated in a different number of trials (min: 43.041, max: 61.142, mean: 52.812). For the sake of simplicity, we presented exactly 50.000 trials in each of our experiments.
\subsubsection{Random Parameter} The random parameter $ r $ was set to 0.65, which proved to be reasonable value. That means, in 65\% of the cases the monkey would guess for either word or nonword with equal probabilities. Therefore, the maximum possible performance $ p_{max} $ is:
$$ p_{max} = 1 - \frac{r}{2} = 0.675$$
In other words, the maximum possible performance is no longer 1.0 (for a very intelligent monkey) but rather restricted by $ r $. If a monkey's performance is slightly better than $ p_{max} $, this is assured to be due to chance.
\subsubsection{Alpha and Beta} Both $ \alpha $ and $ \beta $ were our independent variables which we manipulated over the course of the experiments. We gathered data for every possible combination of $ \alpha $ and $ \beta $ values within an equally spaced range from 0.0 to 0.3. A total of 15 values for each $ \alpha $ and $ \beta $ were combined to $ 15*15 = 225 $ possible combinations. Since $ \alpha $ and $ \beta $ were internally multiplied to a single value, we expected the outcome to be symmetrical due to the commutativity of the multiplication operation and therefore calculated each combination of $ \alpha $ and $ \beta $ only once, which we used as a trick to improve the overall runtime. Therefore, $\sum_{i=1}^{15}i = 120$ combinations remained to be explored.
\subsubsection{Lambda} Choice of $ \lambda $
%TODO explain choice of lambda=1, saliency
\subsection{Running Parallelized Experiments}
Running an experiment with a single combination of $ \alpha $ and $ \beta $ on a normal desktop computer took about 75 minutes. Therefore, the parameter space one could explore within a reasonable amount of time was quite restricted. We decided to write a parallelized version of the code to reduce the overall runtime. Using the R packages foreach, parallel and doParallel (TODO: Cite them properly), we restructured the experiment. Since conflicts can easily occur when more than one core is trying to access a shared data structure at the same time, we implemented a parallelized version that is able to run without even containing critical sections. Instead, each thread has its own data structure, a .txt file, and in the end the results are harvested and combined. This version of the experiment ran on a cluster with 15 cores, each performing a total amount of eight experiments. Altogether, 120 combinations of $ \alpha $ and $ \beta $ were explored overnight, which would have taken about 150 hours in a non-parallelized version.
\section{Results}
results
\begin{figure*}
\includegraphics[width=0.9\textwidth]{plots/plot_accuracy}
\end{figure*}
\begin{figure*}
\includegraphics[width=0.9\textwidth]{plots/plot_numwords}
\end{figure*}
\section{Discussion}
"your conclusions about what is most likely to underlie the different success rates of the baboons"
\appendix
\section{Complete Results}
Here are the complete results of our experiments. The abbreviations used are:
\begin{APAitemize}
\item \#Trials: Number of trials
\item \#LearnedW: Number of learned words
\item \#PresW: Number of presented words
\item GenAcc: General accuracy
\item WAcc: Word accuracy
\item NWAcc: Nonword accuracy
\end{APAitemize}
\onecolumn
\input{result_tables.tex}
\section{Code}
\input{code.tex}
\printbibliography{}
\end{document}
%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:
