\documentclass[a4paper, 12pt]{apa6}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage[american]{babel}
\usepackage[style=apa,sortcites=true,sorting=nyt,backend=biber]{biblatex}
\DeclareLanguageMapping{american}{american-apa}
\addbibresource{references.bib}

\usepackage{listings}
\usepackage{color} 

\usepackage{todonotes}
\title{Simulation of Grainger et al. (2012) with Rescorla Wagner equations}
\shorttitle{Grainger et al. (2012) simulation with RW equations}
\author{Robert Geirhos (3827808), Klara Grethen (3899962), \\David-Elias Künstle (3822829), Felicia Saar (3818590)}
\affiliation{Linguistics for Cognitive Science Course, University of Tübingen}

\abstract{TODO TODO TODO our abstract goes here TODO TODO TODO}

\lstset{ %
  basicstyle=\footnotesize,        % the size of the fonts that are used for the code
  captionpos=b,                    % sets the caption-position to bottom
  frame=single,	                   % adds a frame around the code
  keepspaces=true,                 % keeps spaces in text, useful for keeping indentation of code (possibly needs columns=flexible)
  keywordstyle=\color{blue},       % keyword style
  numbers=left,                    % where to put the line-numbers; possible values are (none, left, right)
  numbersep=5pt,                   % how far the line-numbers are from the code
  rulecolor=\color{black},         % if not set, the frame-color may be changed on line-breaks within not-black text (e.g. comments (green here))
  showspaces=false,                % show spaces everywhere adding particular underscores; it overrides 'showstringspaces'
  showstringspaces=false,          % underline spaces within strings only
  showtabs=false,                  % show tabs within strings adding particular underscores
  stepnumber=2,                    % the step between two line-numbers. If it's 1, each line will be numbered
  stringstyle=\color{mymauve},     % string literal style
  tabsize=2,	                   % sets default tabsize to 2 spaces
  title=\lstname                   % show the filename of files included with \lstinputlisting; also try caption instead of title
}

\begin{document}
\maketitle

%\cite{}
\section{Introduction}
\todo{statement of the problem}

\section{Simulations}
\subsection{Experimental Code}
\todo{why we didn't use the given code, what we improved, how the result is structured - Goal: modular and comprehensive experiment. Problems with paper and given code. What's a block in our experiment.}
Since preliminary experiments showed that the monkeys performed with very high accuracies (>90\%), we decided to introduce a random parameter $ r $ in the experiment, defined as the fraction of times the monkey would make a random guess instead of an experience-based prediction.

\subsection{Choice of Parameters}
\subsubsection{Number of Trials}
The six monkeys in the original experiment participated in a different number of trials (min: 43.041, max: 61.142, mean: 52.812). For the sake of simplicity, we presented exactly 50.000 trials in each of our experiments.
\subsubsection{Random Parameter} The random parameter $ r $ was set to 0.65, which proved to be reasonable value. That means, in 65\% of the cases the monkey would guess for either word or nonword with equal probabilities. Therefore, the maximum possible performance $ p_{max} $ is:
$$ p_{max} = 1 - \frac{r}{2} = 0.675$$
In other words, the maximum possible performance is no longer 1.0 (for a very intelligent monkey) but rather restricted by $ r $. If a monkey's performance is slightly better than $ p_{max} $, this is assured to be due to chance.
\subsubsection{Alpha and Beta} Both $ \alpha $ and $ \beta $ were our independent variables which we manipulated over the course of the experiments. We gathered data for every possible combination of $ \alpha $ and $ \beta $ values within an equally spaced range from 0.0 to 0.3. A total of 15 values for each $ \alpha $ and $ \beta $ were combined to $ 15*15 = 225 $ possible combinations. Since $ \alpha $ and $ \beta $ were internally multiplied to a single value, we expected the outcome to be symmetrical due to the commutativity of the multiplication operation and therefore calculated each combination of $ \alpha $ and $ \beta $ only once, which we used as a trick to improve the overall runtime. Therefore, $\sum_{i=1}^{15}i = 120$ combinations  remained to be explored.

\subsubsection{Lambda}
The independent variable $\lambda$ represents the maximum activation in the Rescorla-Wagner model and therefore limits the learning.
It makes it possible to modulate saliency of a stimulus. A more salient stimulus could not only have higher learning rates but also a higher maximum activation. In the original experiment the stimulus were same colored words and nonwords with four letters on a equally colored background. We assume the single words and nonwords are equally salient and keep therefore $\lambda$ constant (1). 

\subsection{Running Parallelized Experiments}
Running an experiment with a single combination of $ \alpha $ and $ \beta $ on a normal desktop computer took about 75 minutes. Therefore, the parameter space one could explore within a reasonable amount of time was quite restricted. We decided to write a parallelized version of the code to reduce the overall runtime. Using the R packages foreach, parallel and doParallel \todo{(TODO: Cite them properly)}, we restructured the experiment. Since conflicts can easily occur when more than one core is trying to access a shared data structure at the same time, we implemented a parallelized version that is able to run without even containing critical sections. Instead, each thread has its own data structure, a .txt file, and in the end the results are harvested and combined. This version of the experiment ran on a cluster with 15 cores, each performing a total amount of eight experiments. Altogether, 120 combinations of $ \alpha $ and $ \beta $ were explored overnight, which would have taken about 150 hours in a non-parallelized version.

\section{Results}
\todo{results}

\begin{figure*}
\includegraphics[width=0.9\textwidth]{plots/plot_accuracy}
\end{figure*}

\begin{figure*}
\includegraphics[width=0.9\textwidth]{plots/plot_numwords}
\end{figure*}


\section{Discussion}
\todo{"your conclusions about what is most likely to underlie the different success rates of the baboons"}

\appendix
\section{Complete Results}
Here are the complete results of our experiments. The abbreviations used are:
\begin{APAitemize}
\item \#Trials: Number of trials
\item \#LearnedW: Number of learned words
\item \#PresW: Number of presented words
\item GenAcc: General accuracy
\item WAcc: Word accuracy
\item NWAcc: Nonword accuracy
\end{APAitemize}

\onecolumn
\input{result_tables.tex}

\lstinputlisting[language=R]{../baboonSimulation.R}

\printbibliography{}

\end{document}
%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End: