The aim in this paper is to shed light on interactional aspects of researcher and practitioner collaboration in design research in mathematics education. Symbolic interactionism is used to gain understanding of interactional aspects as it has potential to take both individual and social aspects of the interaction into account. Aims and agencies are in focus of the retrospective analysis of the collaboration between two researchers and two practitioners as they collaborate to develop instructional design. The analysis show how referring to authoritative disciplines as the mathematics community influence agency and therefore has great potential to influence how the negotiation of meaning progress and participants acts. I argue that agency could be viewed as an indirect tool that has the potential to direct the collaboration when designing tasks based on what aim different actors put in the foreground.