The aim of the PsyIntEC project is to explore affective and cognitive modeling of humans in human-robot interaction (HRI) as a basis for behavioral adaptation. To achieve this we have explored human affective perception of relevant modalities in human-human and human-robot interaction on a collaborative problem-solving task using psychophysiological measurements. The experiments conducted have given us valuable insight into the communicational and affective queues interplaying in such interactions from the human perspective. The results indicate that there is an increase in both positive and negative emotions when interacting with robots compared to interacting with another human or solving the task alone, but detailed analysis on shorter time segments is required for the results from all sensors to be conclusive and significant.