Nowadays, Artificial Intelligence (AI) is proving to be successful for solving complex problems in various application domains. However, despite the numerous success stories of AI-systems, one challenge that characterizes these systems is that they often lack transparency in terms of understandability and explainability. In this study, we propose to address this challenge from the design thinking lens as a way to amplify human understanding of ML (Machine Learning) and AI algorithms. We exemplify our proposed approach by depicting a case based on a conventional ML algorithm applied on sentiment analysis of students’ feedback. This paper aims to contribute to the overall discourse of a need of innovation when it comes to the understandability and explainability of ML and AI solutions, especially since innovation is an inherent feature of design thinking.