Developing Assessments to Determine Mastery of Programming Fundamentals

Current learning outcomes for introductory programming courses are relatively general, specifying tasks such as designing, implementing, testing and debugging programs that use some fundamental programming constructs. These outcomes impact what we teach, our expectations, and our assessments. Although prior work has demonstrated the utility of single concept assessments, most assessments used in formal examinations combine numerous heterogeneous concepts, resulting in complex and difficult tasks. As a consequence, teachers may not be able to diagnose the actual difficulties faced by students and students are not provided with accurate feedback about their achievements. Such limitations on the nature and quality of feedback to teachers and students alike may contribute to the perceived difficulty and high dropout rates commonly observed in introductory programming courses. In this paper we review the concepts that Computer Science education researchers have identified as important for novice programming. We survey expected learning outcomes for introductory programming courses, and analyse assessments designed for such courses to determine the individual components of syntax and semantics required to complete them. Having recognized the implicit and explicit expectations of novice programming courses, we look at the relationships between components, and progression between concepts. Finally, we demonstrate how some complex assessments can be decomposed into atomic elements that can be assessed independently.