CODE.ORG:
GAMIFICATION RESEARCH
PROBLEM:
How might we strategically integrate gamification features into the CSF (Computer Science Fundamentals) learning experience to enhance retention and attract new teachers?
METHOD 1: COMPETITIVE EVALUATION
What is the current online computer science learning landscape for elementary education?
Process:
Our team identified 7 direct, code-based competitors, and 7 indirect, (non-code-based) influencers with relevant gaming and education oriented applications.
Direct Competitors:
Minecraft Education, Tynker, CodeMonkey, Kodable, CodeSpark Academy, SpriteBox, Scratch Jr.
Indirect Competitors:
Prodigy, ABC Mouse, Duolingo, Duolingo Kids, Khan Academy, Khan Academy Kids, Brilliant
Then, we determined 9 gamification features that pertained to the Program Format, Individual Experience and Group Experiences from direct competitor apps.
Program Format:
Interactive Challenges, Creative Projects, Storytelling and Narrative, Time Challenges, and Hints and Tips
Individual Experiences:
Achievements and Badges, and Avatar Customization
Group Experiences:
Social Features, Ranking
Key Takeaways:
Points System
Customizable Avatars
Various Difficulty Modes
Sharing Project
Pair Programming
Awards
Competition
7 gamification features were found on competitor platforms that are not currently used by Code.org.
These features represent opportunities for enhancement and implementation. These features are as follows:
3 gamification features were found on influencer platforms that are not currently used by Code.org.
These features represent opportunities for enhancement and implementation. These features are as follows:
Program Host Characters
Downloadable Lessons for Offline Use
Brand Merch
METHOD 2: HEURISTIC EVALUATION
Are there any usability concerns in the existing CSF user experience among
the Code.org curricula that can be improved upon?
Process:
We assessed the CSF Express module (our curriculum in scope) and the AI Dance module (the newest curriculum of Code.org’s offerings) in both Desktop and iPad/Tablet formats.
Infractions were based on Jakob’s 10 Usability Heuristics by Nielsen Norman Group
Key Takeaways:
Among the 24 overlapping heuristic violations we identified, we condensed them into 4 primary concerns.
Language and terminology used in the instructions, code, and game are inconsistent
The responsive design for the iPad/Tablet interface fails to resize elements in a legible way
The grid layout of the coding workspace, instructions list, help features, etc. is not visually similar to real-life coding software, divorcing the match between Code.org’s experience and the real-world
Help modules are delivered inconsistently through location and brand voice, resulting in students getting stuck on lessons when in a self-paced learning environment
METHOD 3: LITERATURE REVIEW
What are best practices for teaching CS in a self-paced learning environment with minimal teaching support?
Process:
Factors such as student motivation, gamification of learning, teacher motivation, and factors that could affect a students’ learning, were researched in depth.
High Level Key Takeaways:
Competence, control/autonomy, interest/value and relatedness are crucial pillars of intrinsically motivating students of all ages
Teacher motivation is highly correlated with student motivation
Top reported strategies for motivating students include: set clear goals, demonstrate the need for the lesson, provide liberal praise and limited criticism, integrate incentives, establish psychological safety in the classroom
Gamification is highly successful in linking learning to fun, thus building an excitement for learning
METHOD 4: TEACHER INTERVIEWS
What problems currently exist within the CS Fundamentals Express Course from the point of view of teachers?
Process:
Through a series of 15 interviews with teachers who are current users of Code.org to taste test their reactions to the inclusion of more game-like features in the platform.
High Level Key Takeaways:
Teachers did not emphasize the importance of measuring student performance or grading them. Instead, they were passionate about introducing their students to coding, as it is not often mandated to be taught in schools.
There are three high level enhancements that teachers mentioned that could improve the CSF curriculum: Support and Inclusion, Motivation and Engagement and Feedback and Assessment.
Integrating the CSF curriculum has been successful with interactive activities and offline warm-ups, but challenges include adapting resources and dealing with tech limitations.
Teachers favor Code.org over its competitors due to its structured, user-friendly, and free platform that offers comprehensive, engaging, and progressive learning experiences.
Teachers appreciate Code.org for its foundational skills and gamified content, but do not rely solely on it
METHOD 5: SURVEYS
Which potential gamification approaches from the competitive evaluation would be the most valuable to implement from the teacher’s and student’s perspective?
Process:
199 teachers and 125 students were given separate surveys to understand which ideas from the competitive evaluation are the most valuable to implement that could help increase engagement and adoption
High Level Key Takeaways:
The top three gamification features that appealed most to students and teachers were: the Point System, Avatars, and Awards.
Notably, the awards feature statistically outperformed all other features among the students.
*While examining further data cuts, there wasn't a significant difference between the preferences of younger children and those of older students regarding the gamification features we presented.
METHOD 6: STUDENT PLAYTESTS
How do students currently engage with the platform and what are their major pain points with it?
Process:
We observed 6 classrooms; 125 students; Grades 1- 5 to learn how students interact with Code.org CSF Express modules and identify any pain points that can be addressed in future designs.
High Level Key Takeaways:
The level of engagement decreases as age increases.
Younger students are more collaborative than older students. 2nd graders asked if they could work with a friend while older students (4th, 5th) worked quietly and independently for the most part
Younger students relied heavily on the text-to-voice feature & videos; older students prefer reading.
Hints are not helpful and are rarely utilized. Students said hints are sometimes unrelated to their issue
Many kids do not pay attention to the end completion screen.
Some would immediately exit from the window, but then they were confused about where to go next.
Students often get frustrated when they have to wrap up and finish before completing a level.
If kids have access to skipping the level, they will skip the level once it is challenging.
DELIVERABLE: PERSONAS AND CUSTOMER JOURNEY MAPS
How might we effectively represent our core users—their behaviors, challenges, and opportunities—to guide future feature integrations and modifications, ensuring maximum positive impact?
Process:
Based on data gathered in the methods above, we developed 7 representative user personas and expanded them into journey maps that locate areas for improvement throughout the learning process.
High Level Key Takeaways:
Students benefit from tailored resources that address their individual learning styles. Features like help resources, self-paced learning, and bilingual support ensure that students, regardless of their skill level or language proficiency, can keep up and stay engaged.
Students thrive in environments that allow both collaboration and competition. Tools that foster teamwork without disruption and competitive features like rankings or challenges can motivate different types of learners to stay engaged.
Increased customization options for content and characters, along with connecting coding concepts to real-world applications, are essential for capturing student interest. This personalizes the learning experience and helps students see the relevance of their work.
Positive reinforcement, such as awards and celebration of accomplishments, helps sustain motivation. Feedback systems that acknowledge progress without relying heavily on teacher supervision can improve independent learning and self-confidence.
Accessible tools, like debugging features, video instructions, and visual hints, support independent learning by enabling students to overcome challenges on their own. This builds problem-solving skills and reduces frustration.
Frequent opportunities for review, such as re-attempting quizzes or revisiting past projects, help reinforce learning. Warm-up exercises and simplified step-by-step instructions further support gradual concept mastery and skill retention.
Features like creative projects for faster learners and locked levels for those who might skip ahead ensure that students at different paces remain engaged without compromising their learning journey.