CODE.ORG:
GAMIFICATION RESEARCH

PROBLEM:
How might we strategically integrate gamification features into the CSF (Computer Science Fundamentals) learning experience to enhance retention and attract new teachers?

METHOD 1: COMPETITIVE EVALUATION

What is the current online computer science learning landscape for elementary education?

Process:

Our team identified 7 direct, code-based competitors, and 7 indirect, (non-code-based) influencers with relevant gaming and education oriented applications.

Direct Competitors:
Minecraft Education, Tynker, CodeMonkey, Kodable, CodeSpark Academy, SpriteBox, Scratch Jr.

Indirect Competitors:
Prodigy, ABC Mouse, Duolingo, Duolingo Kids, Khan Academy, Khan Academy Kids, Brilliant

Then, we determined 9 gamification features that pertained to the Program Format, Individual Experience and Group Experiences from direct competitor apps.

Program Format:
Interactive Challenges, Creative Projects, Storytelling and Narrative, Time Challenges, and Hints and Tips

Individual Experiences:
Achievements and Badges, and Avatar Customization

Group Experiences:
Social Features, Ranking

Key Takeaways:

  1. Points System

  2. Customizable Avatars

  3. Various Difficulty Modes

  4. Sharing Project

  5. Pair Programming

  6. Awards

  7. Competition

7 gamification features were found on competitor platforms that are not currently used by Code.org.

These features represent opportunities for enhancement and implementation. These features are as follows:

3 gamification features were found on influencer platforms that are not currently used by Code.org.

These features represent opportunities for enhancement and implementation. These features are as follows:

  1. Program Host Characters

  2. Downloadable Lessons for Offline Use

  3. Brand Merch

METHOD 2: HEURISTIC EVALUATION

Are there any usability concerns in the existing CSF user experience among
the Code.org curricula that can be improved upon?

Process:

We assessed the CSF Express module (our curriculum in scope) and the AI Dance module (the newest curriculum of Code.org’s offerings) in both Desktop and iPad/Tablet formats.

Infractions were based on Jakob’s 10 Usability Heuristics by Nielsen Norman Group

Key Takeaways:

Among the 24 overlapping heuristic violations we identified, we condensed them into 4 primary concerns.

  1. Language and terminology used in the instructions, code, and game are inconsistent

  2. The responsive design for the iPad/Tablet interface fails to resize elements in a legible way

  3. The grid layout of the coding workspace, instructions list, help features, etc. is not visually similar to real-life coding software, divorcing the match between Code.org’s experience and the real-world

  4. Help modules are delivered inconsistently through location and brand voice, resulting in students getting stuck on lessons when in a self-paced learning environment

METHOD 3: LITERATURE REVIEW

What are best practices for teaching CS in a self-paced learning environment with minimal teaching support?

Process:

Factors such as student motivation, gamification of learning, teacher motivation, 
and factors that could affect a students’ learning, were researched in depth.

High Level Key Takeaways:

  1. Competence, control/autonomy, interest/value and relatedness are crucial pillars of intrinsically motivating students of all ages

  2. Teacher motivation is highly correlated with student motivation

  3. Top reported strategies for motivating students include: set clear goals, demonstrate the need for the lesson, provide liberal praise and limited criticism, integrate incentives, establish psychological safety in the classroom

  4. Gamification is highly successful in linking learning to fun, thus building an excitement for learning

METHOD 4: TEACHER INTERVIEWS

What problems currently exist within the CS Fundamentals Express Course from the point of view of teachers?

Process:

Through a series of 15 interviews with teachers who are current users of Code.org to taste test their reactions to the inclusion of more game-like features in the platform.

High Level Key Takeaways:

  1. Teachers did not emphasize the importance of measuring student performance or grading them. Instead, they were passionate about introducing their students to coding, as it is not often mandated to be taught in schools.

  2. There are three high level enhancements that teachers mentioned that could improve the CSF curriculum: 
Support and Inclusion, Motivation and Engagement 
and Feedback and Assessment.

  3. Integrating the CSF curriculum has been successful with interactive activities and offline warm-ups, but challenges include adapting resources and dealing with tech limitations.

  4. Teachers favor Code.org over its competitors due to its structured, user-friendly, and free platform that offers comprehensive, engaging, and progressive learning experiences.

  5. Teachers appreciate Code.org for its foundational skills and gamified content, but do not rely solely on it

METHOD 5: SURVEYS

Which potential gamification approaches from the competitive evaluation would be the most valuable to implement from the teacher’s and student’s perspective? 

Process:

199 teachers and 125 students were given separate surveys to understand which ideas from the competitive evaluation are the most valuable to implement that could help increase engagement and adoption

High Level Key Takeaways:

  1. The top three gamification features that appealed most to students and teachers were:
the Point System, Avatars, and Awards. 

  2. Notably, the awards feature statistically outperformed all other features among the students.

*While examining further data cuts, there wasn't a significant difference between the preferences of younger children and those of older students regarding the gamification features we presented.

METHOD 6: STUDENT PLAYTESTS

How do students currently engage with the platform and what are their major pain points with it?

Process:

We observed 6 classrooms; 125 students; Grades 1- 5 to learn how students interact with Code.org CSF Express modules and identify any pain points that can be addressed in future designs.

High Level Key Takeaways:

  1. The level of engagement decreases as age increases. 

  2. Younger students are more collaborative than older students. 2nd graders asked if they could work with a friend while older students (4th, 5th) worked quietly and independently for the most part 

  3. Younger students relied heavily on the text-to-voice feature & videos; older students prefer reading.

  4. Hints are not helpful and are rarely utilized. Students said hints are sometimes unrelated to their issue

  5. Many kids do not pay attention to the end completion screen.

  6. Some would immediately exit from the window, but then they were confused about where to go next. 

  7. Students often get frustrated when they have to wrap up and finish before completing a level.

  8. If kids have access to skipping the level, they will skip the level once it is challenging.

DELIVERABLE: PERSONAS AND CUSTOMER JOURNEY MAPS

How might we effectively represent our core users—their behaviors, challenges, and opportunities—to guide future feature integrations and modifications, ensuring maximum positive impact?

Process:

Based on data gathered in the methods above, we developed 7 representative user personas and expanded them into journey maps that locate areas for improvement throughout the learning process.

High Level Key Takeaways:

  1. Students benefit from tailored resources that address their individual learning styles. Features like help resources, self-paced learning, and bilingual support ensure that students, regardless of their skill level or language proficiency, can keep up and stay engaged.

  2. Students thrive in environments that allow both collaboration and competition. Tools that foster teamwork without disruption and competitive features like rankings or challenges can motivate different types of learners to stay engaged.

  3. Increased customization options for content and characters, along with connecting coding concepts to real-world applications, are essential for capturing student interest. This personalizes the learning experience and helps students see the relevance of their work.

  4. Positive reinforcement, such as awards and celebration of accomplishments, helps sustain motivation. Feedback systems that acknowledge progress without relying heavily on teacher supervision can improve independent learning and self-confidence.

  5. Accessible tools, like debugging features, video instructions, and visual hints, support independent learning by enabling students to overcome challenges on their own. This builds problem-solving skills and reduces frustration.

  6. Frequent opportunities for review, such as re-attempting quizzes or revisiting past projects, help reinforce learning. Warm-up exercises and simplified step-by-step instructions further support gradual concept mastery and skill retention.

  7. Features like creative projects for faster learners and locked levels for those who might skip ahead ensure that students at different paces remain engaged without compromising their learning journey.

Previous
Previous

Open: UX Analysis

Next
Next

Goodreads: Research Analysis