Refereed Conference Proceedings

1. Hott, J. R. (2026). Providing Choice of Programming Language: Student Outcomes in an Algorithms Course. Proceedings of the 57th ACM Technical Symposium on Computer Science Education V.1, 463–469. https://doi.org/10.1145/3770762.3772520

   DOI BIB Abstract

   @inproceedings{10.1145/3770762.3772520,
     author = {Hott, John R.},
     title = {Providing Choice of Programming Language: Student Outcomes in an Algorithms Course},
     year = {2026},
     isbn = {9798400722561},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3770762.3772520},
     doi = {10.1145/3770762.3772520},
     booktitle = {Proceedings of the 57th ACM Technical Symposium on Computer Science Education V.1},
     pages = {463–469},
     numpages = {7},
     keywords = {computer science education, programming languages},
     location = {USA},
     series = {SIGCSE TS 2026}
   }
   

   Students learn multiple programming languages during their undergraduate studies in Computer Science. In some cases, students learn at least two languages during their first two courses, such as Java and Python. While the transition between languages early in the curriculum is well-studied and usually scripted, little is known about students’ language preferences and outcomes when given a choice in later courses. We provided students in a third-in-a-sequence major-required algorithms course the choice of language on each programming assignment (PA). Following a CS1 course in Python and a CS2 course in Java, students were asked to complete their PAs in either Java or Python, given equivalent scaffolding code. We conducted pre-course surveys of language preferences and analyzed the language use and resulting overall performance of 268 students across two semesters. On average, students had more self-reported familiarity and had taken more courses in Java, but felt Python had a better reputation. Additionally, while students tended to program in the language they were more familiar with, over 25% of students completed at least one PA in the other language. In two individual PAs (one per semester), students who used Python scored significantly higher than those using Java. However, there was no statistically significant difference in overall scores across PAs, problem sets, and quizzes based on their chosen PA language. Students also faced similar struggle—i.e., average number of submissions—on PAs regardless of language. Therefore, educators of upper-level courses should not worry about the impact of programming language choices on student outcomes.

2. Hott, J. R. (2025). Student Outcomes When Provided Programming Language Choice in an Algorithms Course. Proceedings of the 2025 ACM Conference on International Computing Education Research V.2, 26. https://doi.org/10.1145/3702653.3744316

   DOI BIB Abstract

   @inproceedings{10.1145/3702653.3744316,
     author = {Hott, John R},
     title = {Student Outcomes When Provided Programming Language Choice in an Algorithms Course},
     year = {2025},
     isbn = {9798400713415},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3702653.3744316},
     doi = {10.1145/3702653.3744316},
     booktitle = {Proceedings of the 2025 ACM Conference on International Computing Education Research V.2},
     pages = {26},
     numpages = {1},
     keywords = {computer science education, programming languages},
     location = {
     },
     series = {ICER '25}
   }
   

   Students typically experience multiple programming languages early in their Computer Science studies. Some programs have trended towards starting with languages like Python [2, 3, 4, 5] to facilitate learning while enabling instructors to include real-world and engaging examples in the CS1 classroom, such as asking students to write classifiers for cancer data or create games [4]. However, students may then be required to quickly transition to C++ [4], Java [5], or other languages as early as their second CS course. While several studies [3, 4] have shown that students make this transition fairly well, little evidence exists on student preferences and course outcomes in later courses when given a choice of language rather than the curriculum or instructor’s prescribed language.To address this gap, we examined programming language preference and performance of 268 students across two semesters in a third-in-a-sequence major-required algorithms course at the University of Virginia1. In this course, students were given a choice between two familiar languages on all programming assignments. More specifically, following a CS1 course in Python and a CS2 course in Java, students in Data Structures and Algorithms 2 (DSA2) were allowed to complete assignments in either Python or Java and were provided equivalent scaffolding code in both languages.We found that while students who chose to write in Python scored significantly higher on two programming assignments (one of five per semester), there was no statistically significant difference in overall outcomes or struggle—defined in Alzahrani et al [1] as the average number of submissions for programming assignments—between students who complete their programming assignments solely in Python, solely in Java, or a combination thereof. Additionally, there was no statistically significant difference in overall scores on programming assignments, written problem sets, or quizzes from the course based on the language students chose when implementing their solutions.From these results, we conclude that providing students with a choice of programming language, including allowing students to program in a language they are more familiar with, does not appear to dramatically improve student outcomes. Additionally, the use of Python over Java (or consequently Java over Python) in an upper-level algorithms course does not improve performance overall, even though it may provide some benefit in isolated assignments. Therefore, educators need not worry about how the programming language chosen for their courses may impact student outcomes.

3. Basit, N., Floryan, M., Hott, J. R., Huo, A., Le, J., & Zheng, I. (2025). ASCI: AI-Smart Classroom Initiative. Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1, 81–87. https://doi.org/10.1145/3641554.3701957

   DOI BIB Abstract

   @inproceedings{10.1145/3641554.3701957,
     author = {Basit, Nada and Floryan, Mark and Hott, John R. and Huo, Allen and Le, Jackson and Zheng, Ivan},
     title = {ASCI: AI-Smart Classroom Initiative},
     year = {2025},
     isbn = {9798400705311},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3641554.3701957},
     doi = {10.1145/3641554.3701957},
     booktitle = {Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1},
     pages = {81–87},
     numpages = {7},
     keywords = {computer science education, cosine similarity, group formation, office hours},
     location = {Pittsburgh, PA, USA},
     series = {SIGCSETS 2025}
   }
   

   The Artificial Intelligence Smart Classroom Initiative (ASCI) presents a re-imagined set of online course tools, designed primarily to support growing computer science classes. The system has four primary tools: an office hours queue, an automatic student grouping algorithm, a course-specific local large-language model (LLM), and administration tools for detecting students and TAs that need support. These tools interoperate to improve the quality of one another (e.g., LLM conversations support students directly in the office hours queue) and are enhanced by synchronizing data from multiple external sources such as Piazza, Gradescope, and Canvas. The system has been deployed in multiple courses over the past three semesters: initially as a FIFO queue, then supporting manual grouping and smart grouping of office hour attendees, and recently including LLM support. Preliminary results indicate that students who were grouped using the tool were more likely to return to the queue more than twice as often (on average) than those who were not. However, while grouping in office hours has the potential to decrease student wait times, teaching assistants and students tend to favor one-on-one meetings over group meetings. This might be improved in the future with updates to the software, TA training, and incorporation of other supporting tools (e.g., LLM technology). The other, newer, tools will be more thoroughly evaluated in future semesters.

4. Ruth, B., & Hott, J. R. (2025). Auto-grading in Computing Education: Perceptions and Use. Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1, 1008–1014. https://doi.org/10.1145/3641554.3701900

   DOI BIB Abstract

   @inproceedings{10.1145/3641554.3701900,
     author = {Ruth, Barrett and Hott, John R.},
     title = {Auto-grading in Computing Education: Perceptions and Use},
     year = {2025},
     isbn = {9798400705311},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3641554.3701900},
     doi = {10.1145/3641554.3701900},
     booktitle = {Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1},
     pages = {1008–1014},
     numpages = {7},
     keywords = {assessment, autograding, automatic grading, computing education},
     location = {Pittsburgh, PA, USA},
     series = {SIGCSETS 2025}
   }
   

   Auto-grading technologies have become increasingly prevalent in computing education, driven by the need to handle growing class sizes and provide timely and effective feedback. We conducted a survey of 44 computer science instructors at various institutions in order to gather instructor experience and use of auto-graders, the features instructors value most, and the challenges and limitations faced when using these tools. We specifically asked about factors such as grading strategies and policies, opinions on existing tools, and other automated grading methods they employ. Our results indicated that instructors prefer tools that offer significant customizability and integration capabilities, with functionality and program output-based grading as the most commonly used approaches. They emphasized the need for integrated auto-grading solutions that include robust core features and prioritize extensibility to better align with pedagogical goals and to support instructors in managing the increasing demands of computer science education. Based on these findings, we conclude that existing solutions should be improved to address instructor-reported preferences and diverse educational needs.

5. Hott, J. R. (2024). Analyzing Student Performance with Free Late Submission Days. Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 2, 1682–1683. https://doi.org/10.1145/3626253.3635562

   DOI BIB Abstract

   @inproceedings{10.1145/3626253.3635562,
     author = {Hott, John R.},
     title = {Analyzing Student Performance with Free Late Submission Days},
     year = {2024},
     isbn = {9798400704246},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3626253.3635562},
     doi = {10.1145/3626253.3635562},
     booktitle = {Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 2},
     pages = {1682–1683},
     numpages = {2},
     keywords = {assessment, computer science education, late policy},
     location = {Portland, OR, USA},
     series = {SIGCSE 2024}
   }
   

   We investigate the effects of a flexible late policy on student performance submission behavior across two semesters of a lower-level required course (LL) and an upper-level elective (UL). The first semester late policies in both courses were strict: LL incorporated a 10% penalty per day for two days; UL rarely allowed late submissions. In each course, the late policy was relaxed in the second semester to provide two no-penalty late days for each assignment.Our results show that in the LL class with the strict late policy, grades decrease each day past the deadline. When given two no-penalty late days, students in LL tended to submit their assignments later, with 41% of submissions coming after the deadline. While students’ homework scores were higher than their peers with the stricter policy (before penalties) on the first day after the deadline, their scores on the second day of submission were equivalent. With more late submissions, the overall homework scores in LL were lower with the no-penalty late days (controlling for late penalties). In contrast, students in UL submitting two days late in the second semester performed similarly to their peers in the first semester with the stricter penalty, while average scores for assignments submitted on (and around) the deadline improved dramatically. These results suggest that providing an extra day with no penalty is helpful for students, such as those making just-in-time submissions, but that additional days may lower performance overall for students in lower-level courses.

6. Hott, J. R., Floryan, M., & Basit, N. (2024). Towards More Efficient Office Hours for Large Courses: Using Cosine Similarity to Efficiently Construct Student Help Groups. Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 2, 1684–1685. https://doi.org/10.1145/3626253.3635544

   DOI BIB Abstract

   @inproceedings{10.1145/3626253.3635544,
     author = {Hott, John R. and Floryan, Mark and Basit, Nada},
     title = {Towards More Efficient Office Hours for Large Courses: Using Cosine Similarity to Efficiently Construct Student Help Groups},
     year = {2024},
     isbn = {9798400704246},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3626253.3635544},
     doi = {10.1145/3626253.3635544},
     booktitle = {Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 2},
     pages = {1684–1685},
     numpages = {2},
     keywords = {computer science education, cosine similarity, group formation, office hours},
     location = {Portland, OR, USA},
     series = {SIGCSE 2024}
   }
   

   As undergraduate enrollment in computer science rises, instructors continue to investigate methods to improve the student experience at scale. One aspect commonly used in courses at scale is queue-driven office hours, in which students join an online queue and meet with teaching assistants on a first-come, first-serve basis (FIFO).This poster introduces a novel office hours queue feature that automatically groups students in office hours using the cosine similarity metric across their reported issues provided upon joining the queue. Using real office hour attendance data from a 480-person undergraduate course as a basis for simulation, we find that moderate decreases in student wait time during the semester overall (11% on average) are possible, with more significant decreases possible on the busiest days (20% on average). This approach is suitable for real-world testing and these gains are possible without asking students to provide any additional information than they already do when attending office hours. Therefore, this work provides motivation for implementation of such an approach in future courses.

7. Lenfant, R., Wanner, A., Hott, J. R., & Pettit, R. (2023). Project-Based and Assignment-Based Courses: A Study of Piazza Engagement and Gender in Online Courses. Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1, 138–144. https://doi.org/10.1145/3587102.3588833

   DOI BIB Abstract

   @inproceedings{10.1145/3587102.3588833,
     author = {Lenfant, Ryan and Wanner, Alice and Hott, John R. and Pettit, Raymond},
     title = {Project-Based and Assignment-Based Courses: A Study of Piazza Engagement and Gender in Online Courses},
     year = {2023},
     isbn = {9798400701382},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3587102.3588833},
     doi = {10.1145/3587102.3588833},
     booktitle = {Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1},
     pages = {138–144},
     numpages = {7},
     keywords = {assignment-based learning, engagement, gender, online communities, peer parity, piazza, project-based learning},
     location = {Turku, Finland},
     series = {ITiCSE 2023}
   }
   

   Project-based (PB) learning has become increasingly popular in computer science education, particularly as studies have found that the teaching style better prepares students for future careers and improves learning outcomes through increased student engagement. Online forum usage is one measurable component of engagement. In order to study the impact of PB learning on online forum engagement, Piazza usage data from seven online computer science courses at a higher education institution were collected and examined. We analyzed the differences in online forum usage between PB and assignment-based (AB) learning, in addition to differences between men and women in each course type. Specifically, this study builds upon and replicates a previous study on Piazza that measured student engagement, anonymity usage, and peer parity. We found that students in PB courses were less actively engaged in online forums than students in AB courses; they were less likely to ask and answer questions on Piazza but were more likely to view posts and be logged on more days. Across both course types, students posted anonymously a similar amount as a proportion of the total number of questions and answers and experienced a proportionally similar amount of peer parity. Our findings mirror prior results on gender engagement on Piazza. Across both PB and AB courses, women were more engaged, asked and viewed more questions, posted anonymously more frequently, and were less likely to experience peer parity than men.

8. Hott, J. R., Basit, N., Gao, Z., Truslow, E., & Goulmamine, N. (2023). Providing a Choice of Time Trackers on Online Assessments. Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1, 715–721. https://doi.org/10.1145/3545945.3569776

   DOI BIB Abstract

   @inproceedings{10.1145/3545945.3569776,
     author = {Hott, John R. and Basit, Nada and Gao, Ziyao and Truslow, Ella and Goulmamine, Nour},
     title = {Providing a Choice of Time Trackers on Online Assessments},
     year = {2023},
     isbn = {9781450394314},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3545945.3569776},
     doi = {10.1145/3545945.3569776},
     booktitle = {Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1},
     pages = {715–721},
     numpages = {7},
     keywords = {computing education, online testing, test anxiety, time tracking devices, timers, user interface design},
     location = {Toronto ON, Canada},
     series = {SIGCSE 2023}
   }
   

   Online assessments allow instructors to facilitate exams and quizzes in both virtual and large classes. Having a clear online timer during these assessments is vital to help students manage their time. However, these same timers can be a cause of anxiety, affecting student performance. Our goals were to determine (i) which types of visualizations are currently in use, (ii) which styles of online timer were preferred by students, and (iii) if providing students a choice of timer impacted their performance.We carried out a semester-long study employing multiple time-tracking displays across 29 online quizzes in three Computer Science courses with a total of 113 student participants. Timer visualizations included count down and elapsed time text as a text-only display or combined with a graphical representation, such as a color-changing progress bar, gray-scale progress bar, or changing phases of the moon. Overwhelmingly students chose a time tracker that counted down the time left in the quiz, preferred graphical displays to text-only, and visualizations that changed color to better indicate the passing of time. Students who were given a choice on all assessments throughout the study typically picked and kept the same timer throughout or settled on a preferred timer after only a few assessments. Providing students that choice before their quiz had no significant effect on their performance relative to students who were not given a choice.These findings indicate that it is helpful to give students the choice of online timer, providing them a more accommodating and comfortable testing environment.

9. Blanchard, J., Hott, J. R., Berry, V., Carroll, R., Edmison, B., Glassey, R., Karnalim, O., Plancher, B., & Russell, S. (2022). Stop Reinventing the Wheel! Promoting Community Software in Computing Education. Proceedings of the 2022 Working Group Reports on Innovation and Technology in Computer Science Education, 261–292. https://doi.org/10.1145/3571785.3574129

   DOI BIB Abstract

   @inproceedings{10.1145/3571785.3574129,
     author = {Blanchard, Jeremiah and Hott, John R. and Berry, Vincent and Carroll, Rebecca and Edmison, Bob and Glassey, Richard and Karnalim, Oscar and Plancher, Brian and Russell, Se\'{a}n},
     title = {Stop Reinventing the Wheel! Promoting Community Software in Computing Education},
     year = {2022},
     isbn = {9798400700101},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3571785.3574129},
     doi = {10.1145/3571785.3574129},
     booktitle = {Proceedings of the 2022 Working Group Reports on Innovation and Technology in Computer Science Education},
     pages = {261–292},
     numpages = {32},
     keywords = {open source software, educational tools, computing education research, computing education, community software},
     location = {Dublin, Ireland},
     series = {ITiCSE-WGR '22}
   }
   

   Historically, computing instructors and researchers have developed a wide variety of tools to support teaching and educational research, including exam and code testing suites and data collection solutions. However, these tools often find limited adoption beyond their creators. As a result, it is common for many of the same functionalities to be re-implemented by different instructional groups within the Computing Education community. We hypothesise that this is due in part to discoverability, availability, and adaptability challenges. Further, instructors often face institutional barriers to deployment, which can include hesitance of institutions to rely on community developed solutions that often lack a centralised authority and may be community or individually maintained.  To this end, our working group explored what solutions are currently available, what instructors needed, and the reasons behind the above-mentioned phenomenon. To do so, we reviewed existing literature and surveyed the community to identify the tools that have been developed by the community; the solutions that are currently available and in use by instructors; what features are needed moving forward for classroom and research use; what support for extensions is needed to support further Computing Education research; and what institutional challenges instructors and researchers are currently facing or have faced in using community software solutions. Finally, the working group identified factors that limited adoption of solutions. This work proposes ways to integrate and improve the availability, discoverability, and dissemination of existing community projects, as well as ways to manage and overcome institutional challenges.

10. Blanchard, J., Hott, J. R., Berry, V., Carroll, R., Edmison, B., Glassey, R., Karnalim, O., Plancher, B., & Russell, S. (2022). Leveraging Community Software in CS Education to Avoid Reinventing the Wheel. Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 2, 580–581. https://doi.org/10.1145/3502717.3532169

    DOI BIB Abstract

    @inproceedings{10.1145/3502717.3532169,
      author = {Blanchard, Jeremiah and Hott, John R. and Berry, Vincent and Carroll, Rebecca and Edmison, Bob and Glassey, Richard and Karnalim, Oscar and Plancher, Brian and Russell, Se\'{a}n},
      title = {Leveraging Community Software in CS Education to Avoid Reinventing the Wheel},
      year = {2022},
      isbn = {9781450392006},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      url = {https://doi.org/10.1145/3502717.3532169},
      doi = {10.1145/3502717.3532169},
      booktitle = {Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 2},
      pages = {580–581},
      numpages = {2},
      keywords = {community software, educational tools},
      location = {Dublin, Ireland},
      series = {ITiCSE '22}
    }
    

    Historically, computing instructors and researchers have developed a wide variety of tools to support teaching and educational research, including exam and code testing suites and data collection solutions. Many are then community or individually maintained. However, these tools often find limited adoption beyond their creators. As a result, it is common for many of the same functionalities to be re-implemented by different instructional groups within the CS Education community. We hypothesize that this is due in part to accessibility, discoverability, and adaptability challenges, among others. Further, instructors often face institutional barriers to deployment, which can include hesitance of institutions to utilize community developed solutions that often lack a centralized authority. This working group will explore what solutions are currently available, what instructors need, and reasons behind the above-mentioned phenomenon. This will be accomplished via a literature review and survey to identify the tools that have been developed by the community; the solutions that are currently available and in use by instructors; what features are needed moving forward for classroom and research use; what support for extensions is needed to support further CS Education research; and what institutional challenges instructors and researchers are currently facing or have faced in the past in developing, deploying or otherwise using community software solutions. Finally, the working group will identify factors that limit adoption of solutions and ways to integrate and improve the accessibility, discoverability, and dissemination of existing community projects, as well as manage and overcome institutional challenges.

11. Hott, J. R., & Blanchard, J. (2022). Toward a Collaborative Open Source CS-focused Assessment Framework. Proceedings of the 53rd ACM Technical Symposium on Computer Science Education V. 2, 1194. https://doi.org/10.1145/3478432.3499199

    DOI BIB Abstract

    @inproceedings{10.1145/3478432.3499199,
      author = {Hott, John R. and Blanchard, Jeremiah},
      title = {Toward a Collaborative Open Source CS-focused Assessment Framework},
      year = {2022},
      isbn = {9781450390712},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      url = {https://doi.org/10.1145/3478432.3499199},
      doi = {10.1145/3478432.3499199},
      booktitle = {Proceedings of the 53rd ACM Technical Symposium on Computer Science Education V. 2},
      pages = {1194},
      numpages = {1},
      keywords = {tool development, open source, online assessment, assessment},
      location = {Providence, RI, USA},
      series = {SIGCSE 2022}
    }
    

    As computer science educators, many of us share the common challenge of assessing students’ programming skills. While a number of commercial services have popped up to fill this need, numerous members of the CS-Education community develop and maintain their own in-house tools. Commercial services may increase the costs to institutions and students, while instructor-developed tools are usually tailored to their own use cases; as a result, new instructors often, in turn, build their own highly-specialized solutions. This BOF is intended to spark a conversation about how we as a community can address this challenge by working together. In particular, we will discuss the availability of current open source tools and how they can be generalized, with a focus on editors, compilers, questionnaires, and scoring systems for assessments, including web-based and local-software solutions. Looking forward, we will discuss possible avenues to unite platforms and code bases, coordinate, and collaborate in the future-including student involvement as part of their coursework (e.g., capstone projects) to further develop platforms. Finally, by providing a dedicated session, we hope to advertise and disseminate the open source availability of these projects for those who seek a low- or no-cost solution for their students.

12. Truslow, E., Goulmamine, N., Hott, J. R., & Basit, N. (2022). Analyzing Student Experience of Time Trackers on Assessments. Proceedings of the 53rd ACM Technical Symposium on Computer Science Education V. 2, 1113. https://doi.org/10.1145/3478432.3499121

    DOI BIB Abstract

    @inproceedings{10.1145/3478432.3499121,
      author = {Truslow, Ella and Goulmamine, Nour and Hott, John R. and Basit, Nada},
      title = {Analyzing Student Experience of Time Trackers on Assessments},
      year = {2022},
      isbn = {9781450390712},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      url = {https://doi.org/10.1145/3478432.3499121},
      doi = {10.1145/3478432.3499121},
      booktitle = {Proceedings of the 53rd ACM Technical Symposium on Computer Science Education V. 2},
      pages = {1113},
      numpages = {1},
      keywords = {computing education, online testing, test anxiety, time tracking devices, timers, user interface design},
      location = {Providence, RI, USA},
      series = {SIGCSE 2022}
    }
    

    Visualizing time limits during online assessments is a cause of anxiety, affecting student performance. An initial survey of 34 students across two Computer Science courses found that time-tracking devices produced anxiety for 67.7% of students. While students differed on timer color preference, a majority preferred a count-down display showing time remaining with the ability to hide the timer. In a small pilot study across five exams, we employed multiple time-tracking displays. Preliminary data suggests that students presented with a count-down grayscale timer performed better on average than those presented with a green-yellow-red (GYR) version. Other displays, such as text-only digital count-down timer or elapsed time progress bars, did not elicit as large a difference in performance. These findings indicate the need for further study.

13. Siegel, A. A., Zarb, M., Alshaigy, B., Blanchard, J., Crick, T., Glassey, R., Hott, J. R., Latulipe, C., Riedesel, C., Senapathi, M., Simon, & Williams, D. (2022). Teaching through a Global Pandemic: Educational Landscapes Before, During and After COVID-19. Proceedings of the 2021 Working Group Reports on Innovation and Technology in Computer Science Education, 1–25. https://doi.org/10.1145/3502870.3506565

    DOI BIB Abstract

    @inproceedings{10.1145/3502870.3506565,
      author = {Siegel, Angela A. and Zarb, Mark and Alshaigy, Bedour and Blanchard, Jeremiah and Crick, Tom and Glassey, Richard and Hott, John R. and Latulipe, Celine and Riedesel, Charles and Senapathi, Mali and Simon and Williams, David},
      title = {Teaching through a Global Pandemic: Educational Landscapes Before, During and After COVID-19},
      year = {2022},
      isbn = {9781450392020},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      url = {https://doi.org/10.1145/3502870.3506565},
      doi = {10.1145/3502870.3506565},
      booktitle = {Proceedings of the 2021 Working Group Reports on Innovation and Technology in Computer Science Education},
      pages = {1–25},
      numpages = {25},
      keywords = {teaching, resilience, recovery, pandemic, online education, covid-19, coronavirus, computing education, computer science},
      location = {Virtual Event, Germany},
      series = {ITiCSE-WGR '21}
    }
    

    The coronavirus (COVID-19) pandemic has forced an unprecedented global shift within higher education in how instructors communicate with and educate students. This necessary paradigm shift has compelled educators to take a critical look at their teaching styles and use of technology. Computing education traditionally focuses on experiential, in-person activities. The pandemic has mandated that educators reconsider their use of student time and has catalysed overnight innovations in the educational setting.Even in the unlikely event that we return entirely to pre-pandemic norms, many new practices have emerged that offer valuable lessons to be carried forward into our post-COVID-19 teaching. This working group will explore what the post-COVID-19 academic landscape might look like, and how we can use lessons learned during this educational shift to improve our subsequent practice. Following a multinational study of computing faculty, this exploratory stage will identify practices within computing that appear to have been improved through exposure to online tools and technologies, and that should therefore continue to be used in the online space. In the broadest sense, our motivation is to explore what the post-COVID-19 educational landscape will look like for computing education.

14. Choi, E., Meng, L., & Hott, J. R. (2021). Open Source Software Practices in CS2. Proceedings of the 21st Koli Calling International Conference on Computing Education Research. https://doi.org/10.1145/3488042.3488047

    DOI BIB Abstract

    @inproceedings{10.1145/3488042.3488047,
      author = {Choi, Emma and Meng, Lisa and Hott, John R},
      title = {Open Source Software Practices in CS2},
      year = {2021},
      isbn = {9781450384889},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      url = {https://doi.org/10.1145/3488042.3488047},
      doi = {10.1145/3488042.3488047},
      booktitle = {Proceedings of the 21st Koli Calling International Conference on Computing Education Research},
      articleno = {18},
      numpages = {5},
      keywords = {CS2, Computer Science Education, Curriculum Development, Open Source Software},
      location = {Joensuu, Finland},
      series = {Koli Calling '21}
    }
    

    By contributing to open source software (OSS), students can gain professional software development experience and learn about applications of computer science (CS) concepts in pragmatic contexts. However, integrating such projects in classrooms requires substantial logistical planning by instructors as well as adequate programming skills from students. To mitigate these challenges, we propose four model curricula to serve as accessible strategies of integrating practicable learning opportunities in lower-level CS classes. Depending on classroom circumstances, instructors can assign projects that involve student contributions to OSS, custom plug-ins, simulated open source communities, or practical code excerpts. As a result, students will be able to explore the utility of CS and discover an exciting future in computing.

15. MacKenzie, C., & Hott, J. R. (2021). Extracting and Visualizing User Engagement on Wikipedia Talk Pages. Proceedings of the 17th International Symposium on Open Collaboration. https://doi.org/10.1145/3479986.3479995

    DOI BIB Abstract

    @inproceedings{10.1145/3479986.3479995,
      author = {MacKenzie, Carlin and Hott, John R},
      title = {Extracting and Visualizing User Engagement on Wikipedia Talk Pages},
      year = {2021},
      isbn = {9781450385008},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      url = {https://doi.org/10.1145/3479986.3479995},
      doi = {10.1145/3479986.3479995},
      booktitle = {Proceedings of the 17th International Symposium on Open Collaboration},
      articleno = {9},
      numpages = {12},
      keywords = {talk pages, data visualization, data extraction, classification, Wikipedia},
      location = {Online, Spain},
      series = {OpenSym '21}
    }
    

    As Wikipedia has grown in popularity, it is important to investigate its diverse user community and collaborative editorial base. Although all user data, from traffic to user edits, are available for download under a free and open license, it is difficult to work with this data due to its scale. In this paper, we demonstrate how consumer hardware can be used to create a local database of Wikipedia’s full edit history from their public XML data dumps. Using this database, we create and present the first visualizations of how editing on talk pages differs between user groups. Our visualizations demonstrate that low quality edits are primarily performed by IP users, rather than blocked users, and that overall engagement with talk pages has plateaued over the last 10 years across all user groups. Finally, we investigate the feasibility of classifying blocked users using this dataset as an example of future research directions. However, we demonstrate the difficulty of this task and find that additional data or a more advanced model would be needed to classify them, as our approach didn’t provide sufficient information to do this. We anticipate that our visualizations and data extraction process are of interest to the community and will provide researchers with the tools needed to use Wikipedia’s valuable data when resources are limited.

16. Siegel, A. A., Zarb, M., Alshaigy, B., Blanchard, J., Crick, T., Glassey, R., Hott, J. R., Latulipe, C., Riedesel, C., Senapathi, M., Simon, & Williams, D. (2021). Educational Landscapes During and After COVID-19. Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 2, 597–598. https://doi.org/10.1145/3456565.3461439

    DOI BIB Abstract

    @inproceedings{10.1145/3456565.3461439,
      author = {Siegel, Angela A. and Zarb, Mark and Alshaigy, Bedour and Blanchard, Jeremiah and Crick, Tom and Glassey, Richard and Hott, John R. and Latulipe, Celine and Riedesel, Charles and Senapathi, Mali and Simon and Williams, David},
      title = {Educational Landscapes During and After COVID-19},
      year = {2021},
      isbn = {9781450383974},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      url = {https://doi.org/10.1145/3456565.3461439},
      doi = {10.1145/3456565.3461439},
      booktitle = {Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 2},
      pages = {597–598},
      numpages = {2},
      keywords = {online education, coronavirus, computing education, COVID-19},
      location = {Virtual Event, Germany},
      series = {ITiCSE '21}
    }
    

    The coronavirus (COVID-19) pandemic has forced an unprecedented global shift within higher education in the ways that we communicate with and educate students. This necessary paradigm shift has compelled educators to take a critical look at their teaching styles and use of technology. Computing education traditionally focuses on experiential, in-person activities. The pandemic has mandated that educators reconsider their use of student time and has catalysed overnight innovations in the educational setting.Even in the unlikely event that we return entirely to pre-COVID-19 norms, many new practices have emerged that offer valuable lessons to be carried forward into our post-COVID-19 teaching. This working group will explore what the post-COVID-19 academic landscape might look like, and how we can use lessons learned during this educational shift to improve our subsequent practice. The exploration will strive to identify practices within computing that appear to have been improved through exposure to online tools and technologies, and that should therefore continue to be used in the online space. In the broadest sense, our motivation is to explore what the post-COVID-19 educational landscape will look like for computing education.

17. Thinnyun, A., Lenfant, R., Pettit, R., & Hott, J. R. (2021). Gender and Engagement in CS Courses on Piazza. Proceedings of the 52nd ACM Technical Symposium on Computer Science Education, 438–444. https://doi.org/10.1145/3408877.3432395

    DOI BIB Abstract

    @inproceedings{10.1145/3408877.3432395,
      author = {Thinnyun, Adrian and Lenfant, Ryan and Pettit, Raymond and Hott, John R.},
      title = {Gender and Engagement in CS Courses on Piazza},
      year = {2021},
      isbn = {9781450380621},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      url = {https://doi.org/10.1145/3408877.3432395},
      doi = {10.1145/3408877.3432395},
      booktitle = {Proceedings of the 52nd ACM Technical Symposium on Computer Science Education},
      pages = {438–444},
      numpages = {7},
      keywords = {social Q&A, piazza, peer parity, online communities, gender, females in computing},
      location = {Virtual Event, USA},
      series = {SIGCSE '21}
    }
    

    Online discussion forums are being increasingly used in classrooms as a way to encourage collaborative learning and community. Piazza is one such forum that was built specifically for academic institutions, and has been widely adopted. Students have the opportunity to ask questions and seek answers from peers and instructors alike online, allowing them to find the information they need even if they do not know fellow students in the class or if they cannot make an instructor’s office hours. However, recent analysis of the popular online discussion site Stack Overflow, suggests that women are more likely than men to withdraw from such a community if they do not identify other members of the same gender. Women are often a minority in computer science courses and may express difficulty interacting with or seeking help from their peers who are predominantly men. Considering the importance of providing equal access to students regardless of gender and the value of resources like Piazza in one’s education, it is imperative to assess the representation and impact of gender on Piazza. We analyzed data from over 2,500 Piazza users across three computer science courses at the University of Virginia and found that women on Piazza post more questions than men, spend more time on the discussion site, and achieve higher reputation scores on average. However, they are more likely than men to both ask and answer questions anonymously and less likely to receive responses from members of the same gender.

18. Lin, X., Connors, J., Lim, C., & Hott, J. R. (2021). How Do Students Collaborate? Analyzing Group Choice in a Collaborative Learning Environment. Proceedings of the 52nd ACM Technical Symposium on Computer Science Education, 212–218. https://doi.org/10.1145/3408877.3432389

    DOI BIB Abstract

    @inproceedings{10.1145/3408877.3432389,
      author = {Lin, Xinyue and Connors, James and Lim, Chang and Hott, John R.},
      title = {How Do Students Collaborate? Analyzing Group Choice in a Collaborative Learning Environment},
      year = {2021},
      isbn = {9781450380621},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      url = {https://doi.org/10.1145/3408877.3432389},
      doi = {10.1145/3408877.3432389},
      booktitle = {Proceedings of the 52nd ACM Technical Symposium on Computer Science Education},
      pages = {212–218},
      numpages = {7},
      keywords = {group formation, computer science education, collaborative learning, collaboration groups},
      location = {Virtual Event, USA},
      series = {SIGCSE '21}
    }
    

    Collaborative learning has been effective and widely adopted in Computer Science education. Existing studies have controlled for group sizes by assigning members to determine the optimal collaboration environment, with some focusing on a peer-programming environment and others observing a wider range of sizes and tasks.We analyzed collaboration trends through an observational study of 189 students in a large upper-level Computer Science algorithms course, which uses a less-constrained collaborative setting. In the course, the collaboration policy encourages students to choose their own groups for each assignment, up to four other students, offering insight into how groups evolve in size and membership when students are given the freedom to self-select. Since each student is required to submit their own individual work, we collected information about the grade and self-reported collaborators of each research participant for nine assignments, including written and coding homework.Our results show that any collaboration improved individual performance on average. For programming assignments, groups of size four were optimal. Across both written and programming assignments, larger groups performed better, including chains of collaboration greater than the course policy allowed. However, sizes 4-5 performed best within the bounds of the policy. We also demonstrate that factors impacting collaboration include homework difficulty, time of grade release, students’ relative performance with respect to the class, as well as the homework type.

19. Brunelle, N., & Hott, J. R. (2020). Ask Me Anything: Assessing Academic Dishonesty. Proceedings of the 51st ACM Technical Symposium on Computer Science Education, 1375. https://doi.org/10.1145/3328778.3372658

    DOI BIB Abstract

    @inproceedings{10.1145/3328778.3372658,
      author = {Brunelle, Nathan and Hott, John R.},
      title = {Ask Me Anything: Assessing Academic Dishonesty},
      year = {2020},
      isbn = {9781450367936},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      url = {https://doi.org/10.1145/3328778.3372658},
      doi = {10.1145/3328778.3372658},
      booktitle = {Proceedings of the 51st ACM Technical Symposium on Computer Science Education},
      pages = {1375},
      numpages = {1},
      keywords = {academic integrity, cheating, learning environment, undergraduate instruction},
      location = {Portland, OR, USA},
      series = {SIGCSE '20}
    }
    

    We provide a method for assessing self-reported rates of cheating among students. The method is both i) privacy-preserving in the sense that one cannot use answers as evidence that any particular student cheated and ii) non-anonymous in the sense that one can record each student’s answer for use in future correlative studies. Because accuracy relies on students’ willful participation, we describe how to convince students that they take no risk by taking the survey. This method showed that 42% of 847 students willfully cheated in an Algorithms course. Surveying 181 CS Theory students showed no difference in cheating rates on written vs. coding assignments.

20. Brunelle, N., & Hott, J. R. (2020). Fix the Course, Not the Student: Positive Approaches to Cultivating Academic Integrity. Proceedings of the 51st ACM Technical Symposium on Computer Science Education, 1402. https://doi.org/10.1145/3328778.3372535

    DOI BIB Abstract

    @inproceedings{10.1145/3328778.3372535,
      author = {Brunelle, Nathan and Hott, John R.},
      title = {Fix the Course, Not the Student: Positive Approaches to Cultivating Academic Integrity},
      year = {2020},
      isbn = {9781450367936},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      url = {https://doi.org/10.1145/3328778.3372535},
      doi = {10.1145/3328778.3372535},
      booktitle = {Proceedings of the 51st ACM Technical Symposium on Computer Science Education},
      pages = {1402},
      numpages = {1},
      keywords = {cheating, learning-environment, proactive-measures, undergraduate-instruction},
      location = {Portland, OR, USA},
      series = {SIGCSE '20}
    }
    

    The best-studied techniques for reducing academic dishonesty rates rely on increasing the likelihood of consequences. These techniques offer instructors effective tools for identifying dishonest behavior as a means to "encourage" honesty. We wonder if we can promote integrity through proactive measures, such as designing courses’ structures and assignments to reduce temptations for cheating, or by sculpting culture and forming relationships to foster a robust "community of trust." Our discussion will consider students’ perspectives on academic integrity, how those might differ from instructors’ perspectives, and how to build firm yet compassionate systems for promoting honesty in coursework.Through conversations with students and faculty, the discussion leaders have identified several compelling explanations for student cheating that largely derive from student culture and faculty messaging. We hope to share these lessons and inspire tactics for instructors to address student temptations to cheat that do not rely on the threat of penalty. This can help foster mentoring, rather than antagonistic, relationships between students and instructors, making computing courses increasingly welcoming for a greater diversity of students.

21. Noonan, R. E., & Hott, J. R. (2007). A course in software development. Proceedings of the 38th SIGCSE Technical Symposium on Computer Science Education, 135–139. https://doi.org/10.1145/1227310.1227362

    DOI BIB Abstract

    @inproceedings{10.1145/1227310.1227362,
      author = {Noonan, Robert E. and Hott, John R.},
      title = {A course in software development},
      year = {2007},
      isbn = {1595933611},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      url = {https://doi.org/10.1145/1227310.1227362},
      doi = {10.1145/1227310.1227362},
      booktitle = {Proceedings of the 38th SIGCSE Technical Symposium on Computer Science Education},
      pages = {135–139},
      numpages = {5},
      keywords = {object-oriented programming, laboratories, computer science education, CC2001},
      location = {Covington, Kentucky, USA},
      series = {SIGCSE '07}
    }
    

    The paper discusses a course in software development, as advocated by the CC2001 report. The course revolves around a single project divided into six assignments. In addition, the course includes lab assignments covering the tool of the week. The order of coverage of topics and the order of labs is determined using just-in-time learning. Grading criteria and an assessment of the course are discussed.

Peer-Reviewed Educational Materials

1. Hott, J. R., Basit, N., Graham, D., & Stone, D. (2022). Meme Magic: Project in Sprints.

   Abstract

   @book{10.1145/3519933,
     author = {Hott, John R. and Basit, Nada and Graham, Daniel and Stone, Derrick},
     title = {Meme Magic: Project in Sprints},
     year = {2022},
     isbn = {9781450394499},
     numpages = {4}
   }
   

   Meme Magic is a series of six assignments intended to provide progressive exposure to programming in Java using a popular and recent concept: Memes. Memes utilize an image conveying a concept or feeling with a caption provided by the Meme author. The series of assignments, designed as sprints in the context of a larger project, begin with the design and scaffolding of Java classes needed to write a program to produce text-based Memes and end with a fully-functional graphical user interface. For a detailed list of learning goals, please see the Learning Goals section. In the first sprint, students depict the overall project structure of a text-based meme application using Unified Markup Language (UML) and write method stubs in Java. In each of the next two sprints, students implement half of the specified functionality and integrate those components to a fully working application. Students are asked to add Comparators to sort memes to their application in sprint 4 and to unit test all of their code using JUnit in sprint 5. In the final sprint, students extend the functionality once more to a graphical user interface to experience event-driven programming. Once the full sequence is completed, students will be able to generate and save graphical memes. Steps and learning concepts include designing the project structure using UML diagrams, implementing that design, unit testing with JUnit, and event driven programming using Swing.