no code implementations • 14 Apr 2024 • Arav Agarwal, Karthik Mittal, Aidan Doyle, Pragnya Sridhar, Zipiao Wan, Jacob Arthur Doughty, Jaromir Savelka, Majd Sakr
We conduct a preliminary study of the effect of GPT's temperature parameter on the diversity of GPT4-generated questions.
no code implementations • 5 Dec 2023 • Jacob Doughty, Zipiao Wan, Anishka Bompelli, Jubahed Qayum, Taozhi Wang, Juran Zhang, Yujia Zheng, Aidan Doyle, Pragnya Sridhar, Arav Agarwal, Christopher Bogart, Eric Keylor, Can Kultur, Jaromir Savelka, Majd Sakr
While there is a growing body of research in computing education on utilizing large language models (LLMs) in generation and engagement with coding exercises, the use of LLMs for generating programming MCQs has not been extensively explored.
no code implementations • 30 Jun 2023 • Pragnya Sridhar, Aidan Doyle, Arav Agarwal, Christopher Bogart, Jaromir Savelka, Majd Sakr
We evaluated 127 LOs that were automatically generated based on a carefully crafted prompt (detailed guidelines on high-quality LOs authoring) submitted to GPT-4 for conceptual modules and projects of an AI Practitioner course.
1 code implementation • 28 Jun 2023 • Paul Pu Liang, Yiwei Lyu, Xiang Fan, Arav Agarwal, Yun Cheng, Louis-Philippe Morency, Ruslan Salakhutdinov
Learning multimodal representations involves integrating information from multiple heterogeneous sources of data.
no code implementations • 15 Jun 2023 • Jaromir Savelka, Arav Agarwal, Marshall An, Chris Bogart, Majd Sakr
Additionally, we analyze the assessments that were not handled well by GPT-4 to understand the current limitations of the model, as well as its capabilities to leverage feedback provided by an auto-grader.
no code implementations • 16 Mar 2023 • Jaromir Savelka, Arav Agarwal, Christopher Bogart, YiFan Song, Majd Sakr
We evaluated the capability of generative pre-trained transformers (GPT), to pass assessments in introductory and intermediate Python programming courses at the postsecondary level.
no code implementations • 9 Mar 2023 • Jaromir Savelka, Arav Agarwal, Christopher Bogart, Majd Sakr
While questions requiring to fill-in a blank in the code or completing a natural language statement about the snippet are handled rather successfully, MCQs that require analysis and/or reasoning about the code (e. g., what is true/false about the snippet, or what is its output) appear to be the most challenging.