Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions

no code implementations • 4 Aug 2023 • Samia Kabir, David N. Udo-Imeh, Bonan Kou, Tianyi Zhang

Despite this popularity, no comprehensive study has been conducted to evaluate the characteristics of ChatGPT's answers to programming questions.

Misinformation

Is Model Attention Aligned with Human Attention? An Empirical Study on Large Language Models for Code Generation

no code implementations • 2 Jun 2023 • Bonan Kou, Shengmai Chen, Zhijie Wang, Lei Ma, Tianyi Zhang

Through a quantitative experiment and a user study, we confirmed that, among twelve different attention computation methods, attention computed by the perturbation-based method is most aligned with human attention and is constantly favored by human programmers.

Code Generation