We are asking you to participate in a research study titled "GUIDE (Guessing IDE)". We will describe the study to you and answer any questions that come up. The Faculty Advisor for this study is Stephen MacNeil, CIS Department at Temple University. We are interested in participants who are at least 18 years old, speak and read fluent English, and have access to a computer with an internet connection.
The purpose of this study is to understand how your trust and coding behavior may change when you see an AI shows its reasoning behind suggestions in real time. Based on the code you write, the system will generate its best guess of the programming problem you are solving and provide next-line code suggestions to assist you in completing the assignment.
You will use a tool similar to GitHub Copilot within a VS Code–style coding editor to complete multiple programming tasks. The tool will log your coding practices and interactions, which will be stored in an anonymized database. Additionally, the system will collect survey data during the study to gain insights into your experience.
If you agree to participate in this study, you will begin by accessing it through a provided link. You will then complete multiple coding assignments using GUIDE (Guessing IDE). The study session will last about 15 minutes at max. You can complete the study remotely at your convenience.
Inappropriate Content: Generative AI models such as GPT-4o have the potential to generate inappropriate suggestions. While we have not observed any inappropriate or harmful content, we will continue to monitor the AI system throughout the study.
Potential for Incorrect Code Suggestions: GPT-4o may generate incorrect suggestions when providing code. It is important that you verify any generated code yourself before accepting suggestions.
Loss of Confidentiality: Data we collect during the coding assignments, such as your interactions with AI-generated suggestions, will only be shared among members of our research team. Data will be deleted at the conclusion of our study.
Research records will be kept confidential to the extent allowed by law. Research records may also be reviewed by the Temple Institutional Review Board (IRB).
You are not under any obligation to participate in the study. If at any time you choose to end participation in the study, you may notify a researcher, and we will delete your data.
You will receive extra credit in the course you were recruited through, at the discretion of your instructor, for your participation in the study. Choosing not to participate will not affect your actual course grade.
Participation is entirely voluntary. You may end participation early at any time without penalty or loss of the benefits/compensation to which you are entitled. If you decide to end participation early, you may notify a researcher, and we will delete your data.
The PI may remove you from the study without your consent if the PI feels it is in your best interest or the best interest of the study. If you encountered unforeseen technical issues, we will remove you from the study.
No, this study will not come at any cost to you.
If you have any additional questions, you may contact the researchers (Rahad Arman Nabid: rahad.arman.nabid@temple.edu, Ian Tyler Applebaum: ian.tyler@temple.edu). You may also contact Stephen MacNeil, the Faculty Advisor, at stephen.macneil@temple.edu
Participants may contact Temple's Human Research Protection Program through the IRB at irb@temple.edu or (215) 707-3390.
Protocol Number: 32615
PI Name: Stephen MacNeil