Ethical Study Design

The following sub-modules help answer questions like:

• How can a researcher determine whether a study is ethical?
• What factors should be a part of that determination?

The following key terms are important for understanding ethics:

• Beneficence - the idea here is to maximize the benefits to the participants while at the same time reducing possible harmful effects.
• Justice - addresses issues of fairness in receiving the benefits of the research along with bearing the burdens of the research.

Risks

To ethically design a study, the researcher needs to consider the various types of risks that participants may face and seek to reduce these risks as much as possible. Types of risk to consider include:

• Physical Harm - less likely in CS Education studies unless the research involves some type of device like an eye tracker or other data collection device.
• Stress - does the study design and/or data collection cause unnecessary stress to the study participants?
• Loss of Privacy or Confidentiality - does the way a researcher collects or reports data cause the participants to reveal unnecessary information?

In many cases the risks above cannot be completely eliminated. However, a researcher should strive to reduce them as much as possible. In designing a study, the researcher should first seek to identify all potential risks and then consider various design alternatives that can reduce those risks.

Experimental Treatments

When designing a study, the researcher must consider whether the choice of treatments (or application of the independent variable) and the application of those treatments is ethical. Some considerations include:

• Are there services, knowledge, or benefits the treatment group receives that the control group does not? If so, how can the study design make things more equitable?
• How are participants assigned to groups?
• What are the alternatives to the treatments?
• Do the participants receive any sort of compensation?

Compensation

In some cases, it may make sense to compensate participants for the time the spend in a study. Compensation can take different forms, including financial payments and/or course credit. A researcher must take care that any sort of compensation does not unduly influence someone to participate in a study when they otherwise would have declined. Also, a researcher has to take care that the compensation does not encourage participants to alter their behavior to receive the compensation.

For example, if students are compensated by how may bugs they report during code review, they will be incentivized to report more information than normal. This over-reporting could potentially bias the research results. An alternative would be to compensate participants for the time in the study rather than the number of bugs reported.

Deception

Deception is when a researcher does not provide participants with the complete truth about the goals of a study or the data being collected. There are rare cases where this approach may make sense. However, a researcher should not consider using deception unless it is justified by the prospect of significant results and there is no other method for obtaining similar, valid data. In addition, deception is not appropriate if it has the potential to cause emotional distress. If deception is required, then the researcher should explain it to the participants as early as possible during the study.

For example, consider a study where a researcher is trying to study the frequency and type of communication among team members in a group project. If the researcher informs the participants of this specific goal at the beginning of the study, it may affect how the students communicate during their project, thereby nullifying the results. In this case, the researcher may consider deception as a study design option if no other alternative design makes sense.

Debriefing

At the end of a study, a researcher should provide some type of debrief to the participants. This debriefing, session can serve a number of purposes, including:

• Clarifying any deception the researcher used
• Allowing participants to provide feedback on the study which might help the researcher understand or interpret the results
• Remedy any harms that may have occurred during the study

Reporting of Results

Researchers should take care when reporting results to be accurate based on the data and analysis present. Researchers should not either overstate results (that is draw conclusions stronger than the data allow) or omit results that do not match the researchers’ expectations. Where possible, providing links to the raw data and/or analysis scripts can help other researchers have more confidence in the results drawn from the study.

Plagiarism

Researchers should never plagiarize from other researchers or from their own work. If a researcher wants to refer to prior work, they should always properly reference it so readers can clearly identify what information is new.

---