Categories
Game Research

The Potential and Challenges of Internet-Based Game Research

The Internet is becoming an important medium for conducting research. The Internet has broken down the spatial and temporal boundaries between researchers and potential participants. Conducting research with traditionally rare, under-researched, hard to access and culturally diverse populations becomes possible (Hewson & Stewart, 2014). Internet-Based Research, Internet-Mediated Research, Web-Based Research and Online Research are often used interchangeably for describing research conduct via the Internet. Internet-based research has its strengths and weaknesses. This article examines the characteristics and ethical challenges of Internet-based research, and the potential and issues of conducting game research on the Internet .

Internet-Based Research

The Internet presents unprecedented opportunities for empirical researchers. The Internet affords both opportunities and challenges that impact various aspects of conducting research online, from recruiting participants, data collection to dissemination of results. Below is a summary of advantages and challenges when conducting research online.

The advantages of Internet-based research include:

  • low cost and time investment
  • high generalizability of findings
  • access to a large number of participants across spatial and temporal boundaries
  • high statistical power
  • ease of access for participants
  • high ecological validity
  • ability to detect motivational confounding
  • high degree of volition (Benfield & Szlemko, 2006; Reips, 2000)

On the other hand, the challenges of conducting research online include:

  • multiple submissions
  • self-selection bias
  • high dropout rates
  • dependence on technology
  • misunderstandings of procedures and materials (Birnbaum, 2004; Kraut et al., 2004)

Reips (2002b) proposed a 16-recommendation checklist for enhancing the validity and reliability when conducting Internet-based research. Below is a list of selected recommendations.

  • Use Internet-based tools and platforms (e.g., Qualtrics  , WEXTOR, DREAM) to create and administer study materials.
  • Pretest study for clarity (instructions and materials) and compatibility across browsers, operating systems and network settings.
  • Check for configuration errors (e.g., unprotected directories, access to confidential data, improper use of form elements) (Reips, 2002a).
  • Adopt multiple site entry technique to determine effects of self-selection and estimate generalizability.
  • Apply warm-up, high-hurdle, and seriousness check techniques to increase engagement and reduce dropout.
  • Use dropout to determine motivational confounding.
  • Implement access control to avoid multiple submissions.
Ethical Challenges of Internet-Based Research

Research ethics is an important issue to consider when conducting research online. The three key ethical principles underlying research involving human subjects specified in the Belmont Report (National Commission for the Protection of Human Subjects of Biomedical, & Behavioral Research, 1978) include, respect for persons (i.e., rights, privacy and confidentiality), beneficence (i.e., maximize benefits and minimize harm), and justice (i.e., equality in participation and benefit from outcomes). These principles require special considerations for Internet-based research.

One of the greatest risks associated with online research is the breach of confidentiality. Issues such as data security during transmission, and physical and digital security of data storage need to be considered carefully. In addition, extra efforts are required from researchers to protect the anonymity of participants, especially when it involves observation and record of online activity and communication.

Informed consent allows participants to agree, understand and acknowledge their rights, risks and benefits. However, researchers can not easily determine if respondents understood the informed consent statements or if they are indeed who they claim they are (e.g., children pretend to be parents) when respondents consenting online. More pretesting and additional procedures (e.g., questions for confirming understanding, verifiable digital signatures) may need to institute to ensure the effectiveness of informed consent.

Online research involving surveys, experiments and observations often involves minimal risks. However, research may lead participants to relive unpleasant experiences, reveal incompetency, or challenges personal beliefs, values, or attitudes, which results in mental or emotional harm (Kraut et al., 2004). Online researchers have very limited ability to observe and remedy the harm caused. Additional measure (e.g., social support groups) to provide support to participants should be considered.

Finally, internet-based research requires access and proficiency in computer technology. Researchers should take these considerations into account to facilitate equal opportunities for participating and receiving benefits from the research.

Potential and Challenges of Internet-Based Game Research

The Internet provides a unique medium for conducting game research. The Internet allows game researchers to access a large and diverse pool of game players. Specific target populations (e.g., MMO players) with different social skills from different geographical locations can be reached via specific game forums and websites. Various online data collection methods facilitate examination of different psychological aspects of game playing effectively. Online data collection is particularly useful when exploring sensitive issues (e.g., problem video game play, aggression) because of high degree of anonymity (Elliott, Golub, Ream, & Dunlap, 2012).

Online questionnaires such as psychological and cognitive measures can provide standardized and consistent implementations with high validity for investigating various aspects of game playing experience (e.g., motivation, performance) (Yee, 2006). Participant observations allow researchers to become part of the phenomenon for examining game playing culture or specific personal experience (Steinkuehler, 2006). Finally, online interviews can be conducted synchronously or asynchronously for investigating players’ perceptions, values, attitudes, and behaviors with a substantial saving in time (Hussain & Griffiths, 2009).

The usual ethical considerations and issues pertaining to Internet-based research discussed above need to be addressed when conducting game research online. Moreover, game playing is a popular social and cultural phenomenon, extra efforts should be made to anticipate media interests, implications and consequences of the research, and how that would affect the participants (e.g., embarrassed, betrayed) and the society (e.g., legal regulation). In addition, special considerations are required when observing players online. The Internet blurs the boundaries between public and private behavior. These boundaries often depend on legislation, social norms, and perceptions and interpretations of the participants. Researchers should make themselves and their motives known to the participants when participants’ responses will not be adversely affected and anonymity may not be protected (Wood, Griffiths, & Eatough, 2004).

The Internet provides a medium for conducting game research across temporal and geographical boundaries with reduced cost and time. The Internet affords game research to employ a variety of online data collection methods for investigating different aspects of game playing phenomenon among diverse populations effectively. However, special considerations should be taken into account for protecting the rights, privacy, anonymity, and minimizing the risks of the participants.

References

Benfield, J. A., & Szlemko, W. J. (2006). Internet-based data collection: Promises and realities. Journal of Research Practice, 2(2), D1-D1.

Birnbaum, M. H. (2004). Human research and data collection via the Internet. Annu. Rev. Psychol., 55, 803-832.

Elliott, L., Golub, A., Ream, G., & Dunlap, E. (2012). Video game genre as a predictor of problem use. Cyberpsychology, Behavior, and Social Networking, 15(3), 155-161.

Hewson, C., & Stewart, D. W. (2014). Internet research methods. Wiley StatsRef: Statistics reference online, 1-6.

Hussain, Z., & Griffiths, M. D. (2009). The attitudes, feelings, and experiences of online gamers: A qualitative analysis. CyberPsychology & Behavior, 12(6), 747-753.

Kraut, R., Olson, J., Banaji, M., Bruckman, A., Cohen, J., & Couper, M. (2004). Psychological research online: report of Board of Scientific Affairs’ Advisory Group on the Conduct of Research on the Internet. American psychologist, 59(2), 105.

National Commission for the Protection of Human Subjects of Biomedical, & Behavioral Research. (1978). The Belmont report: ethical principles and guidelines for the protection of human subjects of research.

Reips, U. D. (2000). The Web experiment method: Advantages, disadvantages, and solutions. In Psychological experiments on the Internet (pp. 89-117). Academic Press.

Reips, U. D. (2002a). Internet-based psychological experimenting: Five dos and five don’ts. Social science computer review, 20(3), 241-249.

Reips, U. D. (2002b). Standards for Internet-based experimenting. Experimental psychology, 49(4), 243.

Steinkuehler, C. A. (2006). Massively multiplayer online video gaming as participation in a discourse. Mind, culture, and activity, 13(1), 38-52.

Yee, N. (2006). Motivations for play in online games. CyberPsychology & behavior, 9(6), 772-775.

Wood, R. T., Griffiths, M. D., & Eatough, V. (2004). Online data collection from video game players: Methodological issues. CyberPsychology & Behavior, 7(5), 511-518.

By Frankie Tam

Frankie is the founder of Red White Console. He is passionate about advancing the knowledge of game design through game research.

Leave a Reply

Your email address will not be published. Required fields are marked *