Creating Meaningful Learning

ADTED 531 Unit 11 - Reflections

For my final blog entry, I'd like to discuss an element from our reading for this Unit -- Chapter 9 in Delivering Digitally: "Redesigning Support Services." The reading deals with three important aspects of learner support, the first of which is setting up help desk services. Beyond the simple necessity for helpdesk services, which I think can't be emphasized enough and is all too obvious in our "customer-centric" society, pages 160-163 starts to discuss other strategies for support, such as setting up peer support social networks.

I'm not entirely sold on this concept, because I liken it to the self-help "forums" that seem to be growing on customer support websites for products of all kinds. Basically, the idea is that the company is trying to save money by not having you call their customer support hotline to ask your question. Instead, they'd prefer you to utilize their online self-help forums, where you can look up via a site search query, questions that others have asked about your particular product. The question asked and the answer provided is shown, in what the company hopes will help to solve your question before you need to call them and ask it. Most companies today tout this as "24/7 online support" and say that it's more customer-friendly to have all of these answers readily available, but in truth I have found that most self-help forums are wastes of time, and I end up having to call customer support anyway. And, not only that, but I end up having to search harder for a customer support phone number to call, because the companies seem to do a pretty good job of hiding it in the site because they REALLY want you to use their self-help forums instead.

So, the question is, if we were to utilize the peer network model of self-help support for DE, is this what it would look like? I think it probably would, because I can't think of any other way to do it. And, if this is what it would look like, is this an adequate substitution for true helpdesk support?

Beyond issues of quality, which at this point are probably going to be subjective opinion (but you should definitely comment below and tell me your opinion!), I tried to find some hard data to show me whether peer network as a support mechanism was actually more cost-effective than traditional helpdesk support (as in, you call a phone number and someone helps you immediately). To my surprise, I actually couldn't find anything on this subject! I looked on both Google Scholar and regular Google. Maybe I just don't have the search terms correct -- has anyone else ever tried to look for information on the cost effectiveness of online self-help forums? I'd be interested to hear what you find!

ADTED 531 Unit 10 - Reflections

Continuing on this week's discussion of Evaluation Objectives, I'd like to provide a review of the following article, which deals with trying to develop and measure the effectiveness of an instructional design model or process.

Richey, R., and Klein, J. (2005). Developmental research methods: creating knowledge from instructional design and development practice. Journal of Computing in Higher Education. 16 (2), 23-38. Retrieved from: http://ezaccess.libraries.psu.edu/login?url=http://proquest.umi.com.ezaccess.libraries.psu.edu/pqdweb?did=2164864051&sid=9&Fmt=6&clientId=9874&RQT=309&VName=PQD

This article is “a discussion of establishing the credibility of a given developmental study by assuring authenticity and methodological precision.” In short, it’s a study on how to study the development of an instructional design model or process. There are often many ways in which one could study or quantify the results of a particular educational design or intervention, so this article helps to examine and identify how to choose amongst them.

The first step in developmental research methods discussed in the article is to focus the research problem on only ONE particular aspect of the design, development, or evaluation process, rather than focusing on a variable that might impact learning. Also, one must determine whether the research will be collected as the development occurs, or after the fact. Then, we must frame our specific research questions, and identify what limitations we will have in our data. Then, we proceed with normal research methods – review the literature, identify the target population, and decide on a research methodology. There is a great discussion of the types of research methodology used in developmental studies, and an equally detailed description of how to collect and analyze the data. Both review specific case examples, which might help a student to design their own study.

I would recommend this article as an introductory reading assignment to anyone new to designing and developing research studies to measure any aspect of instructional design. There is in particular a very useful table on page 31 (page 9 in the PDF document) that details Common Research Methods Employed in Developmental Research Studies, which highlights typical methodology patterns used in developmental studies and can provide great direction for anyone needing some ideas on how best to design a research study.

ADTED 531 Unit 9 - Reflections

In making the key decisions about how to design and present online course material, whether to use synchronous or asynchronous learning tools, and how to organize the material, we often run into the question of whether to use discussion forums. Most online courses nowadays seem to utilize them in some way, and most learning institutions would defend that they are effective means of communicating the course material. But, have they ever been evaluated for effectiveness? What kinds of course material might they be most effective for?

I went online this week in a search for some research to answer this question. Rather than just searching the internet for magazine articles or editorials, I actually went into the PSU library system looking for a scholarly journal article. I found the following article to be most relevant to my search, and I reviewed it as a means of trying to answer my questions about whether or not to use online discussion forums in an online course structure.

Curran, J.A., & Abidi, S.S.R. (2007). Evaluation of an online discussion forum for emergency practitioners Health Informatics Journal, 13(4), 255-266. Retrieved from: http://jhi.sagepub.com/content/13/4/255

This article evaluated the outcomes of using an asynchronous online discussion forum to share knowledge between emergency healthcare practitioners. The idea is that emergency healthcare clinicians don’t really get the chance to engage in relevant and meaningful interactions about practice knowledge, because the nature of their job is such that they are constantly interrupted and splitting their attention while on the job responding to emergencies. So, the authors implemented an asynchronous online discussion forum of the type found on Communities of Practice, to help the practitioners engage in dialogue around topics relevant to their practice learning needs. Then, the authors evaluated whether this online discussion forum served as an effective means of knowledge seeking, knowledge sharing, and knowledge organization between emergency healthcare practitioners.

The measurement of effectiveness of the communications on the discussion forum was done by using both quantitative statistics about the network posts (demographic analysis), and then a qualitative topic analysis (content analysis) of the content of those posts, and a social network analysis to show the knowledge-seeking relationships amongst those in the medical community. The 3 analyses revolved around measuring episodes of information seeking or sharing. Descriptive statistics showed that across 250 total postings, there were 52 information seeking episodes, and 63 information sharing episodes. Content analysis results showed that the pediatric trauma, conscious sedation, poison management and asthma and airway topics were of greatest interest to participants. Social network analysis showed that the rural health center practitioners most often sought knowledge from the urban health center practitioners, but within a health center there was a unilateral sharing of information. Conclusions were that, given the opportunity, emergency practitioners will take advantage of an online discussion forum to seek and share knowledge and information that is clinically relevant to their practice.

This article was interesting in that it did the same thing as I’m attempting to do in my research, which was to assess the interactions that take place in an online Community of Practice. The research was limited only to the discussion forum of their online Community, but their method of measuring the interactions both quantitatively and qualitatively was similar. I was disappointed that the article purported to measure the effectiveness of the interactions, but really I didn’t see a rubric created or any way to truly measure whether the interaction was effective. It was simply categorized as either information seeking or sharing, and then the social network analysis was done to see whom the interactions were between. So, while overall this article showed initial similarities with my research, it ultimately ended up being a lot more about social network analysis than it originally purported to be. That was how the authors measured “effectiveness” of the interactions, which to my mind is not really the way it should be measured. However, I’ve done quite a bit of social network analysis work in my professional life, and the methods used here are sound. The findings were not revolutionary, but the study is a good foray into measuring social networks. But, not so much in measuring effectiveness of discussion forum posts.