Q & A: Would you like hints or straight answers?

Thank you very much for everyone who commented on my video! Here is the presentation video for your reference. In this post, I will answer some questions you guys brought up, and hopefully it will clear things up for you! The link for the actual paper is here.

Question #1: Application in online community

I think there is a related class this semester that focused on this topic as well. But as an outsider, what is formative assessment-based mobile learning, and whether a personalized assessment is applicable to the online community where a lot of users may hard to know.

Formative assessment is a feedback system in which the users are provided with hints or supplementary materials rather than explicit right 🙆 or wrong 🙅 validity checks for their answers in an assessment. Formative assessment-based mobile learning is a mobile learning system in which we apply such feedback system in a mobile learning environment. In an online community consists of many users, everybody will make different mistakes, but if they are wrong on the same question, they would get the same hints, and they would have to review the same learning materials again to find out the correct answers, which would be the exact same process as what was carried out in this study. Therefore, I think it would work for online community too. I think the core feedback mechanism that the research focused on doesn't change even if we shift the entire learning environment online.

Image Source

Question #2: What emotional response measure?

You mentioned "adding an emotional response measure to see if the students experience any negative emotion with conventional right or wrong feedback". What would these emotional response measures be and how would they impact the results of the study?

In a self-reporting questionnaire, we can break down the students emotional responses into 4 steps: seeing a question, answering a question, receiving a positive feedback and receiving a negative feedback. The questionnaire can be check-box based, and the students will have the ability to add more emotions. The emotions can include excitement, annoyance, fear, satisfaction, aversiveness, surprise, upset, etc. In addition, the students will also need to rate the intensity of each emotion on a scale of 1-10. In my opinion, such measure reflecting emotional response help us evaluate whether the FAML system also out-perform the conventional feedback method in students' emotional regulation. While measuring learning interests and attitudes are important, I believe analyzing the emotional response associated with the action of performing a test or assessment is as important, since testing is also a part of learning. 

Image Source

 Question #3: Would the founding be universally valid?

Do you think the research findings have broader implications for a wider audience than 5th graders? Similarly, was the study domain-specific or universal? I think you raised some good points in your summary, and I feel that so often these results are based on immediate performance, not longer term transfer.

I believe so! I think all learners will be impacted by this significant change in assessment method, maybe at a different level. I assume that learning motivation is more complicated in adults than children, so maybe the learning performance, interest, and attitude of adults won't differ as much. Nevertheless, I would rather have hints than simple right or wrong feedbacks. When I'm explicitly told I'm wrong, I usually just feel really dumb, but when I get some hints to work on, my attention shifts to what the answer could be rather than judging myself, and all failures become successes (because I eventually work them out myself! Well, kind of 😅).

Long term transfer is a great topic to look into! Because the FAML group students dedicate more cognitive efforts investigating the answers, the might have a better consolidation of the knowledge they learned in a long term. A longitudinal study would be fascinating!

Question #4: Gender Difference

Based on an article that I read recently, there are behavioral and psychological differences between men and women, even when it comes to adaption and use of technology. So I'm wondering whether this will affect the effectiveness of mobile learning outcomes?

I think it depends on the scope we are talking about. If we are talking about just the context of this study, I would think NO, since the study is focusing on evaluating the learning outcomes of different feedback mechanisms in a short mobile learning session. Not too much adaptation is required for just learning to use the assessment app, and the students didn't use the device for too long for learning either. However, in a greater scope, if we want to talk about long term effect, then I would think yes. I don't know about the article, but I would love to learn more! If you can provide the link to the article, that would be great!