Evaluation of a Mobile Phone-Based Intervention to Increase Parents' Knowledge About the Measles-Mumps-Rubella Vaccination and Their Psychological Empowerment: Mixed-Method Approach

This study evaluates 2 mobile-phone-based interventions aimed at increasing parents' knowledge of the measles-mumps-rubella (MMR) vaccination (through elements of gamification) and their psychological empowerment (through the use of narratives), respectively. The 2 interventions were part of a randomised controlled trial (RCT). In particular, the researchers were interested in capturing participants' opinion regarding a number of qualities of the app, such as usability and usefulness, and in acquiring information on their broader experience with the tool. The results of these studies are interpreted in light of the quantitative results of the RCT, and practical implications for the design of future mobile-phone-based immunisation interventions are discussed.

Between December 1 and 10 2016, the research team delivered 2 immunisation interventions through a mobile phone app called MorbiQuiz, which is in Italian language and can be downloaded free of charge in the Italian and Swiss Google Play and App Store. In the first intervention, aimed at increasing participants' knowledge about the MMR vaccine, participants received 35 questions distributed over a period of 10 days (3-4 questions per day). Once answered, each question unblocked an explanation of the answer through textual content. Each correct answer would earn participants a number of points (stars) according to the weight of each question, whereas no points were given for wrong answers or if no answer was given by midnight of the day. To provide a gamified experience, participants could see their score and compare it with that of the other participants through a leaderboard. Furthermore, participants were awarded a shopping voucher, which increased their performance in the quiz.

In the second intervention, aimed at enhancing psychological empowerment (defined as a set of 4 subdimensions: self-determination, self-efficacy, impact, and meaningfulness), users received 2 videos and 8 messages via the app. In the 2 videos, an actress acting as a mother reports that she was able to make an empowered decision about the MMR vaccination by collecting reliable information from multiple sources, and by thinking about the importance and the impact of the decision. In the end, she addresses her audience, encouraging them to make an informed, empowered decision. The messages were designed to reinforce the messages delivered in the video. Participants received either the first, the second, or both interventions. A control group did not receive any intervention.

As part of the RCT, the effect of the 2 interventions (combined and alone) was tested on a number of outcomes, such as vaccination knowledge, psychological empowerment, intention to vaccinate, confidence in the vaccination decision, vaccination opinion, intention to recommend the vaccination, and control preference in the vaccination decision making. All experimental groups reported a significant increase in their vaccination knowledge compared with the control (F3,179=48.58, P<.001), whereas only those participants who received both interventions reported a significant increase in their psychological empowerment (t179=-2.79, P=.006). Only those participants receiving the knowledge intervention had a significantly higher intention to vaccinate (t179=2.111; P=.03) and more confidence in the decision (t179=2.76; P=.006) compared with the control group.

To learn more, right after the RCT data collection concluded, the researchers assessed the perceptions of the participants on a number of characteristics of the app and explored their experience with this tool. The effectiveness of the majority of vaccination interventions using new media, such as immunisation apps, is simply evaluated looking at statistics regarding their download and usage. According to the researchers, these evaluative methods provide no insights into participants' perceptions regarding, for instance, the usability of the tool. Furthermore, qualitative evaluations might be useful not only to collect participants' perceptions but also to assess quantitative findings related to the intervention efficacy or explain why certain features did not have a significant effect on a given outcome.

So, the researchers conducted 2 studies with the RCT participants (who had at least 1 child younger than 15 months, resided in the Lombardy region of Italy, and owned a mobile phone with an internet connection): (i) a Web-based survey aimed at assessing their rating of the tool regarding a number of qualities such as usability and usefulness (N=140); and (ii) qualitative telephonic interviews to explore participants' experiences with the app (N=60).

A first main finding from both studies is that overall, participants perceived the app as highly usable and useful to make a vaccination decision. However, the 2 groups receiving the quiz (alone or together with the videos/messages) liked the app significantly better than the group that only received the empowerment intervention through videos/messages (F2,137=15.335; P<.001). Furthermore, participants receiving only the quiz reported higher scores for most app's qualities compared with those receiving the videos/messages in addition to the quiz. The researchers suggest that one reason why the educational version of the app received higher ratings is that participants might not be familiar with empowering interventions delivered through a video format and administered through a mobile phone app.

The results of the interviews also shed more light between-group differences. Parents' qualitative reports indicate that the knowledge intervention (employing the quiz and using elements of gamification) was perceived as an active learning experience, compared with the videos, which were perceived as involving passive exposure to a story. Furthermore, those in the knowledge group highlighted a number of positive aspects relative to learning, praising the gamified way by which they could not only acquire new information and question their previous knowledge but also improve their information seeking skills.

Parents receiving the empowerment intervention, on the other hand, lamented the lack of factual information that they received in the video, highlighting the emotional burden such a call for a self-determined decision might entail. The interview results also showed that mothers liked to identify themselves with the main character of the videos, as they share similar experiences and difficulties. However, beyond recognising similarities with the protagonist, identification did not seem to be associated by parents with important aspects related to their decision making regarding their child's MMR vaccination.

The study confirms previous findings that participation in gamified interventions was associated with users' engagement, enjoyment of activities, increased task performance, higher empowerment, learning, and more positive attitude. Participants in the present study reported feeling more convinced of their vaccination decision after participating in the quiz, which echoes a previous study that found gamification to be effective in reinforcing a behaviour.

The researchers explain that the findings also provide insight into the results of the previous RCT, which found that only the group receiving the knowledge intervention significantly increased their intention to vaccinate against MMR and their confidence in making a vaccination decision. The results of the qualitative study show that "[p]arents need a clear direction or, at least, a comparison between different points of views on vaccinations. Excessively pressuring them to find vaccination-related information and to talk to different people - without providing factual information at the same time - might generate frustration and emotional distress." Finally, they say, parents indicated that they were aware of the impact the app can have on their decision making, with the large majority reporting it could potentially lead parents to opt for the vaccination. "Users' awareness of the goal and the high potential of an app are crucial for making an app trustworthy and worth downloading or being recommended..."

In conclusion: "Using a narrative format that allows identification can be appropriate, as it was reported to be associated with a feeling of social support....This, however, should not be employed alone but rather together with the presentation of more points of views and notions regarding, for instance, the risks and benefits of the vaccination."