How Attendee Confidence Reinforces Knowledge and Identifies Gaps in Continuing Medical Education

Polling throughout a CME symposia is a simple and effective way to assess the initial understanding healthcare providers have on a topic and test their knowledge after the material has been presented. However, adding confidence-based assessment (CBA) to polling can more effectively indicate who has mastered a subject and is ready to implement it in practice, and who may first need additional instruction. Studies have shown that asking HCPs to momentarily reflect on their answers and rate how confident they are in them improves knowledge and retention while identifying topics in which attendees are misinformed.¹

How it works

Confidence-based polling pairs multiple-choice knowledge testing with self-assessment. After submitting an answer, the respondent is asked to rate their level of confidence in their selection. One way this is done is by asking a follow-up polling question which asks them to rank their confidence in the previous answer. Another approach is to use a sub-question format in which the confidence question is a pop-up on the same screen as the parent question. The latter keeps polling moving along without a long pause between the initial question and its confidence counterpart. Regardless of how attendees are asked to rate confidence, the results of the parent questions are shown on screen to be used as part of the presentation, but the confidence scores are only included in the post-meeting report.

Ability to improve knowledge and ID gaps

Ideally, at the end of the meeting, supporters want to see that everyone is highly confident and answered all questions correctly. This would mean they’re all ready to take what they’ve learned back to practice to benefit their patients. However, not everyone learns in the same way or at the same pace, so they may not be as knowledgeable by the end of the symposia as needed or desired. Given this reality, deploying CBA can bring about two significant benefits: improving knowledge acquisition and retention, as well as identifying knowledge gaps.

Simply asking, “How confident are you in your answer?” requires respondents to pause and reflect on the initial question, and the content on which it focuses. Choosing a low level of confidence is a cue that they need to do more to learn or retain. At that point, they can take the necessary steps to improve their understanding and confidence, such as by annotating or saving relevant slides or asking questions.

Following the meeting, the data gathered through CBA can be plotted on a graph to provide a visual representation of where there are knowledge gaps (and where there may have been lucky guesses). Among the benefits of this analysis is that it helps to spotlight people who were highly confident in their wrong answers and therefore are likely to share or act on their misinformation following the meeting. Sponsors and educators need to be made aware of this group and prioritize providing them with additional training.

Setting up confidence assessments

To use confidence-based assessment to its greatest advantage, it’s important to address it as part of the overall CME content, engagement, and insights planning. CME providers should work with their technology partners to identify engagement features to deploy to reinforce the content, what data and insights would be most meaningful to supporters, and how to derive those insights via engagement. As part of this planning, they would also determine when to use confidence questions or sub-questions to add value to the overall meeting data set.

“In CME, attendees may be learning how to use a new drug, class of drug or diagnostic test, so the training is really about what’s applicable to their practice,” says Amanda Glazar, insights director for Array. “The confidence assessment, then would be focused largely around the learning objectives and what the sponsor hopes to achieve from the meeting.”

For example, targeted confidence questions could be, “How confident are you in referring your patients?” or “How confident are you about using updated guidelines around asthma treatment?” Glazar notes this type of question is often useful if the CME is for a primary care audience and focused on a specialty area or treatment for a specific illness.

“Confidence questions such as these that are really practice-focused supplement those behavior changes that we're looking for in CME in terms of what attendees are going to do when they return to their practice. Are they prepared to change patient care based on what they’ve learned?” she says, adding that often there will be both questions like these and confidence scoring sub-questions used in the same meeting.

In many cases, CME providers may ask how confident HCPs are in a particular knowledge set or action prior to an activity and after, to see how much confidence has improved as a result.

Confidence scoring is often used with questions around patient scenarios. Asking attendees to rank their confidence in an answer based on a patient case provides real-world insights into how they are likely to practice.

Richer data set for more insights

Each question attendees answer becomes part of the overall data set for the meeting. Confidence questions are also part of that set and help paint a clearer picture of the meeting as a whole – what lessons HCPs mastered, where there was some confusion, and where there is misinformation—as well as allow for deeper analysis.

“While we often see there has been an increase in confidence, what’s important is that the data set collected through the Array platform includes enough context to dig deeper when there wasn’t,” says Glazar, adding among the additional insights she’d look at are the evaluation responses, to find issues with specific sessions or a topic they struggled with. “We can also look at the individuals to see if they didn’t do scoring on certain questions, at individual polling questions to see where knowledge gaps were and which remain for these attendees, and find out if that correlates with why there wasn’t an increase in confidence.”

Short and Long-Term Benefits

Confidence-based assessment as part of a CME symposia’s audience engagement efforts is a predictor of knowledge gained and inclination to act on those learnings back in practice. The act of simply asking an HCP to consider if they are confident or to rate confidence provides an important moment of reflection that reinforces topics and sheds light on the need to do more to acquire and retain information.

Research has found that individuals who stated they were highly confident in their knowledge retained 91% of their learned knowledge; those who were less confident retained only 25% after one week.

Additionally, for CME providers, deploying CBA through seamless polling questions or sub-questions has significant benefits. It not only provides more data to contribute to the meeting’s outcomes story, but also helps guide future needs assessments by identifying the gaps in knowledge.

Array can help you plan and deploy confidence-based assessments to improve knowledge gain and get a richer data set from your CME meetings. Find out how our technology, services and insights can help you achieve your Symposia goals and demonstrate results.

1. Novacek P. (2013). Confidence-based assessments within an adult learning environment. IADIS International Conference on Cognition and Exploratory Learning in Digital Age, CELDA 2013. 403-406.
2. Hunt D., 2003. The concept of knowledge and how to measure it. Journal of Intellectual Capital, 4 (1), pp. 100-113