Tips for Residents and Faculty

In Competence by Design, competency involves more than ‘know how’, it requires that trainees also ‘show how’ and demonstrate ability to ‘do’ independently.

As a teacher, how will CBD affect me?

Going forward your residents will start to think of you as a coach, someone who helps them develop their skills. You will incorporate both direct and indirect observation into your work on a regular basis and in a way that is practical for the reality of you and your work team. You will give short focused feedback to your learners and complete brief records of the observed performance.

The goal of CBD is to enhance patient care by improving your learning and assessment. This will help you demonstrate the skills and behaviours required to meet evolving patient needs. The Competence by Design title is meant to convey the idea that responsible medical education involves systematically thinking about (i.e. designing) a learner’s journey through their entire career in medicine.

The systems, milestones and resources created for CBD will provide learners with

• more frequent assessment and meaningful feedback from faculty,
• well-defined learning paths and clarity around the competencies needed to progress to next stages of training,
• a learning plan that focuses on personal development,

the chance to prepare for independent practice by honing skills and working more independently during the final stage of residency.

RCPSC Competence by Design (CBD) - A Resident's Guide 

Workplace-based assessment (WBA)maintains this formative focus by having frontline clinical teachers observe and document authentic observations in the workplace on a regular basis. The results of individual observations are shared with trainees in a way that guides learning improvement. When these individual workplace-based assessments are aggregated over time, the data from multiple observations and multiple sources gives a clearer picture about a trainee’s performance and progress.

Is it feasible for me to do all of these assessments for CBD in a busy clinical practice?

Yes, CBD assessments are being designed so they are feasible to do within any busy clinical program. In CBD, you will make use of your clinical oversight activity to engage in work-based assessment of resident performance. With the introduction of EPAs and milestones, the assessment criteria will be more explicit and deliberate than it has been before. In a given encounter, you and your residents will focus on a small number of specific tasks or milestones.

EPAs are linked to a specific stage of the Competence Continuum3 and integrate multiple CanMEDS milestones from various CanMEDS Roles. As residents progress through the stages of the continuum, the Royal College EPAs become progressively more complex reflecting the residents’ achievement of more sophisticated milestones.

As a supervisor, you will observe residents as they perform an EPA multiple times and each time you will coach the residents to improve their performance. You may initially refer to the milestones associated with an EPA to inform your feedback on specific elements of the task, but as you become familiar with the EPAs and milestones, you may find that you ‘unpack’ an EPA (i.e. look at all of the milestones) only to plan your teaching or to help a resident who is struggling to progress.

When doing EPA assessments, the main scale that will be used to rate resident performance is an Entrustability Scale.

• Decision reflects a single observation and NOT a generalizable judgment: O-Score Entrustability Scale - As faculty, you must keep in mind that this scale is simply a retrospective opinion of the task you just observed. Your observation does NOT mean the resident has, or has not, achieved the entrustable professional activity (EPA); it only provides your opinion of their performance on the case you observed today. You do not need to speculate about the future.This information, in addition to the other assessments collected, will provide the competence committee with the information to determine when particular EPAs are achieved.
• What does it mean to give a ‘5’ (i.e. ‘I did not need to be there)? No direction was required for safe independence at the generalist level and in hindsight you did not need to be there. However, while the resident is competent, you may have given and documented coaching feedback to facilitate their continued progression towards expertise (i.e. suggestions for future clinical opportunities to seek or alternate approaches to achieve same end point, etc.).
• Assessment at the level of the generalist: The standard for independent practice is at the level of a generalist in your specialty. This is a particularly important consideration for physicians who work in sub-specialized areas. They need to assess trainees based on the level expected of a generalist in their specialty and not someone who has done additional training (i.e. fellowship) in a particular area.

We encourage you to try and complete EPA forms immediately following the observation. Use of your mobile device can facilitate that process. Visit One45's Mobile eDossier - what you need to know for instructions on completing tasks and viewing information on your mobile device.

For Residents

• Send an EPA assessment form: https://one45software.na2.teamsupport.com/knowledgeBase/12813155
• Report on number of assessments on a competency or EPA: https://one45software.na2.teamsupport.com/knowledgeBase/11851019
• See a summary of narrative feedback by competency or EPA: https://one45software.na2.teamsupport.com/knowledgeBase/11851521
• See a trajectory of performance on an EPA or competency: https://one45software.na2.teamsupport.com/knowledgeBase/12537264
• See a summary of EPAs or Competencies: https://one45software.na2.teamsupport.com/knowledgeBase/12537824

For Attendings

• CBME for faculty: send an EPA assessment form: https://one45software.na2.teamsupport.com/knowledgeBase/12812287

Narrative comments on assessments, be it EPAs or ITERs, enhance the learning derived from the assessment and are essential to defending any numerical or scale data that you provide. As such, these comments are essential to both the learner and the Competence Committee. Some key properties of high-quality comments are that they:

• Justify the ratings
• Have specific examples
• Provide recommendations for improving performance
• Are written in a supportive manner
• Are detailed enough for the Competence Committee to understand the strengths and issues

For more information and some examples, see the ICRE Narrative Assessment of Clinical Performance.

The key difference between EPAs and milestones are that EPAs are the tasks or activities that must be accomplished, whereas milestones are the abilities of the individual.

We know what an EPA is. Let’s look at milestones more in depth:

A milestone is the expected ability of a health care professional at a stage of expertise. CanMEDS milestones illustrate the expected progression of competence from novice to mastery associated with each enabling CanMEDS competency. Every discipline will have hundreds of milestones, but for practical reasons you will rely on the educational concept of EPAs as the basis for assessing your residents.

Milestones:

• Illustrate the developmental nature, features, and progression of the competencies
• Assist learners in monitoring their own developmental progress
• Allow individuals to monitor their progress
• Support the identification of learners whose progress is not following the typical development sequence to assist in early intervention
• Guide curriculum development

• Assessment - Comparison between a trainee’s observed performance and a standard (judgment)
• Feedback - Specific information about the comparison between a trainee’s observed performance and a standard, given to the trainee with the intent to improve the trainee’s performance
• Coaching - One-to-one conversation focused on providing practical suggestions for improvement aimed at enhancing learner performance in a specific activity

For more detailed guidance on how to incorporate coaching into your feedback please see Coaching Through Feedback - Review of the R2C2 Approach.

ALL FEEDBACK IS ASSESSMENT AND ALL ASSESSMENT IS FEEDBACK.

ALL FEEDBACK AND ASSESSMENT ARE NOT COACHING. ALL COACHING REQUIRES BOTH ASSESSMENT AND FEEDBACK.

CBD is intended to help learners develop and maintain a growth mindset. Coaching is instrumental in achieving this goal.

• “Coachability”: an individual’s receptiveness and ability to implement feedback leading to growth and performance enhancement.
• Growth mindset = Foundation of Coachability

RX-OCR is part of the CBD Coaching Model and outlines a step-by-step process for clinicians to follow to coach residents:

1. Establish educational Rapport between the residents and the clinician (an educational alliance or partnership).
2. Set eXpectations for an encounter (discuss learning goals).
3. Observe the resident (directly or indirectly).
4. Coach the resident for the purpose of improvement of that work (“coaching feedback”).
5. Record a summary of the encounter

Learn more about on the Rx-OCR Toolkit

Competence Committees make decisions about residents’ competence on clinical tasks (i.e., entrustable professional activities [EPAs]) using qualitative and quantitative data from multiple observations of their performance of EPAs and their achievement of the CanMEDS stage-specific competencies known as milestones. A Competence Committee looks for patterns of progress over time, which is why residents should ask to be observed across contexts, by multiple observers, on many occasions. Remember, though, that EPA decisions are only ONE important part of an overall program of assessment in CBD. Recommendations can be informed by data from many sources, including EPA observations, narrative assessments, summaries of daily clinical performance, in- training examinations, OSCEs and simulation sessions.

Competence Committee Members