Summary

Inpractice provides doctors registered in a general scope of practice with a robust framework to guide continuing professional development (CPD) and a convenient means of meeting the requirements for recertification.

The aims of Inpractice are to:

• Establish recertification requirements that are achievable by all competent doctors who are in active practise and who take reasonable steps to complete their CPD requirements
• Encourage participation across a range of activities contributing to CPD
• Encourage activities that contribute to the training, development and recognition of the health workforce
• Provide doctors with a convenient means for demonstrating active participation in CPD
• Provide documented evidence to the Medical Council of New Zealand that the doctor has met the required standards for recertification
• Ensure that all doctors who have difficulty in meeting the required standards are provided with encouragement, assistance and clear pathway to remediation
• Develop and operate Inpractice in a way that is fair, transparent and affordable for doctors

Inpractice is based on a 12 month CPD cycle aligned to each doctor's annual practising certificate (APC) cycle.

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Jan

Feb

Mar

Apr

During each 12 month CPD cycle each doctor must complete a minimum of:

• A professional development plan
• 20 hours of continuing medical education
• 10 hours of peer review
• One audit of medical practice
• Four meetings with the nominated collegial relationship provider

In addition to the annual requirements, each doctor must complete:

• The Essentials quiz before the first renewal date and then once every three years
• Feedback on practise, through colleague feedback and patient feedback (where practicable) once every three cycles
• A regular practice review visit when scheduled (See appendix three)

To meet the requirements for recertification each doctor must meet the activity targets displayed on the home page of their Inpractice ePortfolio. For non-annual enrolments, doctors may have their requirements adjusted on a pro rata basis, at the discretion of the Inpractice medical advisors.

1 Inpractice terms and conditions

1.1 Terms and conditions

The Inpractice terms and conditions are outlined below:

• The doctor agrees to abide by the requirements of Inpractice, as set out in this Inpractice programme guide (the most current version of which will be that available on the Inpractice website).
• The doctor agrees to the information recording, exchange and confidentiality provisions of Inpractice as detailed in this Inpractice programme guide.
• The doctor agrees to keep Inpractice informed of their current postal and email address for correspondence and reporting purposes.
• The doctor agrees to ensure that all information recorded in their ePortfolio is an accurate and fair reflection of the CPD activities undertaken.
• The doctor agrees that Inpractice has the right to change any aspect of the Inpractice recertification programme at its discretion. These changes may be in response to changing Medical Council requirements or other considerations that Inpractice considers justify such changes.

1.2 Who is required to participate

Doctors who are registered in only a general scope of practice and have a current annual practising certifcate are required to participate in Inpractice.

The exceptions to this are:

• Doctors who are participating in a vocational training programme
• Doctors who have had their scope limited to non-clinical practice

Short term locums registered within a general scope must sign up to the Inpractice programme. The dates specified in the Annual Practicing Certificate will be used as the dates of enrolment in the Inpractice programme.

1.3 Enrolling with Inpractice

Doctors enrolling for the first time with Inpractice will require a login and password to complete the enrolment requirements and activate their ePortfolios.

Login and password details can be obtained by contacting Inpractice directly, and providing the required details (Name, Medical Council number, date of birth, date of obtaining general scope, and email address).

The enrolment requirements include:

• Completing the online enrolment form
• Entering details of the collegial relationship provider
• Accepting the Inpractice terms and conditions
• Paying the annual fee

Failure to complete all of these requirements may result in the doctor's membership of the programme being suspended. Membership will be reinstated when all enrolment requirements are met.

1.4 Annual fee

All doctors enrolled in Inpractice must pay an annual fee. Inpractice reserves the right to review the fee annually. The current fee is $1,200 GST exclusive; $1,380 GST inclusive.

Doctors who join Inpractice part way through their APC cycle will have their fee calculated on a pro-rata basis for the amount of time remaining until the end of their APC cycle.

There is a minimum enrolment period of 90 days in the Inpractice programme.

Low income policy

Doctors with a medical income in New Zealand or overseas of less than NZ$20,000 per annum may be eligible for a 50% reduction in the Inpractice fee.

To apply for this reduction, doctors will be required to complete a Low Income Declaration form (also available by email from support@inpractice.org.nz).

1.5 Withdrawing from Inpractice

Temporarily parking an account with Inpractice

Any doctor not practising medicine in New Zealand for a period of three consecutive months or more, may apply to temporarily park their account with Inpractice for the period of their absence from medical practice.

Requests to park accounts must be made to Inpractice in writing (email) before the period of absence begins. Retrospective applications will only be considered if there has been extenuating circumstances.

Once parked, the doctor will not be required to record recertification activities for the period of their absence. The extent and nature of the activities required to complete the doctor’s recertification requirements for the current CPD cycle will be determined on a pro-rata basis at the discretion of the Inpractice medical advisor. Any doctor who parks their account will continue to have access to the information stored in their Inpractice ePortfolio, but not the ability to record new information.

The Medical Council will be notified when a doctor temporarily parks their account, and the doctor must notify Inpractice when they return to medical practice in New Zealand. Participation in a recognised recertification programme is a requirement of holding an Annual Practising Certificate.

Permanently withdrawing from Inpractice

Doctors wishing to permanently withdraw from Inpractice should notify the programme in writing (email) stating the date they wish to withdraw.

Any doctor who has withdrawn from Inpractice will continue to have access to the information stored in their ePortfolio, but not the ability to record new information.

The Medical Council will be notified of any doctor who withdraws from Inpractice.

Refund policy

Inpractice will partially refund the annual fee of any doctor who permanently withdraws from the programme, or parks their account for a period greater than three months.

The refund will be calculated from the date Inpractice is advised, on a pro-rata basis with the amount proportional to the number of days the doctor has spent in the programme, subject to a minimum enrolment fee equal to 90 days enrolment.

1.6 Disputes and appeals

Any doctor, who disagrees with the way in which Inpractice recertification requirements have been applied, should, in the first instance, raise their concerns with the management of Inpractice.

If in the doctor’s opinion, the concern cannot or has not been appropriately dealt with by the programme manager, the doctor may refer their concerns to the Inpractice independent disputes commissioner.

The final avenue for appeal in all matters related to recertification will be the MCNZ.

1.7 Data and information

Information held about doctors

Inpractice will maintain a secure database containing the following information:

• MedReg data updated monthly from the MCNZ MedReg database.
• Enrolment data information provided by each doctor when they enrol in the programme.
• Record of CPD activities entered by each doctor into their ePortfolio.
• A register of contact recording all contact (email, phone etc) between each doctor and Inpractice.
• A register of concerns containing a record of all concerns raised about a doctor through Inpractice, including the doctor’s responses. The purpose of maintaining the register is to ensure any pattern of concerns is easily recognisable and to inform any future investigations into the doctor’s practice or behaviour.
• A record of ePortfolio audits including information on any issues that had been identified during the audit of the ePortfolio.
• A record of any structured remediation plans developed in conjunction with the doctor and the Inpractice medial advisor.

Some of the information outlined above is held separately from the doctor’s ePortfolio, but all information held is available on request.

Inpractice recognises the sensitivity of this information and therefore will ensure all appropriate measures are employed to protect the doctor’s privacy.

Confidentiality

All personal information collected through Inpractice and related activities are protected by the Privacy Act 1993.

As part of the terms and conditions, information collected through Inpractice will be available as appropriate to:

• Inpractice staff: all information submitted to the recertification database and all other information that may be held about a doctor.
• The doctor’s collegial relationship provider will have access to the doctor’s ePortfolio for the purpose of reviewing content and commenting on the collegial relationship meetings.
• Doctors employed by Inpractice to undertake a regular practice review (RPR).

No patient identifying data should be entered into the Inpractice ePortfolio system at any stage.

1.8 Extenuating circumstances

Recertification is mandatory for all practising doctors in New Zealand and you are expected to manage your commitments in such a way as to be able to meet your recertification requirements, as the vast majority of doctors do. However we do recognise that from time to time situations will arise that make meeting these requirements difficult, if not impossible. If you are, or anticipate, experiencing difficulty with meeting recertification requirements please contact us as soon as possible. It is considerably easier for us to make arrangements at the time, as opposed to after the end of your CPD cycle.

2 Detailed description of activities

The activities the doctor is required to complete are described in detail in the following section.

The following information is provided for each activity:

Purpose	Describes the function that each activity serves within Inpractice
Requirement	Outlines the actions the doctor is required to complete with respect to each activity
Description	Provides a detailed description of the activity
Standard for completion	Specifies how completion of the activity will be determined
Quality standard	Specifies the standard to which each activity must be completed and is used as the basis of the electronic audit of the ePortfolio
Monitoring	Describes how the activity will be monitored for completion and quality

Recording activities using the web-based ePortfolio

Inpractice provides doctors with secure access to an individual web-based ePortfolio allowing them to create an easily accessible, longitudinal record of their continuing professional development (CPD) activities, which need to be undertaken to satisfy the Medical Council’s recertification requirements.

The Inpractice ePortfolio will be the sole record used for assessing completion of recertification requirements; therefore it is important that all CPD activities undertaken for recertification are recorded in the ePortfolio prior to the end of the CPD cycle (which corresponds to the Annual Practising Certificate renewal date).

It is recommended the ePortfolio is regularly updated as activities are undertaken. Not only will this make it easier at the end of the cycle, it will also allow doctors to demonstrate ‘active participation’ in CPD and recertification should the need arise.

Note: the Medical Council defines ‘active participation’ as follows 'A doctor is actively participating if he or she has completed the most recent branch advisory body recertification cycle and is continuing to report his or her CPD to that body regularly'.

2.1 Professional development plan

Purpose

The purpose of the professional development plan (PDP) is to describe specific learning goals that will guide the doctor’s choice of CPD topics and activities.

Requirement

A PDP is required to be developed upon enrolling in the programme and subsequently at the beginning of each 12 month CPD cycle.

Description

The PDP contains the learning goals for the year. The learning goals should be relevant to the doctor’s scope of practice, reflect any learning needs identified through previous recertification activities and be responsive to the particular needs of the doctor’s practice population.

Ideally the creation of a PDP should be a key focus of the first collegial relationship meeting of the CPD cycle. The doctor should prepare a draft PDP for discussion at this meeting.

It is recommended that the PDP be developed within the first quarter of joining Inpractice.  Each of the goals should be reviewed at the end of the 12 month CPD cycle. Each year a new PDP should be developed, although learning  goals from the previous year may be carried forward.

There is no set number of learning goals required in any one year, although typically three to five goals should be identified. In some instances a single goal if substantial enough will suffice. If a learning goal is anticipated to take more than one year, it is recommended it be broken down into smaller sub-goals that can be achieved within each 12 month cycle.

A PDP template is provided in each doctor’s ePortfolio to enable them to record a brief summary of the learning goals for the year including:

• The name of the goal
• What is planned to be learnt
• How the learning goal was identified
• How the goal can be achieved

Identifying learning needs

Assessment of learning needs may be formal or informal, opportunistic or planned. Each approach has value, and ideally a range of approaches is useful for determining learning needs. Research suggests most people struggle to self-identify learning needs; so it’s important to use some objective assessments to identify learning needs. The Essentials quiz, collegial and patient feedback and regular practice review visits should be useful in this regard.

Create ‘SMART’ goals

‘SMART’ goals are more likely to be achieved. These are goals that are:

• Specific – described in such a way that the goal and what it was intended to achieve can be understood.
  One recognised way of ensuring a goal is specific is to assess it against the six ‘w’ questions:
  • Who is involved?
  • What do I want to accomplish?
  • Where (specify a location)
  • When (establish a time frame)
  • Which (identify requirements and constraints)
  • Why (what’s the purpose or benefit?)
     
• Measurable – specifies how completion of the goal can be determined
• Achievable – the goal should be realistic given the doctors position and resources available
• Relevant – the goal must be relevant to the needs of the doctor, and the goals overall should be relevant to the scope of work undertaken by the doctor
• Time bounded – there must be a specified time by which the goal will be achieved

Standard for completion

A valid professional development plan (PDP) recorded in the doctor’s ePortfolio.

Quality standard

A valid PDP will contain one or more goals along with the following key elements for each goal:

• Goal name
• A description of what was planned to be learnt and how this learning need was identified
• A description of how the goal will be achieved

Monitoring

The following monitoring systems will be used:

• Electronic monitoring of the ePortfolio for completion
• The PDP will be reviewed with the assistance of the doctor’s collegial relationship provider
• The PDP will be reviewed as part of the quality audit of the ePortfolio.
• The PDP will be reviewed as part of the regular practice review (RPR) process

2.2 Essentials quiz

Purpose

The Essentials quiz is an interactive online quiz designed to encourage familiarity with the domains of competence (as described in Good medical practice), Medical Council statements, and to assist the doctor with identifying learning needs.

Requirement

The doctor must complete the Essentials quiz by the first renewal date on entering the Inpractice programme, then once every three years thereafter.

Description

The Essentials quiz is an interactive online quiz designed to encourage familiarity with the domains of competence as described in Good medical practice and in the Medical Council statements. The Medical Council expects all registered doctors to be familiar with these, regardless of their scope or work role, and to follow the guidance it contains. The Essentials quiz may also help identify areas of knowledge or professional skills that the doctor may wish to develop further.

The Essentials quiz is an ‘open book’ formative assessment (i.e. ‘for learning’), is not a pass/fail test. Researching questions is encouraged and links to useful resources are provided for most questions. There are no scope specific questions; instead it explores knowledge of professionalism, communication, collaboration, good medical care and scholarship.

To complete the Essentials quiz, the doctor must correctly answer all questions. Questions answered incorrectly will be re-presented until answered correctly. There are no restrictions on the number of attempts required to complete the quiz.

The Essentials quiz has been developed to a moderate level of difficulty so it is unlikely that most doctors will be able to answer all questions correctly on the first attempt.

The quiz may be parked at any stage, then returned to at a later time. Progress will be saved automatically.

Once the Essentials quiz is completed, a summary report will be available in the doctor’s ePortfolio.

Standard for completion

All questions are answered correctly.

Quality standard

The Essentials quiz must be completed by the first renewal date on entering Inpractice and then once every three years thereafter. It should be completed within the first quarter of the relevant cycle in order to fulfil the goal of assisting with the identification of learning needs and informing the PDP.

Monitoring

The following monitoring systems will be used:

• Electronic monitoring of the ePortfolio for completion
• The Essentials quiz results will be reviewed as part of the quality audit of the ePortfolio.
• The Essentials quiz results will be reviewed as part of the regular practice review (RPR) process
• The Essentails quiz may be reviewed in conjunction with the doctor's collegial relationship provider

2.3 Collegial relationship

Purpose

The purpose of the collegial relationship is to ensure that the doctor’s professional development plan (PDP) and continuing professional development (CPD) activities are appropriate and that the doctor is not working in professional isolation.

Requirement

Inpractice requires all doctors participating in the programme to establish and maintain a collegial relationship with a vocationally registered colleague working in the same or similar scope of practice to that in which the doctor usually works.

Maintaining the collegial relationship is defined as the doctor and colleague having four interactive meetings annually.

Meetings may be conducted face-to-face or at a distance (e.g. teleconference, video phone). The key requirement is that they are simultaneously interactive; email exchanges for example do not meet the requirements.

Description

Colleagues should be a role model of good medical practice, a sounding board for the doctor and a resource in times of difficulty. It is important to note that the collegial relationship is not a supervisory relationship and colleagues are not required to supervise a doctor’s practice. Neither is the colleague responsible or liable for a doctors clinical decisions, unless the colleague has been directly involved in the care of the doctor’s patients.

The doctor is responsible for maintaining the collegial relationship. This means they are required to:

• Organise meetings, and
• Ensure adequate records of the meetings with their colleague are recorded in their ePortfolio

The responsibilities of the colleague providing the collegial relationship are to:

• Be available for meetings
• Ensure the doctor is not professionally isolated
• Review, with the doctor, the appropriateness of the activities recorded in the doctors ePortfolio
• Assist the doctor to review feedback on their practice

To support the colleague in meeting these responsibilities, Inpractice will provide the colleague with independent access to the doctor’s ePortfolio, allowing them to review the doctor’s progress at their convenience. Inpractice also provides a collegial relationship interaction guide including advice on establishing successful mentoring relationships (See Appendix One).

Support from the Inpractice medical advisors will be available for colleagues experiencing difficulties with, or having concerns about, the doctor they are acting as a colleague for.

Standard for completion

Doctors registered in a general scope of practice must record in their ePortfolio a minimum of four collegial relationship meetings per annum. Ideally, these meetings occur at regular intervals throughout the course of the year

Quality standard

A record of each collegial relationship meeting should contain the following information:

• Date
• Type of interaction
• Topics discussed
• Notes from the meeting
• Comments from the colleague (optional)

Monitoring

The following monitoring systems will be used:

• Electronic monitoring of the ePortfolio for completion
• The collegial relationship meeting records will be reviewed as part of the quality audit of the ePortfolio.
• The collegial relationship meeting records will be reviewed as part of the regular practice review (RPR) process

2.4 Peer review

Purpose

Peer review is the supportive evaluation of a doctor’s performance by members of the same profession or team.

Requirement

Each doctor is required to complete at least ten hours of peer review within each 12 month CPD cycle.

Description

Peer review involves the evaluation of the performance of individuals or groups of doctors by members of the same profession or team.

Peer review may be formal or informal and can include any time when doctors are learning about their practice with colleagues. Peer review can also occur in multidisciplinary teams when team members, including other health professionals, give feedback. In formal peer review, peers systematically review aspects of your work, for example, the first six cases seen, or a presentation on a given topic. Peer review normally includes feedback, guidance and a critique of your performance.

Examples of peer review include:

• joint review of cases
• review of charts
• 360° appraisals and feedback
• critique of a video review of consultation discussion groups
• peer review meeting

It is important to keep in mind that peer review is a deliberate reflective exercise. It is intended to allow you to informally and objectively assess your performance, and therefore to continually improve your practice.

Standard for completion

A minimum of ten hours of peer review activity recorded in the ePortfolio within each 12 month CPD cycle. A maximum of two hours of peer review can be claimed for any single activity. If a single activity is greater than two hours, it is recommended it be broken into smaller records. If this can not be achieved, it is recommended the doctor contacts Inpractice for assistance.

Quality standard

The following should be recorded for each peer review activity:

• Key topic
• Date of activity
• Type of activity
• What was learnt
• How patients will benefit
• Time credits

Monitoring

The following monitoring systems will be used:

• Electronic monitoring of the ePortfolio for completion
• Peer review activities recorded by the doctor may be reviewed in conjunction with the doctor’s collegial relationship provider
• The quality and nature of the peer review activities recorded by the doctor will be reviewed as part of the quality audit of the ePortfolio
• The peer review activities will be reviewed as part of the regular practice review (RPR) process

2.5 Continuing medical education

Purpose

The purpose of continuing medical education (CME) is to ensure the doctor maintains an up-to-date clinical knowledge.

Requirement

Doctors are required to complete at least twenty hours of CME within each 12 month CPD cycle.

Description

The focus of CME activities should be guided by the doctor’s professional development plan (PDP).

There are many activities that occur within daily practice that may constitute CME and Inpractice is inclusive of a broad range of activities, in line with the principle that effective CPD is grounded in daily practice. For example CME may include, but is not limited to:

• Attendance at relevant educational conferences, courses and workshops
• Self-directed learning programmes
• Presentations, teaching and preparation of educational material
• Assessments designed to identify learning needs in areas such as procedural skills, diagnostic skills or clinical knowledge
• Observations of procedures
• Writing and publishing reports and articles
• Journal reading

All CME activities should be recorded in the ePortfolio. The doctor is required to record the activity, relevance to their PDP goals, briefly describe what has been learnt, how this will benefit their patients and assign the time to each activity.

Inpractice does not require CME activities to be endorsed in order for them to be counted towards meeting recertification requirements. Endorsement imposes a significant administrative burden and may restrict the range of activities that can be counted, especially those which occur within the context of daily practice.

It should be noted however, that attendance at meetings whose content is developed and presented by pharmaceutical companies or medical device manufacturer representatives is not considered appropriate for the accumulation of CME for the Inpractice programme. Activities cannot be double counted (e.g used for CME and Peer Review), with the exception being the Essentials quiz, which can be recorded as CME for the time taken to complete. The amount of time credits assigned to an activity must be reasonable. Duplicate entries are not acceptable.

Cultural competency

To meet Inpractice requirements, at least two hours of CME must address cultural competence.

These activities should be linked with the following MCNZ documents:

• Statement of cultural competence
• Statement of best practice outcomes when providing care to Māori patients and their whānau
• Best health outcomes for Māori: Practice implications
• Best health outcomes for Pacific People: Practice implications

Examples of activities include:

• Attendance at conferences, courses and workshops focused on Māori, Pacific, Asian and other ethnic health
• Māori, Pacific, Asian and other ethnic language courses
• Reading journals and articles on Māori, Pacific, Asian and other ethnic health

Standard for completion

A minimum of 20 hours CME (including two addressing cultural competence) are recorded in the doctors ePortfolio within their 12 month CPD cycle. A maximum of four hours of CME can be claimed for any single activity. If a single activity is greater than four hours, it is recommended it be broken into smaller records. If this can not be achieved, it is recommended the doctor contacts Inpractice for assistance.

Quality standard

The following information should be included for each CME activity recorded in the ePortfolio:

• Key topic
• Date of activity
• Type of activity
• Rate as a way of learning
• Link to a PDP goal
• What was learnt
• How patients will benefit
• Time allocated to the activity

Monitoring

The following monitoring systems will be used:

• Electronic monitoring of the ePortfolio for completion
• The quality and relevance of the CME activities recorded by the doctor may be reviewed in conjunction with the doctor’s collegial relationship provider
• The quality and relevance of the CME activities recorded by the doctor will be reviewed as part of the quality audit of the ePortfolio
• The CME activities will be reviewed as part of the regular practice review (RPR) process

2.6 Audit of medical practice

Audit is the process of benchmarking or measuring current work practices. If gaps in practice are identified, a plan can be made to improve the practice. Following implementation of the plan, the process is then re-measured at a later date, to determine if the changes were effective in improving the gaps in practice. This process is often referred to as the ‘audit cycle’.

Audit is a frequently used continuous quality improvement (CQI) activity, and is a common component of many CPD programmes encompassing all areas of medicine.

The purpose of an audit of medical practice is to assess, evaluate and improve the care of patients in a systematic way.

The Medical Council has recently changed the requirement to complete a "clinical audit" to a requirement to complete an "audit of medical practice", to reflect that not all doctors are in clinical practice.

The Council has identified seven criteria that should be met when undertaking an audit of medical practice:

1. The topic for the audit relates to an area of your practice that may be improved.
2. The process is feasible in that there are sufficient resources to undertake the process without unduly jeopardising other aspects of health service delivery.
3. An identified or generated standard is used to measure current performance.
4. An appropriate written plan is documented.
5. Outcomes of the audit are documented and discussed.
6. Where appropriate an action plan is developed that will identify and maximise the benefit of the process to patient outcomes. The plan should outline how the actions will be implemented and a process of monitoring.
7. Subsequent audit cycles are planned, where required, so that the audit is part of a process of continuous quality improvement.

There are a number of existing audit tools and packages offered by other organisations which will meet these criteria, and doctors are encouraged to make use of these where possible.

An audit of medical practice may be undertaken by a single doctor or as part of a team. For instance, patients are most often cared for by a team so it is appropriate that an audit of medical practice might assess patient care as provided by a whole team and that the whole team participate in the audit process. This has the other advantage of sharing the workload involved in completing the audit.

If you are having difficulty with the audit requirement we encourage you to seek advice from the Inpractice medical advisors as soon as possible.

Purpose

The purpose of an audit of medical practice is to assess, evaluate and improve the care of patients or practice in a systematic way.

Requirements

The doctor is required to complete one audit of medical practice within each 12 month CPD cycle. Where possible the audit topic should relate to a PDP goal.

Standard for completion

One audit of medical practice must be completed within each 12 month CPD cycle. The topic of the audit, and reason for choosing that topic, should ideally be focussed on patient care or quality of practice. The standards set, methods of data collection and how this was analysed should be recorded where possible. The results and effectiveness of the audit should also be recorded along with any gaps in practice identified.

Quality standard

Each record of an audit of medical practice should contain the following information:

• Audit topic
• Why this topic?
• Audit process
• Results
• Followup

All of these fields need to be entered in order to satisfy the requirements to complete an audit in your ePortfolio.

When deciding on your audit topic, it is important to ask yourself the following question - 'how do I know that what I'm doing is correct'. You can audit any aspect of medical practice as long as you have defined criteria that you are looking for and predetermined standards that you are measuring against. Then you can determine whether your practice meets these standards and potentially find areas where improvements can be made for the future.

 It is important to note the differences between an audit of medical practice and a research study. An audit must include specific criteria and set standards and therefore it will have a result. A research study is the gathering of data and while this is a valuable exercise, it does not satisfy the requirements for an audit. In some cases you may be able to use the data you have collected as the basis for determining the criteria and standards for your audit.

Monitoring

The following monitoring systems will be used:

• Electronic monitoring of the ePortfolio for completion
• The audit of medical practice process and results may be reviewed in conjunction with the doctor’s collegial relationship provider
• The audit of medical practice summary will be reviewed as part of the quality audit of the ePortfolio
• The audit of medical practice will be reviewed as part of the regular practice review (RPR) process

2.7 Feedback from colleagues and patients

Purpose

The primary purpose of feedback from colleagues and patients is to identify strengths and areas for improvement in a doctor’s practice in order to help inform their professional development plan (PDP).

Requirement

At least once every three cycles, or within the 24 months prior to a regular practice review (RPR) visit, each doctor is required to seek feedback on the quality of their professional practice in a systematic way using both the Inpractice colleague feedback and patient feedback tools, where practicable.

Detailed description

Seeking feedback from colleagues and patients is primarily formative (assessment for learning) but may have a summative (assessment of learning) element by providing an additional means to identify doctors at risk of poor performance. Feedback from colleagues and patients is not, however, a pass/fail assessment.

Inpractice provides an online colleague feedback tool and patient feedback packs suitable for use in a variety of practice settings. Each doctor is required to use the feedback instruments provided by the Inpractice programme. The doctor may choose to initiate the feedback process at any time during their CPD cycle.

Colleague feedback

The colleague feedback tool reflects the values and principles set out in Good medical practice and focuses on professional skills, cultural competence and the behaviour of the doctor.

Co-workers and medical colleagues will be nominated by the doctor to complete the feedback process. They must include people working in a range of roles, both clinical and non-clinical, who are sufficiently familiar with the doctor’s current work to be able to provide an informed and representative assessment. Each doctor should aim to receive feedback from a minimum of ten people.

Nominated co-workers and medical colleagues will be contacted by Inpractice, invited to participate, provided with guidance on the process along with a link to the online rating tool. The doctor is required to complete a self-assessment, answering the same questions. Co-workers and colleagues responses will not be individually identified.

Once the self-assessment and the co-workers and colleague assessments have been completed the results will be aggregated and a summary report prepared. Both the doctor and their collegial relationship provider will be notified and provided with guidance as to how best to analyse and use the information contained in the report.

Patient feedback

The patient feedback is a paper based process, based on a questionnaire containing questions that reflect the values and principles set out in Good medical practice with a focus on communication, cultural competence and the behaviour of the doctor.

Patient feedback packs will be available from Inpractice.  The packs will contain all the necessary material to manage the initiation, collection and collation of the patient feedbacks.  These packs will provide advice to ensure that the selection of patients is as representative as possible and that patient confidentiality is maintained. No individual patient’s feedback will be available to the doctor.

Each doctor should aim to receive feedback from a minimum of forty five patients. Once the patient questionnaires have been completed the results will be aggregated and a summary report will be available. Both the doctor and their collegial relationship provider will be notified and provided with guidance as to how best to analyse and use the information contained in the report.

Standard for completion

At least once every three cycles, or within the 24 months prior to a regular practice review (RPR) visit, each doctor is required to seek feedback on the quality of their professional practice in a systematic way using both the Inpractice colleague feedback and patient feedback tools, where practicable.

Quality standard

All doctors should achieve a score of at least 'satisfactory'. Where a doctor receives an aggregated score in any category of ‘less than satisfactory’ or ‘poor’ their feedback will be reviewed by the Inpractice medical advisors. If appropriate, the doctor will be assisted to develop a structured remediation plan to address any identified issues.

Monitoring

The following monitoring systems will be used:

• Electronic monitoring of the ePortfolio for completion
• Aggregate question scores will be monitored electronically for compliance with quality standards
• The feedback report may be reviewed in conjunction with the doctor’s collegial relationship provider
• The feedback report will be reviewed as part of the quality audit of the ePortfolio
• The feedback report will be reviewed as part of the regular practice review (RPR) process

System to identify and remediate poor performance

Results from the colleague feedback, patient feedback and regular practice review visit will be monitored to identify doctors whose performance in a specific area (or overall) has been rated by their colleagues, peers or patients as less than satisfactory or poor.

Where an aggregated rating for any question of ‘less than satisfactory’ or ‘poor’ has been given, the Inpractice medical advisor will contact both the doctor and their collegial relationship provider to discuss the result and work with the doctor to develop a structured remediation plan.

The structured remediation plan provides the doctor with an opportunity to propose actions to address areas of identified poor performance. It would be in addition to the doctor’s PDP and take the form of a brief statement outlining which issues are to be addressed, what actions are to be taken, along with a timeline. The structured remediation plan must be approved by the Inpractice medical advisor.

The doctor’s progress in implementing the structured remediation plan will be closely monitored by the Inpractice medical advisor. The doctor may be required to repeat the assessment (colleague feedback, patient feedback or regular practice review) that identified the issue(s).

For tips on reviewing feedback see Appendix two

3 Managing non-compliance

3.1 Introduction

Each year there are recertification, professional development and collegial requirements a doctor must satisfy in order to qualify for the renewal of their APC.  These recertification requirements help the Medical Council to ensure doctors are competent and safe to practise medicine.

The Inpractice ePortfolio details the type and minimum number of activities required to be completed in each 12 month CPD cycle in order for a doctor to be recertified; these are shown as targets in the activity summary on the ePortfolio home page. Where enrolment in the programme is less than 12 months, targets will have been adjusted on a pro-rata basis at the discretion of the Inpractice medical advisor.

A doctor's CPD cycle corresponds to the dates on their APC; this cycle is displayed at the top of the activity summary on the ePortfolio home page. Doctors need to ensure they have completed and recorded all the required activities by the date they are due to renew their APC. It is recommend activities are recorded as they are completed, to avoid a last minute rush of entering of activities at the time of renewal.

3.2 What happens if requirements are partially incomplete?

continuing medical education (CME), peer review and medical audit

If a doctor has not recorded the required amount of CME or peer review, or completed an audit (if required) by the APC renewal date, the shortfall will be added to the requirements for the next CPD cycle.

Essentials quiz

The Essentials quiz needs to be completed in the first CPD cycle and then once every three years thereafter. If a doctor has not completed the Essentials quiz by the APC renewal date in the cycle that it is due, the doctor will have one month from this date to complete the quiz. If they fail to meet this extended deadline, formal notification of unsatisfactory participation in recertification will be made to the Medical Council.

Feedback on practise

All doctors are required to gather feedback on their practise once every three cycles using both the Inpractice collegial and patient feedback tools (where practicable). If these have not been completed within the three year timeframe, the doctor will have two months from the end of this period to complete them. If they fail to meet this extended deadline, formal notification of unsatisfactory participation in recertification will be made to the Medical Council.

Collegial relationship meetings

Shortfall in meetings

Establishing and maintaining a collegial relationship is considered a key component of recertification for doctors registered in a general scope. However, while high importance is placed on ensuring doctors meet this requirement, it is not considered practical to add any shortfall in meetings to the subsequent CPD cycle. Instead, in order to be able to demonstrate active particpation to the Medical Council, doctors that have had less than the target number of meetings with their colleague, will be required to complete and record collegial relationship meetings on a scheduled basis over the next CPD cycle.

Completing and recording collegial relationship meetings on a scheduled basis means a meeting must be recorded every 3 months. Compliance with this requirement will be monitored through the Inpractice ePortfolio. If a doctor fails to meet this schedule, formal notification of unsatisfactory participation in recertification will be made to the Medical Council.

No meetings

If a doctor has recorded no meetings with their Colleague by their APC renewal date, then they will have one month from this date to complete and record a meeting in order to demonstrate active participation.  If the doctor fails to meet this extended deadline, formal notification of unsatisfactory participation in recertification will be made to the Medical Council. Thereafter the doctor will be required to complete meetings on a scheduled basis over the next CPD cycle. If the doctor fails to meet this schedule, formal notification of unsatisfactory participation in recertification will be made to the Medical Council.

3.3 What happens if requirements are fully incomplete?

If a doctor has not completed any of their recertification requirements by their APC renewal date, formal notification of non-participation in recertification will be made to the Medical Council.

In this instance, re-enrolment in the Inpractice programme will be subject to the doctor agreeing to adhere to a structured remediation plan. This plan will be developed by the doctor with the assistance of Inpractice and will detail how the doctor will ensure they are able to meet all their recertification requirements in a timely manner over the coming year. The plan should include:

• Specific, measureable goals
• Description of the activities that will be undertaken to meet the goals
• Timeline to enable the monitoring of progress

The structured remediation plan must be agreed to by the Inpractice medical advisor. The plan will then be notified to the Medical Council and the doctors colleague.

If no plan is agreed to within one month of being requested, formal notification of non-participation in recertification will be made to the Medical Council. Likewise, if there is significant non-adherence to the agreed plan, then formal notification of unsatisfactory participation in recertification will be made to the Medical Council.

3.4 What happens if requirements are incomplete for two APC cycles

If at the end of a second CPD cycle, requirements carried over from the previous cycle remain incomplete, formal notification of unsatisfactory participation in recertification will be made to the Medical Council.

3.5 Notification to the Medical Council

The Medical Council has authority under the Health Practitioners Competence Assurance Act (the Act) to ensure that doctors satisfy recertification requirements. If doctors do not meet their recertification requirements, there are possible consequences under section 43 of the Act. The Council may (after following due process):

• Alter the doctors scope of practice, by changing the health services they are permitted to perform;
• Place conditions on practice; or
• Suspend registration.

We would therefore strongly encourage all doctors to actively participate in recertification.

3.6 ePortfolio audit for assessing compliance with Inpractice standards

To ensure CPD activities recorded are compliant with the standards set by Inpractice, 100% of doctors’ ePortfolios will be audited by the Inpractice audit team.

The audit process will examine all of the activities recorded in the ePortfolio against the standards established by Inpractice for a valid activity, and will be conducted under the guidance of the Inpractice medical advisor.

3.7 Failure to meet activity standards

Where an audit of the ePortfolio identifies activities which do not meet the required activity standards, the Inpractice medical advisor will review the contents of the ePortfolio to determine what action, if any, is required. The action taken will be dependent on the extent to which the activities recorded in the ePortfolio fail to meet the required activity standards.

Minor breaches of standards

Where minor breaches in recorded activities are found, the Inpractice medical advisor will contact the doctor concerned and notify them of the breach. The doctor will be reminded of the required standards, and a follow up audit of the ePortfolio may be completed to ensure future activities are appropriate.

Serious breaches of standards

Where serious breaches in recorded activities are found, or the Inpractice ePortfolio audit team have determined that there is fraudulent recording of activities, the doctor concerned will be notified and the situation discussed with them. If no satisfactory explanation is forthcoming the Medical Council will be notified the doctor has not met the Inpractice requirements for recertification. The Medical Council has stated that non-compliance with requirements may result in conditions being placed on a doctor’s practice; this may include alteration of a doctor’s scope of practice or suspension of registration.

Appendix one: Collegial relationship guide

The collegial relationship

What is a collegial relationship?

A collegial relationship is an on-going formal relationship between two doctors. The purpose of this relationship is to provide guidance and mentorship for doctors registered in a general scope, with the objectives of maintaining safe clinical practice and facilitating CPD.

The doctor providing collegial support should be a role model of good medical practice, a sounding board for the doctor and a resource in times of difficulty. It is important to note that the collegial relationship is not a supervisory relationship and colleagues are not required to supervise a doctor’s practice. Neither is the colleague responsible or liable for a doctors clinical decisions, unless the colleague has been directly involved in the care of the doctors’ patients.

Mentoring

All doctors need personal, educational and professional support throughout their careers. A collegial relationship is one way of providing such support, as the person providing the collegial relationship can offer a valuable source of mentoring. Mentoring is a type of formal social support that is important for medical professional development for both career selection and advancement.

To make the most out of the mentoring process, it is worth remembering that mentoring is a shared job. While the colleague may provide the support, the doctor must be open and receptive to the feedback, and trust the colleague to speak openly to them.

To develop a successful mentoring relationship, the following attributes are recommended:

• Develop and manage the mentoring relationship: It is worthwhile putting some time into getting to know each other, which will eventually help build trust, leading to a more effective relationship.
• Sponsor: It is useful where appropriate for the colleague to advocate for the doctor. This can help them gain more opportunities and gain visibility within the professional environment.
• Survey the environment: The colleague can keep a watchful eye on the professional environment, being aware of both threats and opportunities for the doctor.
• Guide and counsel: The colleague can serve as a confidant and sounding-board to the doctor. Ideally, the colleague should be able to provide advice on dealing with conflict and help explore ways to deal with problems.
• Teach: If times allows, teaching can be a valuable aspect of mentoring. This can include sharing knowledge as well as reflecting on experiences and challenges.
• Model: The colleague should be a role model of high standard. The doctor will be influenced by their behavior, which includes: ethics, values and standards, style, beliefs and attitudes, methods and procedures.
• Motivate and inspire: Colleagues should support, validate and encourage the doctors they provide collegial support to. Linking goals, values and emotions to the larger organisational agenda, can mean doctors become more engaged in their work and in their own development.

In this role, the collegial relationship provider can support doctors to achieve their potential, enable them to make progress and to reflect on problems in order to move forward. They are in an ideal position to assist the doctor in creating a plan for working toward their professional development goals and will provide the doctor with insights into the realities of building a medical career.

Who can provide a collegial relationship?

The person providing the collegial relationship needs to be vocationally registered in the same or similar scope of practice to the doctor. Doctors still in vocational training, or waiting for their change of scope to be registered by the Medical Council, are not eligible to provide the collegial relationship.

Ideally the person providing the collegial relationship should have certain characteristics, such as:

• A diverse background and interests that are similar to the doctor
• A good rapport with colleagues and peers
• An open mind and commitment to ongoing education
• Knowledge in the area of interests of the doctor

It is preferred that a collegial relationship is established between two doctors who are based in the same geographic location. If this is not possible, the relationship may be set up at a distance. If face to face meetings are not always possible, the meetings can be supplemented by interactive meetings using the telephone or video conferencing.

What is the person providing the collegial relationship required to do?

The person providing the collegial relationship will be expected to:

• Be available for four meetings per annum
• Assist the doctor to develop a professional development plan (PDP) each year
• Informally assess progress on the PDP and review needs
• Ensure the doctor is not professionally isolated
• Assist the doctor in any other mutually agreed way to enhance his or her practice, skills and personal growth

The collegial relationship provider will have access to the doctor’s ePortfolio containing a record of their CPD activities and assessments.

What are the responsibilities of the doctor establishing the collegial relationship?

The doctor is required to:

• Organise meetings (preferably face-to-face) lasting approximately one hour
• Ensure adequate records of meetings are kept
• Prepare in advance for meetings to ensure meetings are of benefit and the best use is made of the time

Finding someone to establish a collegial relationship with

The person providing the collegial relationship needs to have vocational registration in the same or similar scope to the doctor. The vocational registration of a particular doctor can be checked by going to https://www.mcnz.org.nz/support-for-doctors/list-of-registered-doctors/. When establishing a collegial relationship, the colleague should first be approached to ensure they are happy to provide it.

Some doctors may be nervous about approaching a more senior doctor, but experience shows most doctors approached to provide a collegial relationship are more than willing to help. It is worth remembering that most ongoing learning in medicine is based on collegiality, and all doctors will have been involved in collegial relationships previously (either by providing them or during their training). Therefore approaching a doctor to provide the collegial relationship will be a request that doctors will be familiar with.

Changing collegial relationship provider

If a doctor changes the person with which they have a collegial relationship, they are required to update their Inpractice profile with the details of the new person. The colleague will then be contacted by Inpractice.

Do you need to change collegial relationship provider when changing runs?

Some doctors are required to complete various ‘runs’ during their first year of registration, but it is not always necessary to change colleagues with each run. It is important to remember your colleague is there to provide support guidance and mentorship, not to supervise your practice.

Once a doctor has settled into one particular branch of medicine it is strongly advised that at that time a collegial relationship is established with a doctor vocationally registered in that branch.

If the person providing the collegial relationship is unavailable, can an alternative collegial relationship meeting be held with another doctor?

The relationship between the doctor and the colleague benefits from the ongoing nature of the collegial relationship, and the collegiality can be expected to increase as time progresses. Therefore the collegial relationship should be viewed as a long term relationship.

There may be times when the usual collegial relationship doctor is unavailable. If this occurs, the doctor may have a one-off meeting with another colleague who is vocationally registered. In this situation, the meeting should be entered as usual into the profile, but the MCNZ number of the doctor with whom the meeting took place should be entered.

What should be discussed in a collegial relationship meeting?

One of the key purposes of the collegial relationship is to provide guidance and mentorship for the doctor. These meetings represent an opportunity for the doctor to discuss in a non threatening environment things that may be impacting on their professional lives and their ability to practice medicine safely and competently.

A key role of the colleague is to help guide the doctor’s professional development. To assist in fulfilling this role, outlined below is a number of activities focussed on professional development that would be suitable topics for a collegial relationship meeting. These are not compulsory but are strongly recommended.

It is important that collegial relationship meetings are not used as ‘how to treat’ teaching sessions. This type of activity is better considered peer review or CME as it fails to make the most of the unique opportunity for mentorship, guidance and support that the collegial relationship provides.

Collegial Relationship meeting schedule

Below you’ll find a list of suggested topics to guide the content of your collegial relationship meetings.

Many of these suggestions focus on activities recorded within the doctor’s ePortfolio, therefore it will be useful to have access to the internet during meetings

Topic suggestions:

1. Professional development plan
2. Reviewing colleague feedback
3. Reviewing peer review activities
4. Reviewing continuing medical education (CME) activities
5. Regular practice review visit
6. Audit
7. Discussions of non-clinical issues
8. Self care and career planning
9. Peer relationships
10. Future planning, including moving to vocational training or planning retirement

Prompts for guiding discussion in collegial relationship meetings

1. Professional development plan (PDP)

The primary purpose of collegial relationship meetings is the development and review of the doctor’s PDP.

This section of the guide provides some suggested prompts to help frame your discussions in collegial relationship meetings to encourage reflection on the PDP.

The data to inform the discussion can be drawn from formative assessments that help with the identification of areas for improvement that may then be developed into PDP goals. These assessments include colleague and patient feedback and the regular practice review.

The discussion should also cover:

• Activities that involve evaluation of performance such as audit and peer review.
• Activities that help with attainment of PDP goals by increasing knowledge, skills, and professional performance, for example CME.

Access to a range of information will allow the doctor to discover areas for development as well as their areas of strength. A useful assessment of potential learning needs can then be made which, with further consideration and discussion, should result in an action plan (goals) in the doctor’s PDP to address these areas.

Over the course of an annual cycle it is important to revisit progress towards achieving the goals in the PDP, to indicate where these have been completed and potentially add new ones, where appropriate. The colleague can help the doctor ensure that their professional development activities are well chosen and are an effective and efficient use of the doctor’s time and energy.

Feedback to inform learning needs

There is good evidence that doctors often find it difficult to identify their own learning needs. This section of the guide provides some suggested prompts to help frame your discussions in collegial relationship meetings to encourage reflection on the PDP utilising feedback to inform learning needs.

Forms of feedback include:

2. Colleague feedback reports

Such reports provide a good opportunity for those who are most familiar with your practice to provide a reasonably objective assessment of your strengths and identified areas for improvement.

In reviewing these reports, it can be helpful to reflect on the following:

• Are there any domains where there are a wide range of scores, why might colleagues or patients have such differing opinions of you in this area?
• Are there any areas where your self-assessment differs significantly from the assessment of your peers?
• Your results relative to those of your peers - the average scores doctors achieve in these assessments tend to be high - are there any areas where your scores are significantly below those of your peers i.e. outside the standard deviation?
• During the process of soliciting feedback did you consciously, or unconsciously, make any changes from the way you would normally practise? Why?
• In what ways could your practice change as a result of this assessment? Can this process of change be turned into a PDP goal?

3. Peer review

Peer review involves evaluation of the performance of individuals or groups of doctors by members of the same profession or team. It may be formal or informal and can include any time when doctors are learning about their practice with colleagues.

When discussing peer review, it can be helpful to reflect on the following:

• Do the peer review activities focus on the evaluation of performance by peers?
• Is there evidence of reflection on the standards of performance?
• Are any areas of performance identified as requiring improvement? Can these be addressed by developing them into PDP goals?
• How do you ensure patients benefit from your participation in peer review?

4. Continuing medical education (CME)

CME consists of educational activities which serve to maintain, develop, or increase the knowledge, skills, and professional performance and relationships that a doctor uses to provide services for patients, the public, or the profession.

When reviewing CME activities, you should reflect on the following:

• How well does the CME that has been recorded support the achievement of the PDP goals?
• Does the current approach to CME provide the most value for the limited learning time available? Medical Council of New Zealand – Collegial Relationship Guide - 23/11/2018
• What improvements, if any, could be made to the way CME is approached and recorded?
• What changes to your practice will you make as a result of the CME being undertaken? How would patients, the public or the profession benefit?

5. Regular practice review (RPR)

RPR is a supportive and collegial review of a doctor’s clinical practice and professional development activities.

The primary purpose of the RPR is to provide doctors with independent feedback on their practice in order to help them identify both areas for improvement and learning needs in order to appropriately target professional development activities. The RPR is a significant formative assessment and it is recommended that time be spent reflecting on both the process and outcome with your colleague.

When reviewing an RPR, it can be helpful to reflect on the following:

• It is common for doctors to feel anxious in the lead up to a review, what if anything were you concerned about? Why?
• During the visit did you consciously, or unconsciously, make any changes from the way you would normally practise? Why?
• What was your feeling at the completion of the visit? Were there areas where you felt reassured? Uncomfortable?
• What areas of your practice were identified as strengths or areas for development? Did any of these surprise you?
• What areas for development have /can you turn into PDP goals?

6. Audit

Audit entails a systematic, critical analysis of the quality of a doctor’s own practice and is used to improve clinical and/or health outcomes, or to confirm that current management is consistent with current available evidence or accepted consensus guidelines. Audit involves a cycle of continuous improvement of care, based on explicit and measurable indicators of quality.

When reviewing an audit, it can be helpful to reflect on the following:

• Can the findings of the Audit be used to improve outcomes or to confirm that current management is consistent with current best practice?
• How could your practice change? How could your patients benefit from this audit?
• Where a gap has been identified, what learning/professional development activities or tasks could you undertake to help close this gap? Can this be turned in to a PDP goal?

7. Discussion of non-clinical issues

The colleague can also be a sounding board for the doctor's ideas and a resource in time of difficulty. Therefore, it’s anticipated that meetings may include discussions about non-clinical issues such as self-care, career planning and relationships with peers and others.

8. Self-care

During discussions about self-care, it can be helpful to reflect on the following:

• What things about your work do you enjoy? What do you not enjoy?
• How is balance maintained between work, leisure, holidays and family?
• What do you do to assess and maintain your health in the widest sense – physical, mental, family, and spiritual?
• Do you have appropriate support structures in place?

9. Career planning

Career planning has benefits for everyone. No matter what stage you are at in your medical career, having a plan can help you achieve both career and lifestyle goals.

During discussions about career planning, it can be helpful to reflect on the following:

• Are you considering a change in career or lifestyle? Do you have a plan for achieving this?
• If you have no plans for change what can you do to maintain satisfaction in your current role?
• If you are considering retiring, what is your plan for doing this? Have you considered making changes to your work load, case mix complexity and after-hours work? Is there the opportunity to transition to new roles such as mentoring, teaching or governance? How will your retirement impact on your patients and peers and what can be done to mitigate these impacts?

10. Peer relationships

The importance of maintaining healthy working relationships with colleagues and peers is recognised throughout the medical profession. The great majority of doctors behave appropriately and work well together. However, at times working relationships can become strained for a variety of reasons. When this occurs, it can create a significant amount of stress.

During discussions about peer relationships, it can be helpful to reflect on the following:

• Do you have any difficult relationships with colleagues?
• How might these difficult relationships affect your ability to do your job?
• How can the relationship be improved? What external assistance might be required?

Meeting: Planning for the future

Purpose: To discuss with the doctor any plans for the future, and in the case of older doctors, provide an opportunity to discuss succession planning and retirement.

Suggested timing: It is recommended that discussions about retirement should start several years before the doctor intends to retire.

Discussion: During the course of any career there are various stages along the way. If appropriate, it is useful to consider some issues that might be applicable to the doctor, and consider how they could be guided through this process.

Younger doctors might consider pursuing vocational training. This is an area where the collegial relationship provider can provide specific guidance and share their experience. It would be worthwhile to ensure the doctor is clear about the process for applying for a vocational training programme, that they are aware of any specific entry requirements, and are clear about relevant dates.

Older doctors should start considering their retirement plans, well before they intend to retire. Retirement is a major life event that is likely to have major impact on one’s life. It is important to start considering a retirement plan early; this gives the doctor the opportunity to incorporate retirement into their future, and to give themselves sufficient time to adapt to this significant change. The risk of no retirement planning is that the doctor may have the decision taken out of their control, in the event they are required to finish work suddenly due to ill health, or due to cognitive decline. Suddenly finishing work is likely to be significantly more traumatic.

Appendix two: Tips for reviewing feedback

Interpreting your patient feedback report

Once all responses have been completed, a report will be generated and available for viewing in your ePortfolio. This report contains the aggregated responses of your patients who completed questionnaires. The report doesn’t contain any individual responses, except for the results from your self assessment questionnaire.

Results will be presented on a simple bar graph:

(orange) self assessment: how you rated yourself

(green) patients average: average result for all patients who completed your survey

(blue bar and line) average and standard deviation for all feedbacks completed in a year

Analysing your report

You should take time to analyse your report and reflect on the following:

1. Patient sample
   • What proportion of patients considered their consultation was very important or reported they were seeing their usual doctor? Patients who consider their consultation was ‘very important’ or who reported seeing their ‘usual doctor’ tend to give more favourable feedback.
   • Are the demographics of your patient sample representative of the patients that you usually see?
   Consider whether any of the above factors could have affected your results.
2. Distribution of responses
   • What is the spread or range of responses for each question? If there is a wide range consider why there might be such disparate views among your patients
3. Identifying areas of strengths and weakness
   • Do the results show any obvious areas of strength and weakness?
   • Concentrate on those results that fall farthest from the average, rather than whether your scores are higher or lower than average.
   • How do these compare to your self assessment? Are there areas of strength or weakness you were unaware of?
   When interpreting results, it is recommended you focus less on individual scores, and more on overall trends and relative scores. By comparing your own results to that of all doctors, as well as comparing your own results for the various attributes, the effect of outliers is removed and genuine areas for improvement are more likely to be identified.

Acting on the results

It is recommended that you meet with your collegial relationship provider to discuss your report. It is strongly recommended you consider the outcome of your report in terms of an opportunity for quality improvement, and this should translate into specific goals incorporated into your professional development plan.

Interpreting your colleague feedback report

Once all responses have been completed, a report will be generated and available for viewing in your ePortfolio. This report contains the aggregated responses of your colleagues who completed questionnaires. The report doesn’t contain any individual responses, except for the results from your self assessment questionnaire.

Results will be presented on a simple bar graph:

(orange) self assessment: how you rated yourself

(green) patients average: average result for all patients who completed your survey

(blue bar and line) average and standard deviation for all feedbacks completed in a year

Analysing your report

You should take time to analyse your report and reflect on the following:

1. Colleague sample
   • How did you decide which colleagues to nominate?
   • Did you get an appropriate mix of colleagues?
   • Is the sample representative of the colleagues that you usually work with, and the colleagues who were nominated to provide feedback?
   Ideally feedback should be provided by a balanced mix of medical and non-medical colleagues.

   Note that research shows colleagues in managerial or administrative roles, and health professionals in non-medical roles, tend to give more favourable feedback than medical colleagues. Colleagues with whom you have more frequent contact tend to give more favourable feedback.

   Consider whether any of the above factors could have affected your results.
2. Distribution of responses
   • What is the spread or range of responses for each question? If there is a wide range consider why there might be such disparate views among your colleagues.
3. Proportion of invalid responses
   • What proportions of the responses are invalid, i.e. the colleague picked the ‘don’t know option or did not answer the question. A high proportion of invalid responses may suggest inappropriate colleague selection and will affect the usefulness of the results. In this instance you should consider repeating the process.
4. Identifying areas of strengths and weakness
   • Do the results show any obvious areas of strength and weakness?
   • Concentrate on those results that fall farthest from the average, rather than whether your scores are higher or lower than average.
   • How do these compare to your self assessment? Are there areas of strength or weakness you were unaware of?
   When interpreting results, it is recommended you focus less on individual scores, and more on overall trends and relative scores. By comparing your own results to that of all doctors, as well as comparing your own results for the various attributes, the effect of outliers is removed and genuine areas for improvement are more likely to be identified.

Acting on the results

It is recommended that you meet with your collegial relationship provider to discuss your report. It is strongly recommended you consider the outcome of your report in terms of an opportunity for quality improvement, and this should translate into specific goals incorporated into your professional development plan.

Suggestions on the feedback interview: Doctor and Colleague

A Collegial meeting should be conducted in protected time with no interruptions. It will require excellent skills of giving feedback on the part of the Colleague and receiving feedback on the part of the Doctor. Different individuals may require different lengths of time for reflection. Discussion should centre on the doctor’s expectations in relation to their scores. Some writers have suggested a move from the more traditional role as ‘;provider of feedback’ to a ‘facilitator of feedback’ – these are Colleagues who engage the doctor in a reflective and clarifying discussion about the feedback, its impact and potential outcomes.

The power of feedback to change behaviour depends on the quality and approach of the feedback given. Ideally, Colleagues giving feedback ought to have some training. Skilled, unthreatening feedback is particularly important when the doctor is learning new behaviours.

Skilled feedback needs to be backed up by educational planning, which itself implies familiarity with educational goals through a robust professional development plan.

Using Feedback

The doctor and colleague may wish to reflect on the following points:

• the process used to nominate colleagues
• the number of questionnaires returned
• the characteristics of the colleague sample including the proportion of ‘valid’ responses
• the proportions of ‘Don’t know’ response option
• the distribution of responses
• the range (or spread) of responses across the scale
• variance across the different items on the questionnaire
• the response options that colleagues have used most commonly
• whether colleague ratings are mainly positive, neutral or negative
• possible areas of relative strength and weakness
• obvious areas of strength or weakness,
• where any items of the doctor’s self-rating was unexpectedly higher than the average rating given by their colleagues
• where any items of the doctor’s self-rating was unexpectedly lower than the average rating given by their colleagues (i.e. areas of relative ‘strength’ they may have been unaware of)
• and ways in which they might improve the way they work.

The doctor is encouraged to incorporate the feedback report into their future professional development plan, alongside their own priorities. If several areas of ‘weakness’ are identified, the doctor may wish to prioritise one or two areas of practice, and tailor an action plan towards these first and consider a repeat of the survey at a later date (for example, a year later).

Engaging the doctor's emotional reaction, identifying objective results, and developing an action plan has been shown to improve the acceptance and utility of the feedback, clinical care, professional development, working with others or in teams and personal development; e.g. building self-confidence generally and in clinical skills and patient interactions. The record the discussion and the action plan arising from the Collegial meeting should be recorded in the doctors ePortfolio within the Inpractice website.

1. Sargeant J, Mcnaughton E, Mercer S, Murphy D, Sullivan P, Bruce DA. Providing feedback: Exploring a model (emotion, content, outcomes) for facilitating multi-source feedback. Medical Teacher. 2011;33(9):744–9.

Further Reading and other References

Wood L, Hassell A, Whitehouse A, Bullock A, Wall D. A literature review of multi-source feedback systems within and without health services. Medical Teacher. 2006;28(7):185–91.

Campbell JL, Richards SH, Dickens A, Greco M, Narayanan A, Brearley S. Assessing the professional performance of UK doctors: an evaluation of the utility of the General Medical Council patient and colleague questionnaires. Quality and Safety in Health Care. 2008 Jun 1;17(3):187 –193.

Baker R. Can poorly performing doctors blame their assessment tools? BMJ. 2005 May 28;330(7502):1254.

Archer J, Norcini J, Southgate L, Heard S, Davies H. mini-PAT (Peer Assessment Tool): A Valid Component of a National Assessment Programme in the UK? Adv in Health Sci Educ. 2006 Oct;13(2):181–92.

Tyler KM. Peer‐level multiple source feedback for fitness to practice. Medical Education. 2006 May 1;40(5):482–3.

Sargeant J, Mcnaughton E, Mercer S, Murphy D, Sullivan P, Bruce DA. Providing feedback: Exploring a model (emotion, content, outcomes) for facilitating multisource feedback. Medical Teacher. 2011;33(9):744–9.

Evans R, Elwyn G, Edwards A. Review of instruments for peer assessment of physicians. BMJ. 2004 May 22;328(7450):1240.

Vleuten CPM. The assessment of professional competence: Developments, research and practical implications. Adv Health Sci Educ. 1996;1(1):41–67.

Bailey C, Fletcher C. The impact of multiple source feedback on management development: Findings from a longitudinal study. Journal of Organizational Behavior. 2002;23(7):853–67.

Bono JE, Colbert AE. Understanding Responses To Multi‐Source Feedback: The Role Of Core Self‐Evaluations. Personnel Psychology. 2005 Mar 1;58(1):171–203.

Archer JC, Norcini J, Davies HA. Use of SPRAT for peer review of paediatricians in training. BMJ. 2005 May 28;330(7502):1251 –1253.

Schuwirth LWT, Southgate L, Page GG, Paget NS, Lescop JMJ, Lew SR, et al. When enough is enough: a conceptual basis for fair and defensible practice performance assessment. Med Educ. 2002 Oct;36(10):925–30.

Appendix three: Regular practice review (RPR) visits

Inpractice RPR is a supportive and collegial review of a doctor’s clinical practice and professional development activities.

The RPR process is designed to:

1. Provide doctors with independent feedback on their practice in order to help them identify both areas for improvement and learning needs in order to appropriately target professional development activities.
2. Ensure doctors have the knowledge, skills, attitudes and judgment required to practise safely and to an acceptable standard.

The ultimate goal of RPR is to improve the quality of care that a doctor’s patients receive by facilitating the doctor’s professional development.

The basic principles underpinning the RPR process are that it is:

• Robust
• Supportive and collegial
• Informed by information contained in the doctors ePortfolio, observation of the doctor in practice and discussion with them
• Results in identified learning needs

Eligibility

All doctors enrolled in the Inpractice Programme who have held general registration for a period of three years or more are required to participate in RPR once every three years.

If a doctor is selected a RPR while they are undergoing a review process with the Medical Council, they should advise Inpractice, as they may be eligible to have the RPR deferred.

Selection

Eligible doctors will be selected on a random basis for RPR.

Cost

There will be no additional charge to doctors selected for RPR as this component of the programme is covered in the enrolment fee.

RPR process

Each RPR is undertaken by one experienced and vocationally registered peer trained as an RPR reviewer. The RPR consists of a four components:

• Selection and organisation of visit
• Pre-visit review of the doctors ePortfolio
• Practice visit
• Report back the doctor

Selection and organisation of visit

When selected for an RPR the doctor will be notified approximately 2 months in advance of the proposed month of the visit. The doctor will be asked to ensure the pre-visit requirements have either been completed or underway and to nominate suitable days for the practice visit.

Based on availability and the nature of the doctor’s practice, a reviewer will be selected. The doctor will be notified of the name of the proposed reviewer and provided with the opportunity to request an alternative reviewer if they believe there is a potential conflict of interest.

The doctor will be provided with an RPR participant pack one month prior to the date of the practice visit which contains detail on the arrangements and programme for the day of the visit.

Pre visit review of ePortfolio

Once arrangements for the RPR have been confirmed the reviewer will be provided with access to the doctors ePortfolio. They will review the following, with a view to discussing professional development during the practice visit:

• The professional development plan
• CME, peer review and medical audit activities undertaken
• Meetings with the collegial relationship provider
• Results of patient and colleague feedback

Prior to the visit the reviewer will also:

• Be provided with a copy of the doctors bpac^nz prescribing report where available
• Speak with the collegial relationship provider

The doctor is required to have completed both a colleague feedback and a patient feedback in the 24 months prior to the visit. These can both be initiated on the assessment page within the Inpractice ePortfolio.

The doctor is encouraged to ensure their ePortfolio is up-to-date and provides a full record of CPD activities completed to date.

Practice visit

The practice visit will take place in the doctors usual practice setting, typically taking around 5 to 6 hours to complete. It will involve both time observing the doctor in consultation with patients with time set aside for review of various aspects of practice.

Activities will include:

• An initial interview between the doctor and the reviewer
• Review of the ePortfolio including results of patient or colleague feedback
• Observation of patient consultations
• Records review
• Case based discussion of individual cases exploring clinical reasoning
• Discussion of reviewers observations and recommendations for ongoing professional development

The reviewer may also wish to talk to the doctors colleagues during the day.

The reviewer will use an Inpractice Assessment Guide to record their observations of the various aspects of the doctors practice and their recommendations for professional development.

RPR report

Following the RPR visit the reviewer will return the completed Assessment Guide to Inpractice. The contents will be reviewed by the Inpractice medical advisors and a report compiled following a standard format which will be posted in the doctor's ePortfolio. This process will take approximately two weeks and the doctor will be notified when the report is available.

The doctor will be provided with a guide for reviewing the report and are encouraged to discuss it with their collegial relationship provider.

Where concerns are raised in a report

1. Where there are small areas of practice identified that need improvement, the doctor is encouraged to target their CPD activities to those areas by incorporating appropriate goals into their professional development plan.

2. If the areas identified are more significant, the doctor may be required to develop a more formal structured remediation plan in conjunction with the collegial relationship provider and the Inpractice medical advisors. This plan would include:

   • Specific, measureable goals
   • Descriptions of the activities that will be undertaken to meet the goals
   • Timeline to enable the monitoring of progress

3. Where a concern regarding competence or fitness to practice is raised in a report, the Inpractice medical advisor will discuss the issue with both the reviewer and the doctor concerned. They will determine whether the issues warrant referral to the Medical Council or can be addressed through a formal structured remediation plan.

   The Inpractice medical advisor may, at their discretion and without revealing the identity of the doctor, seek advice and guidance from the Medical Council Medical Advisors.

   When reviewers have concerns that a doctor’s practice is placing patient health and safety at risk, the reviewers and Inpractice have a professional obligation to report this separately to the Medical Council, just as they would do if concerns about a colleague had been identified in any other way.

   Should a notification of a concern be made to the Medical Council, the Medical Council will consider the information through its usual processes and decide whether a performance assessment is necessary.

Right of appeal

If a doctor has concerns about any aspect of the RPR they should raise these with the Inpractice manager in the first instance.

When the concern relates to an aspect of the RPR process itself this will be dealt by the Inpractice management team. They will undertake to find a fair and transparent resolution.

When the concern relates to the RPR Report, this will be dealt with by the Inpractice medical advisor. Again they will undertake to find a fair and transparent resolution.

If the concern is unable to be resolved by the Inpractice medical advisor, the doctor may request a review by the Inpractice Clinical Advisory Group. The Inpractice Clinical Advisory Group may propose a range of remedies from having the concerns recorded and included in the RPR report through to, in exceptional circumstances, requiring another RPR be undertaken.