What Are the Nurse Practitioner Core Competencies
From training to practice: how prepared are students of osteopathy to be independent practitionersAre they fit to practice?
The purpose of this paper is to overview various methods of assessing undergraduates within different medical institutions and compare the monitoring of preparedness of newly qualified osteopaths with other medical graduates for practice. Do osteopathic educational institutions take any responsibility for their graduates after they qualify, and how much monitoring and support are provided by the statutory regulatory body after new graduates are given license to practice?
In the last two decades there has been an increase in number of osteopathic colleges in the UK as well as within European Union. These newly formed educational institutions often work together with existing, well established programmes.
The present structure of osteopathic education in the UK as well as in Europe vary from full time four year or part time five year courses at the end of which successful graduates qualify with bachelor’s degree in osteopathy – BSc(Hons), BOst or BOstMed – or a masters degree in osteopathy (MOst) and are able to enter onto register and practice as an osteopath. Every programme is validated by university and accredited by General Osteopathic Council (GOsC) (1), which is a statutory regulating body in the UK. All osteopaths in the UK are regulated by GOsC and it is against the law for anyone to call themselves an osteopath unless they are registered with GOsC.
Like most of medical curricula osteopathic programmes are mapped against learning objectives in view of testing student’s knowledge, skills and attitude.
It is fundamentally designed to drive assessments which in turn provide evidence to make a decision to reward student an appropriate qualification and allow him to practice.
So, how do we set to assess competence?
If competence means the ability to do something well is that enough to deem a student fit to practice It is very difficult to define and measure, therefore assess competence especially with regards to clinical skills and capability.The GOsC favours term ‘capability’ but this description means that being capable is having a talent or ability for further development. I prefer word ‘competent’ even with all the complexities of defining it.
At which point do we ask that important question: is the student fit to practiceSurely, at the end of their training seems to be rather late to be asking. I believe we do not examine this issue enough throughout the course of the training. Initial interview amounts to verifying academic qualifications, inviting student to join on the course rather than making sure that they are aware that the profession they entering at the end of this learning journey has real responsibilities, social accountability and ethical commitment. If colleges are expected to charge students such a high fee are we going to ask this question at all Do we keep our students on the course regardless of their suitability for the job especially when they are capable academically and pass all the exams and hope that they will learn what they need to learn along the way I do appreciate that we should not act on a hunch or a gut feeling but surely many experienced clinicians know which student will and which will not ‘DO’.
Before we question fitness to practice further lets overview the learning process of medical and osteopathic student based on various assessments that are in use in current medical training.
Most curricula in medical and osteopathic colleges identifies objectives in terms of specific technical and clinical skills together with core of the subjects and professional and personal attitudes, therefore problem presents as to how to best assess it all, making sure it is appropriate at all different stages of student’s training. It is clear that there is a need for plethora of methods and formats to fulfill as many requirements as possible.
Historically, choices with regards to the method of assessment use to depend mainly on their validity and reliability, as they are not only critical but can be proven. Validity being the degree to which the inferences made about medical competence based on assessment scores is correct (Messick, 1989). Reliability or generalizability is a measure of the relative magnitude of variability in scores due to error, with the aim of achieving a desired level of measurement precision (Shavelson & Webb, 1991).
Many models have been introduced over the years to show educational journey that students undertake to become experts in their chosen fields; Miller (1990) in his pyramid of competence below demonstrates how this is achieved via different assessments.
The first layer of the pyramid “Knows/Knowledge” represents how the assessments aim at testing straight recall of knowledge. Many different formats will serve that purpose; they include factual tests such as Multiple Choice Questions (MCQ), essays and oral examinations. The most popular written tests are MCQ used very extensively in education to assess knowledge. True/false or best option choice is amongst the favorites.
Schurwirth & Vleuten (2004) point out some intrinsic disadvantages of MCQs; for example when candidate answers correctly but it is but not based on right knowledge. They considered to be artificial rarely reflecting real life clinical situations.
Quinn (2000) supported by Brady (2005) point out further limitations as far as their content is concerned. They also suggest that questions should be short, understandable and discriminating.
But as they are easy to mark, reliable and objective they are being widely used. If they are poorly designed they only test low-level recall knowledge. As long as they are well designed MCQs combined with other assessments can play important part of the teaching and learning process.
Schurwirth & Vleuten (2004) continue their overview of another written assessment where context-free questions are used. Again they best at determining factual knowledge and as knowledge is important in medical competence it is an essential that they are used. To insure validity of this type of testing they recommend clarity as to the purpose of the content being tested.
In the next level of Miller’s pyramid of competence “Knows how/Understand”, where demonstration of problem solving is required similar assessments are being used but emphasis are more on the “context-rich” approach so candidates are able to reason and negotiate different pieces of information before making a decision. Therefore these tests will have their structure adjusted to test required knowledge across various contexts.
Subsequently when medical undergraduates are expected to apply their knowledge and “show how”, the transition from the factual knowledge (“knows”) testing in respect of clinical practice much different types of assessment are required. These may be simulations, standardized patients, computer-based simulation, computer programs and model-driven simulations and so on; I will concentrate on more traditional types which are widely used in osteopathic colleges.
Essays and oral exams together with long and short cases are no longer being used in North America but remain popular in UK and other European countries; but as consistency of marking as well as examiners variability are amongst many issues falling into question of reliability of those types of tests this has led to increase in use of different methods of assessments, especially when we need to test candidates ability to apply their knowledge so they need to “show how”.
Long case as a format of assessment, still widely used as a part of the Final Clinical Competence Exam in the final year of osteopathic education but less and less in other medical institutions.
It resembles real life clinical situations often unpredictable therefore authentic.
Ponnamperuma et al (2009) describes long case as follows: “The long case is a traditional clinical examination that assesses candidate competence at the ‘shows how’ level in Miller’s pyramid.2 The candidate initially spends time (30–60 minutes) with a patient, taking a history and carrying out physical examination, without examiner observation. Then the candidate presents the findings to one or more examiners and answers oral questions. In most instances each candidate is given a unique patient and a unique examination. Traditionally, the candidate is scored with unstructured marking criteria that are based on neither standardised checklists nor on rating scales with descriptors related to candidate competence.”
To outline a number of the advantages and disadvantages of this type of assessment we need to look at common features of the format and how it can be affected by unpredictability of certain factors:
- variability of examiners – difference in examining experience, possible bias, harsher/softer markers
- variability of patients on a day, even possibility of cheating
- patients presenting with symptoms that are more familiar to some students than others (similar cases experience during studies)
- students not really sure as how to manage time on different parts of the exam (case history taking, examination, patient management etc.)
These factors may influence the validity and reliability of the assessment but because real patients are used it is certainly authentic and close to real life after graduation. It is also a very good indicator of student’s development. It can be used to identify any weaknesses in individual students as well as being reflective of different parts of the educational programme (technique, clinical skills etc).
Fletcher (2008) quotes Godfrey and Heylings who propose several solutions to improve validity and reliability of the long case:
- standardization (or simulation) of the patient, e.g. using trained or primed patients
- ensure adequate examiners training
- observe the encounter and specify criteria for observation and assessment
- increase the number of long cases contributing to a decision on a candidate
- keep a record of the long cases being allocated linking this to the candidate’s course content
- do not use the long case to test obscure knowledge
In my view long cases are exceptionally valuable in the formative evaluation of the student’s abilities in the clinical setting. It also provides an important platform for constructive feedback from more experienced colleagues as well as from clinic tutors but it is also a great opportunity to gather knowledge of different approaches from different clinicians. It is however, unless sufficiently modified, not the most reliable assessment especially in the summative, high stakes context especially if this is the one and only instrument of assessment. To improve reliability is often far too expensive for individual colleges.
As a possible solution to the issues already mentioned in long-case evaluation objective structured clinical examination (OSCE) has been proposed. OSCE was first introduced in June 1999 as a part of the final-year clinical examination. The short (five minute) station format was adopted and used in most medical schools in the UK (Newble 2002) as well as many licensing bodies (e.g. General Medical Council).
OSCE’s format in many osteopathic colleges consists of various stations around which students are tested. They are usually between 5 and 20 minutes long, consisting of variety of standardized clinical tasks. Examiners mark student’s performance using structured marking sheet. They have been used in the 3rd and in the final year of osteopathic programmes.
Newble (2004) states that: “traditional clinical examinations have serious limitations in terms of validity and reliability. Objective structured clinical examinations (OSCEs) have the capacity to improve the validity and reliability of assessments of many aspects of clinical competence. “
He continues by saying that OSCE provides a format particularly suitable for assessing many components of clinical competence, especially clinical, technical and practical skills, often with a high degree of fidelity. The testing of relevant knowledge, including aspects of diagnosis, investigation and management, can be more feasibly (and cheaply) tested with written formats.
To achieve high levels of reliability OSCEs have to be longer than is often practicable. OSCEs can be combined with other methods of assessment to enhance reliability. Easy to apply standard setting procedures are now available. A considerable body of evidence on OSCEs exists to guide decisions and further developments.”
Using simulated as well as standardized patients in OSCE examinations together with sufficient number of stations will increase the reliability and content validity of the OSCE.
Now, looking at the very top of the pyramid and the assessment of “does”, we have to agree that this is probably the biggest challenge of all. We are entering the world of “expert”, “professional”, “master” and so on, so how can you measure; assess a clinician’s fitness to practice.
Tynjala et al., (2006) below propose a learning model that includes reflection, practical experiences together with mentoring and coaching which suggests postgraduate experiences.
Tynjala states that there are at least three implications of this model for learning:
- “ the development of vocational and professional expertise must be seen as a holistic process in which theory cannot be separated from practice—or practice cannot be separated from theory.
- Second, when students are solving real life problems either in authentic working life or in simulated contexts, they need to be provided with conceptual and pedagogical tools which make it possible for them to integrate theoretical knowledge with their practical experiences.
- Participating in real life situations is a necessary but not a sufficient condition for the development of high level expertise. Only deep integration of theoretical, practical and self-regulative knowledge creates expertise.”
I would like to explore the final point that Tynjala makes and pose the question as to: how do we continue evaluating our professionals in the clinical practice in view of maintaining high standard in osteopathic profession as a whole. London (2008) proposes concept of the “gold standard” in education as well as in practices but admits that even though it is a valuable measure to which all other standards can be compared; need to satisfy criteria such as reliability, validity feasibility and fairness may not always be achievable.
Before we look at assessing competence in practice I would like to explore the transition of qualified graduates to independent practitioners.
Evans et al (2006) states that: “new graduates from medical school feel unprepared for their first post.” Despite some improvements to curriculum there are still high levels of anxiety as well as feeling of low preparedness than desired.
Berridge et al. (2007) explore the effect of supported transition in the new workplace. In this research newly qualified doctors undertook a two-week induction which combined life support, emergency and clinical skills training with administrative induction and shadowing the outgoing house officer. The programme was perceived to be helpful although it had some failings.
Prince et al (2005) state:” The most frequently mentioned stressful transition concerned that from pre-clinical science student to apprentice doctor on the ward. Causes of stress included changes in learning environment, teaching styles and expectations.”
It has been concluded that further studies are required into improving curriculum as well as providing further support to newly qualified doctors so they feel better prepared and easier assimilated into their new responsibilities especially in the multidisciplinary team.
Let’s have a look at how does the teaching in the osteopathic clinics equips student independent work ahead.
The classroom teaching of technical and clinical skills continues in teaching clinics. Many of them provide variety of disciplines – specializing in pediatrics, sports injury and rehabilitation. Clinic-based learning allows students with to practice skills such as patient case history taking; physical examinations, differential thinking, and patient management as well as skills of communication inter professional relationship under close supervision and guidance of clinic tutors. Learning in this environment gives students a chance to deal with real problems that they will come across in their practices. Some osteopathic colleges do assess student in the clinic as a part of the curriculum and run this as a module where students have a clinic portfolio that have their reflective log bridging the gap between classroom teaching and finally being able to integrate into clinical practice.
Parsell & Bligh (2001) are asking a big question: What is wrong with clinical teaching They report that:
“Learners see a limited range of patients in single clinics, case discussions are infrequent and short with little teaching, feedback is minimal, and learners appear to have less effective interviewing and physical examination skills.”
Wallace (2008) states that: “the effectiveness of clinical learning depends on the professional clinician acting in a supervisory role.”
Who is then responsible for learning in the clinicsAre clinic tutors still teachers or should their role should changeI feel that too often students expect to be told what to do in the similar way as in the classrooms. I have seen not only excitement in the new group of students entering the clinics for the first time but fear; where reality that next stage has arrived hits home. They no longer just observing colleagues but actually taking full responsibility for a patient. Occasionally clinic tutors will take over decision making or take over the treatment when they see students struggling. They do need to be very careful not take over to often but make sure that they are allowed to make their mistakes and tutors remain in the role of mentor and coach for the student until they are ready to deal with complex clinical situations on their own.
Competent clinical teachers require not only extensive clinical knowledge and have expertise about their patients but are familiar with methods in teaching and learning strategies. Their role is very complex and multifaceted and changes at different stages of student progress and development.
In Pursell & Blight (2001) adding to previous comments that survey also showed graduate medical students commented that the teaching was graded to be of the highest importance whereas supervisory role was ranked more highly with more experienced doctors.
As students gain more experience in the clinical setting they begin to discuss and review patients cases rather than follow recommendations of the tutors.
The debate is still ongoing within the medical education literature regarding different approaches by experienced clinicians to come up with diagnostic decisions; and teaching that to their learners.
Eva (2004) proposes that clinical teachers need to teach students multiple reasoning strategies and suggests that further studies are needed to show that analytic and non analytic models need to be thought with greater flexibility shown by experts in clinical teaching.
It also points at difficulties that arise from wanting to design a single model that students can use to arrive at working or final hypothesis from a patient case presentation. The emphasis is put in developing students to become competent at solving clinical problems and assessing individuals throughout their clinical learning. Arriving at accurate diagnosis does not imply that the student is competent. It is similar as to providing to accurate mathematical formulae but unless you can show rationale how you arrived at that point it may be meaningless. On the other hand if the student can explain the processes that lead to diagnosis does not automatically leads to correct diagnosis.
“Finally, the flexibility inherent in clinical reasoning and the prevalence of context specificity has very real implications for clinical teachers’ evaluation of trainees. One should not assume that because a student has provided an accurate diagnosis and ? or management plan, he or she fully understands the physiological mechanisms underlying the process. Similarly, even if the student can explain the underlying physiological mechanisms, one should not assume that he or she would provide an accurate diagnosis upon encountering the next case.”
We have tested and assessed our students, we have given the seal of approval, hopefully we made sure as much as it possible to make them competent professionals; but are they “fit to practice?” are they ready to practiceShould we provide a buffer and safeguard not necessarily for the public and other professionals but the new graduates. As mentioned before, various surveys express that medical students value programmes that support the transition from medical student to practicing clinicians.
There is another aspect I would like to consider. How do the students know weather they fit to practice I would like to look at four stages of learning proposed by 1940’s psychologist Abraham Maslow and together with Johari window that deals with development of self-awareness we may try to answer that question.
The individual neither understands nor knows how to do something, nor recognizes the deficit, nor has a desire to address it.
Though the individual does not understand or know how to do something, he or she does recognize the deficit, without yet addressing it.
The individual understands or knows how to do something. However, demonstrating the skill or knowledge requires a great deal of consciousness or concentration.
The individual has had so much practice with a skill that it becomes “second nature” and can be performed easily (often without concentrating too deeply). He or she may or may not be able to teach it to others, depending upon how and when it was learned.
Johari window, named after Joseph Luft and Harry Ingham has been first used in 1955 during group development session.
The four parts represent the following:
- The open area is that part of our conscious self – our attitudes, behaviour, motivation, values, way of life – of which we are aware and which is known to others. We move within this area with freedom. We are “open books”.
- Our hidden area cannot be known to others unless we disclose it. There is that which we freely keep within ourselves, and that which we retain out of fear. The degree to which we share ourselves with others (disclosure) is the degree to which we can be known.
- (Blind) There are things about ourselves which we do not know, but that others can see more clearly; or things we imagine to be true of ourselves for a variety of reasons but that others do not see at all. When others say what they see (feedback), in a supportive, responsible way, and we are able to hear it; in that way we are able to test the reality of who we are and are able to grow.
- (Unknown) We are more rich and complex than that which we and others know, but from time to time something happens – is felt, read, heard, dreamed – something from our unconscious is revealed. Then we “know” what we have never “known” before. (internet)
Will Taylor added 5th stage to the competence model: ‘Conscious competence of unconscious competence.’
Taylor states that: “We revisit conscious incompetence, making discoveries in the holes in our knowledge and skills, becoming discouraged, which fuels incentive to proceed (when it does not defeat). We perpetually learn, inviting ongoing tutelage, mentoring and self-study (ongoing conscious competence). We continually challenge our ‘unconscious competence’ in the face of complacency, areas of ignorance, unconscious errors, and the changing world and knowledge base:
We challenge our unconscious competence when we recognize that a return to unconscious incompetence would be inevitable. We do this in part by self-study and use of peer review – such that mature practice encompasses the entire ‘conscious competence’ model, rather than supercedes it as the hierarchical model might suggest.”
(Courtesy of Will Taylor, Chair, Department of Homeopathic Medicine, National College of Natural Medicine, Portland, Oregon, USA, March 2007. Please reference the diagram accordingly if you use it.) (internet access)
Finally, let’s look at three Greek words:
Episteme – “to know”; Techne – “craftsmanship”, “craft”, “art” and Phronesis – “practical wisdom” or “prudence”; as we, in osteopathic education, osteopathic research and practice are moving forward rather slowly in comparison to medical school across the world. We should take a very close look and start a process of reforms of our profession. Start to put pressure on our governing body to lead and take us into 21st century that all other medical professional are heading. Than the question posed at the beginning will more straightforward answer.
For my final conclusion I would like to quote proposal by Panel 9 published in The Lancet (2010):
“Instructional reforms should encompass the entire range from admission to graduation, to generate a diverse student body with a competency-based curriculum that, through the creative use of information technology (IT), prepares students for the realities of teamwork, to develop flexible career paths that are based on the spirit and duty of a new professionalism.
- Adoption of competency-based curricula that are responsive to rapidly changing needs rather than being dominated by static coursework. Competencies should be adapted to local contexts and be determined by national stakeholders, while harnessing global knowledge and experiences. Simultaneously, the present gaps should be filled in the range of competencies that are required to deal with 21st century challenges common to all countries—e.g., the response to global health security threats or the management of increasingly complex health systems.
- Promotion of interprofessional and transprofessional education that breaks down professional silos while enhancing collaborative and non-hierarchical relationships in effective teams. Alongside specific technical skills, interprofessional education should focus on cross-cutting generic competencies, such as analytical abilities (for effective use of both evidence and ethical deliberation in decision making), leadership and management capabilities (for efficient handling of scarce resources in conditions of uncertainty), and communication skills (for mobilization of all stakeholders, including patients and populations).
- Exploitation of the power of IT for learning through development of evidence, capacity for data collection and analysis, simulation and testing, distance learning, collaborative connectivity, and management of the increase in knowledge. Universities and similar institutions have to make the necessary adjustments to harness the new forms of transformative learning made possible by the IT revolution, moving beyond the traditional task of transmitting information to the more challenging role of developing the competencies to access, discriminate, analyze, and use knowledge. More than ever, these institutions have the duty of teaching students how to think creatively to master large flows of information in the search for solutions.
- Adaptation locally but harnessing of resources globally in a way that confers capacity to flexibly address local challenges while using global knowledge, experience, and shared resources, including faculty, curriculum, didactic materials, and students linked internationally through exchange programmes.
- Strengthening of educational resources, since faculty, syllabuses, didactic materials, and infrastructure are necessary instruments to achieve competencies. Many countries have severe deficits that require mobilising resources, both financial and didactic, including open access to journals and teaching materials. Faculty development needs special attention through increased investments in education of educators, stable and rewarding career paths, and constructive assessment linked to incentives for good performance.
- Promote a new professionalism that uses competencies as the objective criterion for the classification of health professionals, transforming present conventional silos.
A set of common attitudes, values, and behaviours should be developed as the foundation for preparation of a new generation of professionals to complement their learning of specialties of expertise with their roles as accountable change agents, competent managers of resources, and promoters of evidence-based policies.
Institutional reforms should align national efforts through joint planning especially in the education and health sectors, engage all stakeholders in the reform process, extend academic learning sites into communities, develop global collaborative networks for mutual strengthening, and lead in promotion of the culture of critical inquiry and public reasoning.
7. Establishment of joint planning mechanisms in every country to engage key
Stakeholders, especially ministries of education and health, professional
Associations, and the academic community, to overcome fragmentation by assessment of national conditions, setting priorities, shaping policies, tracking change, and harmonizing the supply of and demand for health professionals to meet the health needs of the population. In this planning process, special attention should be paid to sex and geography. As the proportion of women in the health workforce increases, equal opportunities need to be in place—e.g., through more flexible working arrangements, career paths that accommodate temporary breaks, support to other social roles of women such as child care, and an active stance against any form of sex discrimination or subordination. With respect to geographical distribution, emphasis should be placed on recruitment of students from marginalized areas, offering financial and career incentives to providers serving these areas, and deploying the power of IT to ease professional isolation.
8. Expansion from academic centers to academic systems, extending the traditional discovery-care-education continuum in schools and hospitals into primary care settings and communities, strengthened through external collaboration as part of more responsive and dynamic professional education systems.
9. Linking together through networks, alliances, and consortia between educational institutions worldwide and across to allied actors, such as governments, civil society organizations, business, and media. In view of faculty shortages and other resource constraints, every developing country is unlikely to be able to train on its own the full complement of health professionals that is required. Therefore, regional and global consortia need to be established as a part of institutional design in the 21st century, taking advantage of information and communication technologies. The aim is to overcome the constraints of individual institutions and expand resources in knowledge, information, and solidarity for shared missions. These relations should be based on principles of non-exploitative and non-paternalistic equitable sharing of resources to generate mutual benefit and accountability.
10. Nurturing of a culture of critical inquiry as a central function of universities and other institutions of higher learning, which is crucial to mobilise scientific knowledge, ethical deliberation, and public reasoning and debate to generate enlightened social transformation.”
- About Osteopathy/General Osteopathic Council. http://www.osteopathy.org.uk/information/ (Accessed 10 March 2011).
- Messick, S. (1989). Validity. In R. Linn (Ed.), Educational measurement, (3rd ed.). Washington DC: Oryx Press
Shavelson, R. J., & Webb, N. M. (1991).
- A primer on generalisability theory. Thousand Oaks: Sage PublicationsMiller GE. The assessment of clinical skills/competence/performance. Acad Med 1990; 65: 563-67.
- Lambert W T Schuwirth & Cees P M van der Vleuten Different written assessment methods: what can be said about their strengths and weaknesses Medical Education 2004; 38: 974-9
- Brady AM Assessment of learning with multiple-choice questions. Nurse Education in Practice (2005) 5, 238–242
Quinn, F.M., 2000. The Principles and Practice of Nurse Education, fourthed. Stanley Thorne (Publishers) Ltd, Cheltenham.
- Newble, D.I. (2002) Assessing Clinical Competence at the Undergraduate Level, Medical Education Booklet, No. 25 (Edinburgh, Association for the Study of Medical Education).
- NEWBLE, D.I. (2004) Techniques for measuring clinical competence:objective structured clinical examinations, Medical Education, 38, pp. 199- 203
- Ponnamperuma G. G., Karunathilake I. M., McAleer S., Davis M. H.,. The long case and its modifications: a literature review. Medical Education 2009: 43: 936–941
- Fletcher, P., Clinical competence examination – Improvement of validity and reliability. International Journal of Osteopathic Medicine 11 (2008) 137 – 141
- Tynjala et al., 2006 P. Tynjala, V. Slotte, J. Nieminen, K. Lonka and E. Olkinuora, From university to working life: Graduates’ workplace skills in practice. In: P. Tynjala, J. Valimaa and G. Boulton-Lewis, Editors, Higher education and working life: Collaborations, confrontations and challenges, Elsevier, Amsterdam (2006), pp. 73–88.
- Tynjala P. Perspective into learning at the workplace. Elsevier 2007 (p145).
- Berridge EJ, Freeth D, Sharpe J, Roberts C M. 2007. Bridging the gap: supporting the transition from medical student to practicing doctor – a two – week preparation programme after graduation. Medical Teacher 29:119-127.
- Evans DE, Roberts CM. 2006. Preparation for practice: How can medical schools better prepare PRHOs Medical Teacher 28:549-552.
- Glennys Parsell & John Bligh. 2001 Recent perspectives on clinical teaching
- Medical Education 2001;35:409±414
- Wallace SS 2008. Critically, research, scholarship and teaching: Osteopaths as educators – what makes a good teacherInternational Journal of Osteopathic Medicine 11, 52-55.
- Eva KW 2004.What every teacher needs to know about clinical Reasoning. Medical Education 2004; 39: 98–106
- Phil Race Using feedback to help students to learn. The Higher Education Academy. www.york.ac.uk/…/learningandteaching/id432_using_feedback.pdf (accessed 6th of March 2011 18.47)
- London S. , 2008. The assessment of clinical practice in osteopathic education: I there a need to define a gold standard International Journal of Osteopathic Medicine 11; 132-136.
- Katinka J A H Prince,1 Henny P A Boshuizen,2 Cees P M van der Vleuten3 & Albert J J A Scherpbier. Students’ opinions about their preparation for clinical practice. Medical Education 2005; 39: 704–712
- Conscious competence learning. http://www.businessballs.com/consciouscompetencelearningmodel.html (Accessed 10 of March 2011)
- Four stages of competency model of competency. http://en.wikipedia.org/wiki/Four_stages_of_competence (Accessed 6 of March 2011)
- The Johari window. http://www.gp-training.net/training/leadership/johari.htm (Accessed 13 of March 2011)
- Julio Frenk*, Lincoln Chen*, Zulfi qar A Bhutta, Jordan Cohen, Nigel Crisp, Timothy Evans, Harvey Fineberg, Patricia Garcia, Yang Ke, Patrick Kelley, Barry Kistnasamy, Afaf Meleis, David Naylor, Ariel Pablos-Mendez,
- Srinath Reddy, Susan Scrimshaw, Jaime Sepulveda, David Serwadda,Huda Zurayk. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. The Lancet. Volume 376, Issue 9756, 4 December 2010-10 December 2010, Pages 1923-1958