Student-athlete annual evaluations and exit interviews should never be the sole determinant or even a significant determinant of employee termination and compensation decisions. Research on student-teacher evaluations (SET) clearly reveals the lack of reliability of such instruments:
"SET primarily do not measure teaching effectiveness, that they are strongly and non-uniformly biased by factors including the genders of the instructor and student, that they disadvantage female instructors, and that it is impossible to adjust for these biases. SET should not be relied upon as a measure of teaching effectiveness. Relying on SET for personnel decisions has disparate impact by gender, in general."1
Research clearly supported the existence of gender bias of students who regularly rate female teachers lower than males. In fact, Boring, Ottoboni, and Stark caution, “class action lawsuits against universities that rely on these evaluations for employment decisions will start this year, and that there’s evidence to support such cases.” They further state that the “onus should be on universities that rely on SET for employment decisions to provide convincing affirmative evidence that such reliance does not have disparate impact on women, underrepresented minorities, or other protected groups.”
Further, the literature regarding the use of SET in athletics settings (Levoi, Crossett, et al) is clear. When female coaches “coach” like males (i.e., act like males) they are sanctioned more harshly because they are not conforming to traditional gender norms. Lavoi, in her most recent research study to be published in 2017, reveals that the strongest female coaches are rated lowest by female athletes compared to male counterparts who coach in identical ways. This implicit gender bias of female athletes occurs when they perceive, evaluate and interpret strong female coach behavior and results in discrimination against female coaches when SETs are misused for compensation and employment non-renewal purposes. Lavoi also notes that this implicit and conscious gender bias is present in athletic directors who evaluate and interpret SETs for female coaches, most of whom are men.
This elevated susceptibility to gender bias in athletics settings takes on increased emphasis because in many athletic departments there are no women employed as coaches of men’s teams and therefore no men evaluating female coaches, which would be required to demonstrate lack of gender bias in the evaluation of coaches. In addition to this inherent problem with gender bias, there are other considerations that affect reliability in athletic settings such as whether a player starts or participates in a game frequently or infrequently or whether the team as a whole is experiencing success. Last, we must always remember that student-athletes lack expertise and training in the evaluation of pedagogy practices which is the responsibility of the trained supervisor of the employee.
Thus, student-athlete evaluations should be a small part of the larger picture of a comprehensive coach’s annual performance evaluation. Each coach’s sport program should be evaluated based on sport success goals (won-lost records, rank nationally or within the conference, etc.), the academic performance of players, player improvement from a skill and human development perspective, quality of pedagogy (from supervisor observation of games and practices), operational efficiency (staying within budget, meeting deadlines, etc.), and recruiting success. In higher competitive divisions, sport program success on the playing field is more heavily weighted with regard to importance but not to the exclusion of any other element. Ideally, all institutions want coaches who win, coaches who follow NCAA rules, student-athletes who graduate and experience high levels of academic success, individual athlete improvement in skill development, emotional maturity and resilience in highly competitive environments and exemplary and sportsmanlike conduct that reflects well on the reputation of the institution on the part of both coaches and student-athletes
Student Athlete Exit Interviews. This caution regarding student evaluations being said, there is a proper place for use of student-athlete exit interviews and annual end of season student-athlete evaluations. Student-athlete exit interviews should occur at the end of an athlete’s four years of athletic eligibility, upon graduation, prior to transfer to another institution or departure from the team if prior to the completion of eligibility, whichever occurs first and should include all department services (i.e., academic support, coaching, athletic training, etc.). Model practice (see Lopiano, D. and Zotos, C., 2013, Athletic Director’s Desk Reference. Human Kinetics. Sample policy 8.15) should encompass the following processes:
- “General. The athletic administrator with student development responsibilities should make every effort to have every student-athlete who leaves the athletic program complete an exit interview. Each student-athlete should be given the choice between completing an oral interview with a member of the intercollegiate athletics advisory council or submitting a written questionnaire, which may be signed or unsigned. The results of such interviews should remain confidential with regard to the identity of the interviewee.
- Oral Interview. The oral interview should be conducted by a faculty member of the intercollegiate athletics advisory council. Every effort shall be made to have the faculty member be of the same gender and race as the student-athlete to maximize the comfort level of the interview. The interview shall include all the questions on the written questionnaire, which should be used to record the student-athlete’s comments. The completed form should be submitted to the faculty athletic representative.
- Written Interview. If a student-athlete declines an oral interview, the athletic administrator with student development responsibilities should give the exiting student-athlete a written exit interview questionnaire and a stamped return envelope for its return. The envelope should be addressed to the faculty athletic representative.
- Exit Interview Summary. The faculty athletic representative should prepare a summary of the results of all exit interviews, which shall be reviewed annually by the intercollegiate athletic council and athletic department senior staff.”
This end of collegiate athletic experience snapshot is intended to provide the faculty athletic council, an advisory body to the president, and athletics department senior staff with a mature student-athlete perspective on their academic and sport experience. The exit interview is not intended to be a coach evaluation instrument or mechanism to detect coaching misconduct.
Student-Athlete Annual Evaluation. The annual student-athlete evaluation, like the annual employee performance evaluation, has as its primary purpose, improvement of coaching instruction. It should be rigidly constructed to provide forced choices regarding the frequency of occurrence of model instructional practices (i.e., always, almost always, sometimes, seldom or similar objective choice system) and each question should include the opportunity to comment in a constructive way, usually asking the athlete to offer suggestions for improvement. It should not contain “open ended” questions which would be an invitation to complain. This is because student-athlete complaints regarding coach conduct should be raised immediately and corrected immediately via a separate student complaint process. They do not belong in an end-of-year assessment.
The annual evaluation instrument should be administered by the coach’s supervising senior staff member who should explain that the purpose of the evaluation is to assess the athlete’s experience under the direction of a member of the coaching staff from whom the athlete has received instruction in sports skills and strategy and that the purpose is to improve the athlete’s experience. The supervising staff member should explain that the form will be destroyed after the data have been transferred to the aggregate summary to protect confidentiality with regard to the identity of athlete. The evaluation should not be signed by the athlete. All student-athlete assessments should be aggregated into a summary report that is discussed as part of each coach’s annual performance evaluation. Any concerns expressed by students should ignite a collaborative coach/supervisor conversation about strategies to improve the student-athlete experience.
There are a number of important “DON’Ts” which should be kept in mind.
- Student-athletes should not be queried on subjects for which they are not trained to evaluate such as athletic training taping and bracing techniques, sensitivity of training room staff to the need to refer complicated problems to medical specialists, quality of special game promotions, etc.
- Asking student-athletes to offer reasons why they think teammates have left the program is highly improper and invites opinion, hearsay and rumor.
- The extensive use of open-ended comments invites non-specific and overly broad opinions. (i.e. “list the major weaknesses of athletics student services.”) Comments should be solicited in a manner that defines a specific service and asks for the athlete’s opinion about that service, invites constructive solutions and asks for improvements rather than complaints.
- The inclusion of “Did you feel comfortable discussing personal problems with the head coach?” is a highly improper query. Personal problems should be referred to a trained professional counselor. The coach has no duty or training to offer advice in this regard and has a build-in conflict of interest that should preclude the offering of such advice.
- Care must be taken to sue unbiased question construction. It is improper to invite responses of strongly disagree, disagree, neutral, agree and strongly agree and then include these same word modifiers in the statement requiring one of these assigned values (i.e., Players felt a strong sense of respect for the head coach.)
- Questions about the experiences or needs of “all” student-athletes are improper (i.e. “Student athletes could benefit from more personal counseling services.” or “The Athletics Department provides adequate professional help for students with personal issues.”). Further, the athletic department should not be providing such services because of a possible conflict of interest. The student should be directed to the Student Counseling Center or Student Health Center. The student-athlete should only be asked about his or her experience.
- Student-athletes should not be asked questions about a sport program budget of which they have no knowledge.
- Coach and athlete code of conduct or coach misconduct queries do not belong in end-of-season or end of career student-athlete evaluations. The instrument should never specifically seek this information (i.e., Were you subjected to physical abuse, sexual abuse, verbal abuse, mental abuse?) Incredibly, such questions are often asked without providing a definition of terms. Thus, YES/NO or other responses to such questions should not be considered valid or reliable.
1 Boring, A., Ottoboni, K. and Stark, P.B. (2016) Student evaluations of teaching (mostly) do not measure teaching effectiveness. ScienceOpen Research 2016, DOI: 10.14293/S2199-1006.1.SOR-EDU.AETBZC.v1