Sports

Coach Evaluation and the Team Model

By: Sean Douglas • 4 years ago •

“HALF the season gone, half the coaches gone; forget the Tasmanian devil – the A-League coach is Australia’s most endangered species”

That was the headline in a major Australian newspaper half way through the 2013-14 season. Available statistics from both the A-League and the English leagues (kindly provided by the League Managers Association) make for grim reading if you are a football coach. There have been 46 permanent A-League coaching appointments (excluding interim stints), with an average lifespan in the job of just 44 games. So forget all those proclamations of five-year plans and long-term philosophies. If you’re appointed head coach of an A-League team, your job security is roughly a season and a half.

The statistics for the Football Leagues in England do not make much better reading, so your life expectancy as a coach in either the A-League or the Football Leagues is a roughly one and a half seasons. With the professionalization of the game comes the notion of accountability and evaluation of coaching work.

But how are professional coaches evaluated?

“If I was advising the people at United who appointed Moyes, I would have said 12 months ago: ‘Are we sure we have the right person? Do we really understand why we are appointing this guy? What is the realistic/optimistic/pessimistic assessment of how this is going to go and do we have a plan for each of those situations? Please tell me that, as a board of a multi-billion dollar company, you have thought about all these scenarios and have a plan for each one.”

– Michael Finnigan : Performance Psychologist

Typically we default to win loss records because it is easier. The following questions may provide a rationale for looking further than win-loss statistics.

  • What contribution does a coach make to the performance of a team?
  • How do we know if a team would have performed better under a different coach?

Unfortunately, research into how teams perform after a change of manager shows no difference between teams that experienced an early-season slump and changed their manager within-season and the win ratios of a control group that experienced a similar slump but kept their manager. The recovery pattern following the initial slump was very similar in both groups. In fact, some studies show that teams who changed their manager after a slump actually took longer to recover than teams who didn’t.

Although winning is dependent largely upon the talent, skills, knowledge and performance of players, coaches are currently evaluated primarily on this singular outcome measure which is neither completely controllable by coaches nor reflective of their ability to develop players.

In their study of coaches in the NBA, Berri, Leeds and Mondello [5] find that coaches make a difference to team performance but that the typical performance standard – wins and losses – can be misleading and is not a good measure by itself.

What is Coaching Expertise/Effectiveness?

How do you evaluate the performance of the coach given that each coach is working with different squads, different support staff, different budgets and even different facilities?

Côté and Gilbert (2009) proposed an integrated definition of coaching effectiveness and expertise that focuses on the integration of coaches’ knowledge, athletes’ outcomes, and the different contexts in which coaches typically work

“The consistent application of integrated professional, interpersonal, and intrapersonal knowledge to improve athletes’ competence, confidence, connection, and character in specific coaching contexts”

(Côté and Gilbert 2009: 316)

This definition of coaching effectiveness is comprised of three components:

  1. Coach Knowledge
  2. Athlete Outcomes
  3. Coaching Contexts

The literature outlines four coaching contexts that are defined by two factors:

  • Competitive Level – Participation or Performance
  • Major life periods – Childhood, Adolescence, Adulthood

This results in the following contexts:

  • Participation coaches for children (sampling years 6-12)
  • Participation coaches for adolescents and adults (recreational years 13+)
  • Performance coaches for young adolescents (specializing years 13-15), and
  • Performance coaches for older adolescents and adults (investment years 16+).

It is this last category that we are interested in, when considering the other two components of coaching effectiveness.

  1. Coach Knowledge

Research (Jones, 2007; Becker, 2009; Jowett 2007; Gilbert and Trudel, 2001) has identified three forms of coaches’ knowledge that underpin coaching effectiveness and expertise. These are:

  • Professional Knowledge

Sport-specific knowledge including understanding of the sports sciences (psychology, physiology, biomechanics, etc), sport specific demands and techniques, and the pedagogical knowledge used to teach sport skills

  • Interpersonal Knowledge

Understanding that coaching is essentially about the relationships developed with players/athletes and that these relationships are based primarily on social interactions (coach to player, player to coach, and coach to other stakeholders involved in supporting performance. This may be best summarized as a combination of emotional intelligence and transformational leadership

  • Intrapersonal Knowledge

This is most aligned with the concepts of self-awareness and reflection. Effective coaches have a keen sense of self-awareness, are aware of their strengths and limitations and are prepared to act on insights gained (continual learning/development). The development of this awareness occurs through constant reflection about current and past actions.

  1. Athlete Outcomes

As mentioned earlier, the role of the coach is central to performance. To improve performance, the coach needs to apply his/her knowledge (professional, interpersonal & intrapersonal) to help the players/team perform to the best of their ability. The 4 C’s is a concise yet comprehensive framework to measure performance (competence) and the psycho-social outcomes (confidence, connection, and character) required for elite team sport.

Competence

Competence can be broken down into general dimensions – such as the teaching of Technical, Tactical, Mental, Physical skills – in such a way that they can be reproduced in competitive situations where decision making and positive responses to pressure are of the essence.

Confidence

Performance coaches should try to instill the belief and confidence in their players/athletes that they possess the capability to be successful and compete in the sport they practice.

Connection

Performance coaches must also develop their players’ connection with others because elite sport is a social venue that requires interactions with a broad range of individuals for optimal performance. Performance Coaches need to foster a climate in which their players engage in meaningful and positive relationships with their team-mates, staff and others involved.

Character

It is important for coaches to promote the development of character so that athletes make appropriate and ethical decisions regarding their training and their overall involvement in sport.

From this framework we can see that the ultimate role of a Performance coach is to develop their players competence, confidence, connection and character so that they can compete at their highest level of performance on a consistent basis.

In sum, a coach’s ability to maximize athletes’ outcomes rests not only on being able to consistently apply extensive professional knowledge and interpersonal knowledge, but also on constant introspection, review, and revision of one’s practice.

The current emphasis on only professional knowledge appears to guide both traditional coach education and coach recruitment around the world. For example, it is extremely rare for a professional sport team to hire a coach who is not a former elite athlete. The assumption here is that the primary requirement to become an effective coach is an extensive knowledge of the sport (professional knowledge). Based on experience, unfortunately this same trend is pervasive across all levels of sport. Seldom is consideration given to how well an individual connects with others (interpersonal knowledge) or their openness to continued learning and self-reflection (intrapersonal knowledge). This myopic view of the requisite knowledge for coaching effectiveness may in part explain the high turnover rates of professional coaches, where research has shown that previous athletic ability is not correlated with coaching success [54].

You only have to look at the numerous real-world examples of coaches who “lost the dressing room” to realize that Interpersonal knowledge is crucial to being an effective coach.  A coach can only be effective over time if there is a two-way relationship built on trust and respect. If this relationship breaks down (for example, the Matilda’s), then performance will drop.

How do we measure coach expertise?

Effective coaching is the achievement of goals that are shared by all stakeholders, and which are bounded by time and place (i.e. the context the coach is working in, for example, the quality of players or available resources). Therefore Lyle (2002) proposes that:

  • Coaching effectiveness should be judged by evaluating instances of specific coaching performance;
  • The effective coach is one whose capacity for coaching effectiveness has been evaluated over time and circumstance; and
  • The effective coach will acquire and display expertise and may in time be termed an expert and in appropriate circumstances, may be successful (in terms of win-loss).

What does the research say about evaluating high performance coaches?

There have been several attempts by researchers to produce a tool to measure coach effectiveness. Most of them have looked at specific contexts or individual settings (e.g. training only). Gilbert and Trudel (1999, 2004) identified this and produced an instrument to comprehensively evaluate a coach’s work.

The Coaching Behaviour Scale for Sport (CBS-S) is a tool that offers seven dimensions of coaching behaviours that have been identified both by coaches and players as being important aspects of high performance coaching. These are preferable coach behaviours (that are underpinned by the three forms of knowledge outlined earlier) to improve player’s competence, confidence, connection and character.

The 7 dimensions of coaching behaviours that have been identified both by coaches and athletes as being important aspects of HP coaching: (Players rate their coaches on each of the items on a 7 point likert scale)

  1. Physical Training and Planning (items about the coach’s involvement in the athletes physical training and planning for training and competition)
  2. Goal setting (items assessing the coach’s involvement in the identification, development, and monitoring of the athlete’s goals)
  3. Mental Preparation (MP; five items focusing on how the coach helps the athlete to perform under pressure, stay focused, and be confident)
  4. Technical Skills (TS; eight items about coaching feedback, demonstration, and cues)
  5. Personal Rapport (PR; six items assessing the approachability, availability, and understanding of the coach)
  6. Negative Personal Rapport (NPR; eight items examining the coach’s use of negative techniques such as fear and yelling)
  7. Competition Strategies (CS; seven items focusing on the coach’s interaction with the athlete in competition).

The CBS-S is designed to evaluate the coaching that athletes received from one coach or a group of coaches. It is psychometrically sound because it has been grounded in coaches and athletes experiences.

Some coaches may be concerned that player feedback is involved in their evaluation. But given that the coach’s role is to get the best out of the players, the players need to feel confident and trust in the coach. The perceptions of the players in evaluating the quality of the work performed by high performance coaches are critical to the understanding of the quality of the player-coach relationship. If the relationship is poor and the players do not trust the coach, or do not feel that the coach can help them or the team improve then individual and team performance will not improve.

Coaches should be evaluated on how well they develop the 4 C’s and their Team Model, taking into account the resources available (financial, facilities, staff, players), to achieve the result on match day in a manner that entertains the fans.

Evaluating high performance coaches with a tool like the CBS-S involves three steps:

  1. The CBS-S is administered to players at one or two points in the season – around mid season and towards the end. Data collected mid-season provides useful feedback to the coach, which gives the coach an opportunity to address issues in the second half of the season. Ideally an independent assessor meets with the players and gives direction as to the purpose and aims of the evaluation. If an independent assessor is not available, a player can act as representative to administer the test, gather the forms and place them in envelopes to ensure anonymity. No coaching staff should be present.
  2. Data analysis and report generation from an independent team of assessors. This can be done electronically and sent back to the Football Director/administrator.
  3. The report is reviewed by Football Director/relevant staff and appropriate feedback provided to coaching staff. This can form the direction for professional development.

It should be noted that any single assessment of coaches work is problematic, so this should be taken into account with this instrument. Here are some guidelines when interpreting players’ evaluations of their coaches’ work through questionnaires:

  1. Complement results from athlete evaluations with data from other sources such as objective indicators of performance (e.g win-loss records, annual progression over the past 3 years)
  2. Use more than one set of evaluation results before making decisions about competencies.
  3. Ensure that sufficient numbers of players have responded to the evaluation of a coach – the absolute number and the proportion of athletes responding are both important.
  4. Consider the contextual factors that can affect the sport program as well as the athletes and coaches characteristics
  5. Remain open to situational explanations when using specific results to make decisions about coaching competencies (e.g low scores on mental preparation but coach has limited or no access to a sport psychologist)

The data provided by the CBS-S presents a comprehensive profile of coach behaviours, which can be useful for consultants or administrators in reviewing coaches’ performances and competencies. Progressive coaches may also use these data as a means of getting feedback on their own coaching practice and identifying areas where improvement is required.

Although the primary focus of CBS-S is to provide criteria other than “winning” and “losing” to evaluate the work of professional football coaches, it can also be used as an intervention tool. The data provided by the CBS-S presents a comprehensive profile of coach behaviors which can be useful for consultants or administrators in reviewing coaches’ performances and competencies. Progressive coaches can also use this data as a means of getting feedback on their own coaching practice and identifying areas where improvement is required.

Summary and Recommendations

The following principles should guide any coaching evaluation process:

  • Coach evaluation should be based on the effectiveness of the coaching
  • Any evaluation is understood within the operating context (e.g. support, time, resources etc)
  • Effectiveness is measured against both outcomes (win/loss) and processes (coach behaviours)

A professional coach cannot control the outcome of the match. He can only control how he prepares his team. Focusing on developing a player’s competence, confidence, connection and character (4 C’s) within a clearly articulated playing style and tactical framework (Team Model) will give the team the best opportunity to win the match.

A club-centered philosophy has a number of advantages over a coach-centered one.

  • The club knows the type of coach they are looking to appoint/retain with coaches.
  • Player purchases should that fit the club model, so a change of coach is unlikely to require a clean-out of players. This means finances are used to strengthen the squad instead of replace existing players.
  • A clear development model for younger players can be put in place
  • Fans generally become/remain fans because of the philosophy.

It is important for a coach to implement and execute a Team Model that fits the club. This ‘philosophical fit’ between coach and club should be the main reason why the coach is contracted.

Sean Douglas

National Head of Coach Education at Football Federation Australia

Powerful Stories, Tips and Amazing Insight

  • First Name
  • Last Name
  • Phone
  • Email

Ready, Set, Go! Everything you need to know to start coaching from the legends in the field. As well as the Business and Life coaches, our launch edition features David Parkin (AFL), Lisa Alexander (Netball Australia), Adam Commens (Hockey Australia), Simon Cusack (Swimming Australia), Sean Douglas (FFA) and many more!

x
4
Articles Remaining