Chapter 5 – Measuring Performance

Learning Objectives

5.1    Adopt a results approach to measuring performance including the development of accountabilities, objectives, and standards.

5.2    Determine accountabilities and their relative importance.

5.3    Identify objectives that are specific and clear, challenging, agreed upon, significant, prioritized, bound by time, achievable, fully communicated, flexible, and limited in number.

5.4    Identify performance standards that are related to the position, concrete, specific, measurable, practical to measure, meaningful, realistic and achievable, and reviewed regularly.

5.5    Adopt a behavior approach to measuring performance including the identification and  assessment of competencies.

5.6    Develop competencies that are defined clearly, provide a description of specific behavioral indicators that can be observed when someone demonstrates a competency effectively, provide a description of specific behaviors that are likely to occur when someone doesn’t demonstrate a competency effectively (what a competency is not), and include suggestions for developing them further.

5.7    Develop comparative performance measurement systems such as simple rank order, alternation rank order, paired comparisons, and forced distribution--being aware of the relative advantages and disadvantages of each.

5.8    Develop absolute performance measurement systems such as essays, behavior checklists, critical incidents, and graphic rating scales, and understand their advantages and disadvantages.

Chapter Outline

Measuring Results and Behaviors

Overview
1. Measuring Results
2. Measuring Behaviors

1. Measuring Results
• Key Questions
o Where should each individual focus efforts? (Accountabilities)
o What are the expected objectives? (Objectives)
o How do we know how well the results were achieved? (Performance Standards)

• Key Accountabilities
Broad areas of a job for which employee is responsible for producing results

• Objectives
Statements of important and measurable outcomes

• Performance Standards
Yardstick used to evaluate how well employees have achieved objectives

 Why do organizations with MBO philosophies implement PM systems that measure results?

• Determining Accountabilities
o Collect information about job (Job Description)
o Determine importance of task or cluster of tasks
 Percent of employee’s time spent performing task
 Impact on unit’s mission if performed inadequately
 Consequences of error

 How might one determine accountabilities for the Training Specialist/Consultant-Leadership & Training Development for Target Corporation? 

• Determining Objectives
o Purpose: to identify
 Outcomes
• Limited number
• Highly important
 When achieved
• Dramatic impact on overall organization success

o Characteristics of Good Objectives
 Specific and clear
 Challenging
 Agreed upon
 Significant
 Prioritized
 Bound by time
 Achievable
 Fully communicated
 Flexible
 Limited in number

 What kinds of objectives could be developed for the position at Target where we identified accountabilities earlier?
 How do the example objectives in the book comply with these characteristics?

• Determining Performance Standards
Standards refer to aspects of performance objectives, such as:
o Quality
 How well the objective is achieved
o Quantity
 How much, how many, how often, at what cost?
o Time
 Due dates, schedule, cycle times, how quickly

Standards must include:
o A verb
o The desired result
o A due date
o Some type of indicator
 Quality and/or
 Quantity

o Characteristics of Good Performance Standards
 Related to position
 Concrete, specific, measurable
 Practical to measure
 Meaningful
 Realistic and achievable
 Reviewed regularly

 Develop some performance standards for the position at Target, using accountabilities and objectives developed earlier.
 How do the example standards in the book comply with these characteristics?

2. Measuring Behaviors

Overview
• Identify Competencies
• Identify Indicators
• Choose Measurement System

Identify Competencies
 Measurable clusters of KSAs (critical in determining how results will be achieved)
• Knowledge
• Skills
• Abilities
 
Types of Competencies
• Differentiating
o Distinguish between superior and average performance
• Threshold
o Needed to perform to minimum standard

 Why would “change management” be a threshold competency for an IT Project manager position? 
 How is “process management” a differentiating competency for this position?

Identify Indicators
• Indicators are observable behaviors
• Indicators are used to measure the extent to which competencies are present or not

 What are some indicators that a supervisor is demonstrating consideration?  Initiating structure?

Describing Competencies:  Necessary components
• Definition
• Description of specific behaviors
o When competency demonstrated
o When competency not demonstrated
• Suggestions for developing the competency

 How might a supervisor develop the competency of consideration?

Choose Measurement System
• Comparative system - compares employees with each other
• Absolute system - compares employees with prespecified performance standard

• Comparative systems
o Simple rank order
o Alternation rank order
o Paired comparisons
o Forced distribution

 Why did CEO Jack Welch label GE’s forced distribution system the “vitality curve”?  How did it work?

• Advantages of Comparative Systems
o Easy to explain
o Straightforward
o Better control for biases and errors found in absolute systems
- Leniency
- Severity
- Central tendency
• Disadvantages of Comparative Systems
o Rankings may not be specific enough for
- Useful feedback
- Protection from legal challenge
o No information on relative distance between employees
o Specific issues with forced distribution method

 USC Professor Ed Lawler compares the forced distribution method to a salamander.  Is this appropriate? 

• Absolute systems
o Essays
o Behavior checklists
o Critical incidents
o Graphic rating scales

Essays
• Advantage:
 Potential to provide detailed feedback
• Disadvantages:
 Unstructured and may lack detail
 Depends on supervisor writing skill
 Lack of quantitative information; difficult to use in personnel decisions

Behavior checklists
• Advantage:
o Easy to use and understand
• Disadvantages:
o Scale points used are often arbitrary
o Difficult to get detailed and useful feedback
 Several examples of language for behavior checklists are in the text - which seemed effective?  Why?  Why not?
Critical incidents
o Two kinds of measurement
o Report of specific employee behavior
- Allows focus on specific behavior
- Very time consuming
• Examples of behavior illustrative of core competencies – manager circles best description
 Easier to use
 Describes behavior desired

 How could the Tom Jones report be changed from the first kind of measurement to the second?
 Which of these kinds of measurements is shown in the City of Irving example?

Graphic rating scales
• Clear meaning for each response category
• Consistent interpretation by outside readers
• Supervisor and employee should have same understanding of rating

Graphic rating scales: BARS improvement
Behaviorally Anchored Rating Scales (BARS)
o Uses critical incidents as anchors
o Involves multiple groups of employees in development
 Identify important job elements
 Describe critical incidents at various levels of performance
 Check for inter-rater reliability

Measuring Performance (Wrap up)
• Several types of methods
• Differ in terms of:
• Practicality (time and effort)
• Usefulness (quantifiable)

Measuring Results and Behaviors:  Summary
• Measuring Results
o Identify accountabilities
o Set objectives
o Determine standards of performance
• Measuring Behaviors
o Identify competencies
o Identify indicators
o Choose measurement system

Worked Solutions for End-of-Chapter Cases

Case Study 5.1 Accountabilities, Objectives, and Standards

Below is a job description for a sourcing and procurement internship position for Disney Consumer Products Studio.  Based on the information in the job description, create accountabilities, objectives, and standards for the job described.
Title:  Graduate Associate, Sourcing & Procurement
Accountabilities:
• Provide analytical support for sourcing projects impacting business units. Successful analytical support will ensure that project decisions are made on well-researched reasons.  Poorly made project decisions may delay client deliverables or damage the Disney brand.  (20% of time)
• Benchmark current pricing models and develop new approaches to pricing/buying various products and services that yield creative and business advantages.  This is extremely important to the continued sustainability of Disney, and will be critical to capture new portions of the market share.  Incorrectly identifying pricing models would be crushing to the Disney business and would directly impact the bottom line. (30% of time)
• Assist in the development of spend profiles, key stakeholder lists, savings opportunities where existing contracts are leveraged, savings opportunities in commodity areas that have not been sourced, and sourcing and procurement strategy.  Will play a small role in these extremely important activities.  Successfully developing spend profiles, identifying key stakeholders, identifying savings opportunities, and developing a successful business strategy are keys to the financial stability of Disney.  (50% of time)

Objectives:
• Develop spend profiles, key stakeholder lists, and a sourcing and procurement strategy. These tasks need to be completed by July 1, 2004.  Identifying savings opportunities where existing contracts are leveraged and identifying savings opportunities in commodity areas that have not been sourced are ongoing tasks.  Report weekly on savings identified.
• Benchmark current pricing models and develop new approaches to pricing/buying various products and services that yield creative and business advantages. Benchmarking project due by August 1, 2004.  Developing new approaches to pricing is an ongoing project.  Turn in bi-weekly reports on new findings to your manager by COB every other Friday. 
• Provide analytical support for sourcing projects impacting business units. Ongoing project where information must be delivered inside project deadlines and budget guidelines.

Performance Standards:
• Identified savings opportunities where existing contracts were leveraged and identified commodity areas that have not been sourced on a weekly basis for total savings of 5% over previous quarter spending.

• Benchmarked current pricing models by August 1, 2004 in 80 hours or less of time billed.  Submitted new approaches to pricing/buying various products and services on a weekly basis resulting in at least two new pricing approaches being adapted by the company by September 1, 2004.
• Provided ongoing analytical support for sourcing projects as rated by monthly project evaluations conducted by the relevant project managers and communicated to your manager on a monthly basis. 
(Suggested points: 10, .4[5.2], .3[5.3], .3[5.4])

Case Study 5.2 Evaluating Objectives and Standards

Answers will vary based on objectives and standards met.  The greater the number of boxes checked in the table, the better the objectives and standards. 
Objectives must have the following characteristics: Comments
 X Specific and clear 
 X Challenging  
 X Agreed upon   Have to assume that the manager and employee agreed on all objectives.
 X Significant   
 X Prioritized   
 X Bound by time  
 X Achievable 
 X Fully communicated Again, have to assume that the supervisor communicated objectives to subordinate and to all stakeholders involved in objectives.
 X Flexible Assume that the manager takes into consideration any valid reasons that the subordinate may not have met the objectives
 X Limited in number 
Performance standards must have the following characteristics: Comments
 X Related to the position  
 X Concrete, specific, and measurable 
 X Practical to measure  
 X Meaningful 
 X Realistic and achievable 
 X Reviewed regularly 

(Suggested points: 5, .5[5.3], .5 [5.4])

Case Study 5.3  Measuring Competencies at the Minnesota Department of Transportation

Indicators of Learning and Strategic Systems Thinking:

Frequency
This employee engages in the following behaviors:  Always     5 Very often 4 Fairly many times           3 Occasionally 2 Never 1
Enrolls in training classes         
Works to develop new skills         
Takes into account all forms of transportation and their role in the DOT when making decisions         
Solicits information from team members         
Solicits information from customers          
Volunteers relevant information to team discussions          
Able to explain role of each unit         
Able to explain link between work units and Mn/DOT         

Indicators of Organizational Learning:

Agreement
This employee engages in the following behaviors:  Very much 5 Pretty much     4 Moderately          3 Somewhat         2 Slightly           1
Demonstrates understanding of Mn/DOT culture         
Accurately explains major products and services offered         
Demonstrates understanding of how work units contribute to one another and the organization as a whole         
Establishes formal channels to accomplish work         
Establishes informal channels to accomplish work         
Understands and explains key policies, practices, and procedures         
Understands and communicates political realities and implications to work         

(Suggested points: 5, [5.6])

Case Study 5.4 Creating BARS-based Graphic Rating Scales for the Evaluating Business Student Performance in Team Projects

1. Competencies:
• Team work: Ability to work in a team to finish all project deliverables on time and according to instructions.  Assists others as necessary and constantly seeks to further relationships with team members.  Failure to develop team work will result in reduced team effectiveness due to lack of communication and cooperation between team members.
• Business knowledge: Aware of current business trends and applies relevant business research skills to projects at hand.  Always learning new skills, staying current on business news and talking with business professionals.  Failure to develop business knowledge will result in a decrease in the quality of work produced by the team.
• Communication: Ability to articulate verbally or through written communication ideas on project deliverables.  Use of correct grammar, professional presentation, and clear and concise messages characterize good communication.  Failure to communicate well will result in the decreased performance of the team, because team members will no longer understand ideas put forth.

2. Behavioral Indicators:
• Team work
o Friendly environment exists
o Team members’ input is accepted
o Offers to help other team members when necessary
• Business knowledge
o Student offers input when group is discussing business problems
o Actively researches solutions presented
o Reads business news sources on a daily basis to stay up-to-date on business trends
• Communication
o Clearly articulates points in meetings
o Uses correct grammar and word usage in written communications
o Professionally presents ideas and findings in project presentations
 
3. Critical Incidents:
Indicator High Performance Average Performance Poor Performance
Student offers input when group is discussing business problems Continuously provides input into group discussions Offers input on areas of expertise in group discussions Does not offer input into group discussions
Actively researches solutions presented Researches practical and theoretical implications of decision Researches practical implications of decision Does not research implications of business decisions
Reads business news sources on a daily basis to stay up-to-date on business trends Comments on a daily basis on the status of ongoing business developments of interest Comments on a weekly basis on the status of ongoing business developments of interest Does not comment on the status of ongoing business developments of interest
Clearly articulates points in meetings Presents ideas in a clear and concise manner and ensures that team members understand point  Presents ideas in a clear and concise manner Does not get point across in meetings, resulting in arguing with other team members or miseducation of team members
Uses correct grammar and word usage in written communications Written communications are clear, free of spelling and grammatical mistakes Clear use of language in written reports  Poor spelling and grammatical usage in written reports
Professionally presents ideas and findings in project presentations Prepares presentation so ideas are communicated well and in a professional manner Minimally prepares for presentations, so communication is rushed or drags on  Does not prepare for presentations
Friendly environment exists Takes an interest in team members outside of project Talks to team members to complete tasks Withdrawn from team
Team members’ input is accepted Supportive of other team members’ ideas Solicits, but does not support team members’ ideas Does not solicit input from other team members
Offers to help other team members when necessary Volunteers for extra assignments so that team members can finish projects on time and to the highest standards Offers to help team members if deadline is approaching Does not offer to help team members 

4. Scales:

Team Work: Ability to work in a team to finish all project deliverables on time and according to instructions.  Assists others as necessary and constantly seeks to further relationships with team members.  Failure to develop teamwork will result in reduced team effectiveness due to lack of communication and cooperation between team members.
5 Exceptional: Takes an interest in team members outside of project.
3 Competent: Talks to team members to complete tasks.
1 Major Improvement Needed: Withdrawn from team.
  
5 Exceptional: Supportive of other team members’ ideas.
3 Competent: Solicits, but does not support team members’ ideas.
1 Major Improvement Needed: Does not solicit input from other team members.
  
5 Exceptional: Volunteers for extra assignments so that team members can finish projects on time and to the highest standards.
3 Competent: Offers to help team members if deadline is approaching.
1 Major Improvement Needed: Does not offer to help team members. 

Business Knowledge: Aware of current business trends and applies relevant business research skills to projects at hand.  Always learning new skills, staying current on business news and talking with business professionals.  Failure to develop business knowledge will result in a decrease in the quality of work produced by the team.
5 Exceptional: Continuously provides input into group discussions.
3 Competent: Offers input on areas of expertise in group discussions.
1 Major Improvement Needed: Does not offer input into group discussions.
  
5 Exceptional: Researches practical and theoretical implications of decisions.
3 Competent: Researches practical implications of decisions.
1 Major Improvement Needed: Does not research implications of business decisions.
  
5 Exceptional: Comments on a daily basis on the status of ongoing business developments of interest.
3 Competent: Comments on a weekly basis on the status of ongoing business developments of interest.
1 Major Improvement Needed: Does not comment on the status of ongoing business developments of interest.

Communication: Ability to articulate verbally or through written communication ideas on project deliverables.  Use of correct grammar, professional presentation, and clear and concise messages characterize good communication.  Failure to communicate well will result in the decreased performance of the team, because team members will no longer understand ideas put forth.

5 Exceptional: Presents ideas in a clear and concise manner and ensures that team members understand point.
3 Competent: Presents ideas in a clear and concise manner.
1 Major Improvement Needed: Does not get point across in meetings, resulting in arguing with other team members or miseducation of team members.
  
5 Exceptional: Written communication clear, free of spelling and grammatical mistakes.
3 Competent: Clear use of language in written reports.
1 Major Improvement Needed: Poor spelling and grammatical usage in written reports.
  
5 Exceptional: Prepares presentation so ideas are communicated well and in a professional manner.
3 Competent: Minimally prepares for presentations, so communication is rushed or drags on.
1 Major Improvement Needed: Does not prepare for presentations.

(Suggested points: 5, .5[5.6], .5 [5.8])

Additional Cases and Worked Solutions

Case Study:  St. Jessica’s Urban Medical Center

The Board of Directors at St. Jessica’s Urban Medical Center has hired you as a consultant to guide them in establishing a results approach to measuring performance. 

1. Discuss the key features of a results approach to measuring performance.   Use the job description for a Registered Nurse that was provided in “Case Study:  St. Jessica’s Urban Medical Center Emergency Room” (Instructor’s Manual, Chapter 4) to develop an example of each of these factors.
(Suggested points: 10, .4[5.1], .2[5.2], .2[5.3], .2[5.4])

After reviewing your proposal, the management at St. Jessica’s realizes that, in some aspects, the hospital’s employees do not have control over many results, so they convince the Board to include a behavior approach to measuring some aspects of performance.

2. The Board asks you to provide them with a discussion of the key features required to include a behavior approach to performance measurement.  Use the job description for a Registered Nurse that was provided in “Case Study: St. Jessica’s Urban Medical Center Emergency Room” (Instructor’s Manual, Chapter 4) to develop an example of each of the key features of a behavior approach to performance measurement.
(Suggested points: 10, .5[5.5], .5[5.6])
Answers:

1. (Note to Instructors:  The student may use any examples relating to the job description of Registered Nurse in Module 4 as long as they are pertinent and demonstrate ability to apply the material in the text.  Examples given are for illustration only.)

To fully implement a results approach to measuring performance, the Medical Center management must be able to answer the following questions:
• What are the different areas in which this individual is expected to focus efforts (key accountabilities)?
• Within each area, what are the expected objectives?
• How do we know how well the results have been achieved (performance standards)?

A. Actual implementation of the results approach will require you to develop key accountabilities.  Accountabilities are determined to identify a limited number of highly important results that, when achieved, will have a dramatic impact on the overall success of the organization.  The first step will be to collect information about the job and identify areas of the job where the employee is responsible for producing results and their relative degree of importance; e.g., percent of time spent performing the accountability, consequences of inadequate performance, and consequences of error.  For example, in the job description for Registered Nurse, one of the accountabilities might be:

Documentation:  Maintaining accurate, detailed reports and records.  An RN might spend 30% of his/her time on recording and maintaining information in patient records.  If the records are not complete and accurate, it would be possible for a doctor to order inappropriate medication, which could lead to deterioration of the patient’s condition or even death.

B. When all the key accountabilities are identified, the second step is to set objectives for each.  Objectives should be: (a) specific and clear, (b) challenging, (c) agreed upon,      (d) significant, (e) prioritized, (f) bound by time, (g) achievable, (h) fully communicated, (i) flexible, and (j) limited in number.  For example, two objectives that could be developed for the Registered Nurse accountability of Documentation might be:

• Provide prompt updates in the patient’s record of any changes in condition, medication, and vital signs. 
• Ensure that all written documentation is clear, legible, and can be understood by other medical personnel.

C. Finally, the third step involves determining performance standards. These are yardsticks designed to understand to what extent the objective has been achieved.  In creating standards, we must consider the quality, quantity, and time dimensions.  Good standards are: (a) related to the position, (b) concrete, specific, and measurable, (c) practical to measure, (d) meaningful, (e) realistic and achievable, and (f) reviewed regularly.  For example, a performance standard for one of the objectives given for Registered Nurse might be:
Prompt Updates: Each entry in a patient’s record will be made within 20 minutes of the action or observation.  The patient’s hospital number, room number, the time of the observation, and the name of the person taking action will be included with the description of the action or observation.

After reviewing your proposal, the management at St. Jessica’s realizes that, in some aspects, the hospital’s employees do not have control over many results, so they convince the Board to include a behavior approach to measuring some aspects of performance.

2. (Note to Instructors:  The student may use any examples relating to the job description of Registered Nurse in Module 4 as long as they are pertinent and demonstrate ability to apply the material in the text.  Examples given are for illustration only.)

A. In measuring performance adopting a behavior approach, the first step involves identifying competencies. Competencies are measurable clusters of KSAs (knowledge, skills, abilities) that are critical in determining how results will be achieved.  For example, in the job description for Registered Nurse, one of the competencies might be customer service, which includes knowledge (of principles and processes for providing customer needs assessment, meeting quality standards for services, and evaluation of customer satisfaction), the skills of active listening and speaking, and the ability to identify problems.

B. The second step involves identifying indicators that will allow us to understand the extent to which each individual possesses the competency in question.  These indicators are behavioral manifestations of the underlying (unobservable) competency.  For example, an indicator that could be developed for the Registered Nurse competency of customer service might be:

In describing competencies, one must:
a. First clearly define it; for example (adapted from the job description for Registered Nurse): customer service includes knowledge (of principles and processes for providing customer needs assessment, meeting quality standards for services, and evaluation of customer satisfaction), the skills of active listening and speaking, and the ability to identify problems.  Another definition from dictionary.com:  “assistance and other resources that a company provides to the people who buy or use its products or services” is found at http://dictionary.reference.com/search?q=customer%20service.
b. Describe behavioral indicators showing the presence of the competency; for example (adapted from the job description for Registered Nurse): customer service includes the use of active listening, which is described as: “Giving full attention to what other people are saying, taking time to understand the points being made, asking questions as appropriate, and not interrupting at inappropriate times.”
c. Describe behavioral indicators showing the absence of the competency; for example (adapted from the job description for Registered Nurse): customer service includes the use of active listening.  Failure to use active listening would include interrupting a patient, looking at a watch, tapping a foot, looking away from the patient, sighing and rolling eyes, changing the subject without responding to the patient’s comment, etc.
d. List suggestions for developing the competency;  for example (adapted from the job description for Registered Nurse): the competency of customer service can be developed through attending various training courses.  One popular definition of customer service is “exceeding the customer’s expectations.”  Note: If management wants to develop a culture of such customer service, such expectations must be spelled out and consistently supported throughout the organization.

C. Once the indicators are identified, the last step includes choosing an appropriate measurement system: comparative or absolute.
a. Comparative systems base the measurement on comparing employees with each other and include simple rank order, alternation rank order, paired comparisons, and forced distribution.  Comparative systems are easy to explain and the resulting data are easy to interpret, which facilitates administrative decisions.  On the other hand, employees are usually compared to each other in terms of one overall single category instead of in terms of specific behaviors or competencies.  This produces less useful feedback that employees can use for future improvement.  For example, in the case of the Registered Nurse, such a measurement system could be used to let each nurse know his or her ranking among the other nurses.
b. Absolute systems include evaluations of employees’ performance without making direct reference to other employees. Such systems include essays, behavior checklists, critical incidents, and graphic rating scales. For example, in the case of the Registered Nurse, such a measurement system could be used to describe his or her behavior in terms of what is expected.

Case Study:  The College of Business at VIP University

You are a newly hired professor with a Ph.D. in Human Resources Management.  You have special interests in both training and in performance measurement.  You decide to develop a comparative performance measurement system to measure student performance in your HR Management classes.

1. Critically assess the advantages and disadvantages of implementing such a system.
(Suggested points: 5, [5.7])

2. Explain the kinds of systems that are available for you to consider.  Choose one approach and provide both a description of how you would implement it in a class of 30 students and an explanation of why this was the best choice of the variety of comparative systems.
(Suggested points: 10, [5. 7])

At the end of the school year, a student initiative calls for a revision of the grading system at VIP University.  Every professor will be required to develop an absolute performance measurement system.  The process is complicated by the fact that many students are distance learners who interact on the Internet rather than in person. 

3. Describe how you will implement an absolute performance measurement system in your classes next year, recognizing that you can devote no more than 10 hours per week to preparing for each class, actually participating in the class, and developing and grading any homework and exams.  What will you consider in choosing a system and how will you go about implementing it?
(Suggested points: 10, [5.8])

Answers:

1. Comparative performance measurement systems are easy to explain, straightforward, and generally control better for the leniency, severity, and central tendency biases and errors that can be found in absolute performance measurement systems.  However, rankings tend not to be specific enough to provide useful feedback to students and they may subject the university or the professor to legal challenges.  There is no information in such scales about the relative distance between student performances and there are specific concerns about the usefulness (and appropriateness) of using the forced distribution method in a university setting.  In addition, it may be difficult to assign letter grades to the various levels.

2. Comparative performance measurement systems compare employees with each other.
a. Simple rank order - employees are ranked from best performer to worst performer.
b. Alternation rank order - in an alternation rank order procedure, the supervisor initially lists all students.  Then, the supervisor selects the best performer (#1), then the worst performer (#n), then the second best (#2), then the second worst (#n - 1), and so forth, alternating from the top to the bottom of the list until all students have been ranked.
c. Paired comparisons - explicit comparisons are made between all pairs of students to be evaluated.  The number of pairs of students to be compared is computed by the following equation:
 
where n is the number of students to be evaluated.
d. Forced distribution - students are apportioned according to an approximately normal distribution.  This method assumes that performance scores are normally distributed, with some students performing very highly, some poorly, and the majority in between.  However, this assumption may not hold true for all units within an organization.  Some units may have a high performing culture and systems in place such that the majority of members perform at a very high level.  Conversely, other units may have a majority of members who perform at a below-expectations level.  Another disadvantage of implementing a forced distribution system is that this may discourage students from engaging in contextual performance behaviors.

At the end of the school year, a student initiative calls for a revision of the grading system at VIP University.  Every professor will be required to develop an absolute performance measurement system.  The process is complicated by the fact that many students are distance learners who interact on the Internet rather than in person. 

3. Absolute performance measurement systems compare students with prespecified performance standards.  I considered the following approaches, including their advantages and disadvantages:
a. Essays:  Here I could write an essay describing each student’s strengths and weaknesses, making suggestions for improvement. An advantage is that essays have the potential to provide detailed feedback; disadvantages to essays include the fact that they are unstructured and may lack detail, their usefulness depends on my writing skill, and they are difficult to use in grading decisions, partially due to the lack of quantitative information provided.  They are also quite time consuming, considering that each instructor may have thirty or more students per class. 
b. Behavior checklists:  I could use a form listing behavioral statements that are indicators of the various competencies to be measured. An advantage is that such a checklist is easy to understand; disadvantages are that the scale points used are often arbitrary and it is difficult to get detailed and useful feedback from the numerical rating.
c. Critical incidents: This approach involves gathering reports of situations in which students performed behaviors that were especially effective or ineffective in accomplishing their learning objectives. An advantage is that this allows me to focus on actual student behavior rather than on vaguely defined traits; the disadvantage is that collecting such information is very time consuming.
d. Graphic rating scales are considered the most popular tools to measure performance. It is important to ensure that the response categories are clearly defined, that interpretation of the rating by an outside party would be clear, and that the student and I would both understand the rating. 
e. Behaviorally Anchored Rating Scales (BARS) improve on graphic rating scales by using critical incidents as anchors.  Their development is dependent on active student involvement at several points, as well as final professor and student checking of the items for dependability of the ratings. 

Since I have already developed class syllabi that outlined required student behaviors and what was necessary to earn various grade levels, I opted to use the essay format to further explain my grading system and how student strengths and weaknesses (as displayed in their papers, exams, and Internet class participation) contributed to their final grades.  I would like to use BARS sometime, but this is an expensive and time-consuming undertaking that would involve full university support and participation in developing the response categories and identifying how critical incidents could be used productively in a distance learning arrangement.