Background Information:
A challenge when creating reliable and valid assessments is that the content of the questions
need to not only be objective (based on the facts and current research of a topic/issue), they need
to be effective too. In other words, the purpose of each question needs to clearly challenge the
test taker’s understanding of the content and the choices are not too difficult or too easy. It’s the
assessment equivalent of the three little bears: what’s just right?
Two methods to test the quality of your questions are the difficulty and discrimination indexes.
The data that these indexes provide can help you identify what questions are showing validity
and reliability, as well as identifying some questions that should be replaced.
The difficulty index is used to compare how many students were able to answer the question.
The formula has you divide the number of students who selected the correct answer by the total
number of students taking the exam.
The formula looks like this:
Difficulty = Students with Correct Answer for the Question / Total Students
For example, Table 1 illustrates how many students selected each answer choice for two
questions on a multiple choice exam (answers were labeled A, B, C, D).
Table 1
Question A B C D
#1 0 3 24* 3
#2 12* 13 3 2
* Denotes correct answer.
For Question #1, you may already at a glance notice that choice A was not a very good distractor
(an answer that sounds likely but is not fully accurate; they are used to challenge the student to
think critically)– no one selected that answer. When we do the math and compute the difficulty,
we’re going to get more information.
For Question #1, we divide the number of students who choose the correct answer (24) by the
number of total students (30), which has a result of 0.8. Once calculated, the index score is going
to indicate a few things that will impact how you evaluate the question:
○ Scores 0.75 or above: the question is very easy. It’s possibly too easy since
everyone knew the answer or was able to guess correctly.
○ Scores 0.25 to 0.75: the question is in the middle of the road, not too easy and not
too difficult. It’s challenging, but it’s not so easy that you could guess and get it
right.
○ Scores 0.25 or lower: the question is too hard. It’s so challenging that students
who scored relatively well struggled with identifying the purpose of the question
or there was possibly more than one right/best answer.
For Practice 1: Try calculating the difficulty index for Question 2 in Table 1, and see if you can
calculate that question’s index score (the answer is at the bottom of the page).
The discrimination index looks at comparing how well the students who scored high on the
overall test scored versus the students who had an overall low score. To get started, it’s best to
organize a chart listing each student based on their overall scores from highest to lowest, as well
as additional columns for each question on the exam. For each question, you want to note
whether a student then got the question incorrect (0) or correct (1).
Next, split the class into an upper group (the top third of performers) and a lower group (the
bottom third). Once you have determined how many students are in each group. Count the the
number of students in the lower group who got the item correct and the number of students in the
upper group who got the item correct. Subtract the total of the lower group from the total of the
upper group, and note the difference between the lower and the upper group.
Finally, you will divide the difference between the lower group from the upper group by half the
total number of students in the class.
The formula look like this:
Discrimination = (Number of students in the upper group – Number of students in the
lower group) / (Total number of students in the class / 2)*
*complete the computations that appear in parenthesis first, then divide the totals. This is where
most students run into an issue.
When interpreting the discrimination index results, remember:
● Positive discrimination (0 to 1) means the upper group did better on the exam than the
lower group; this is normal since it means the students who did well overall on the exam
did well on this question; it means this question is in line with the rest of the exam.
● Negative discrimination (-1 to 0) means the lower group students did better on the exam
than the upper group. This means that the overall most knowledgeable students are
getting the item wrong and students who struggled more with the overall exam are getting
the question right more often. It may indicate that you’re measuring something other than
what the rest of the test is measuring with this question.
Lets take a look at Table 2 for a sample test item analysis.
Table 2
Student Total
Score (%)
Questions
1 2 3 4 5
1 90 1 0 1 1 1
2 90 1 0 1 1 0
3 80 0 0 1 0 1
4 80 1 0 1 1 0
5 70 1 0 1 0 0
6 60 1 0 0 0 1
7 60 1 0 1 0 1
8 50 1 1 0 1 0
9 50 1 1 0 0 1
10 40 0 1 0 0 0
Table 2 features the overall scores for the students in the upper and lower group in a class of ten
(10) students. In Question #1, there were four (4) students in the upper group who got it correct
and four (4) students in the lower group, and the difference between them is zero (0). Since there
are ten (10) students total, the number in each group is five (5). So, I would be dividing zero (0)
by (5) to get my final result.
After finishing this calculation, the discrimination index score is zero (0), which indicates
positive discrimination, so this question is valid and reliable since it’s on-topic.
For Practice 2: Try calculating the discrimination index for Question 2 in Table 2, and see if you
can calculate that index score (the answer is at the bottom of the page).
Part 1:
You’re being asked to complete a test item analysis using Table 2. Follow these steps to
determine the Difficulty Index and the Discrimination Index.
1. After the students are arranged with the highest overall scores at the top, count the
number of students in the upper and lower group who got each item correct.
2. Determine the Difficulty Index by dividing the number who got it correct by the total
number of students.
3. Determine the Discrimination Index by subtracting the number of students in the lower
group who got the item correct from the number of students in the upper group who got
the item correct. Then, divide by the number of students in each group (in this case, there
are five in each group).
4. Record your responses for Questions 3-5 by completing the provided Table below.
Item # Correct (Upper
group)
# Correct (Lower
group)
Difficulty
(p)
Discrimination
(D)
Question 3
Question 4
Question 5
After completing the computations, please answer these four questions based on your test item
analysis:
1. Which question was the easiest? Why?
2. Which question was the most difficult? Why?
3. Which item has the poorest discrimination? Why?
4. Which questions would you eliminate first (if any) – why?
Part 2:
Write a two-page (2) reflection about how this exercise is or is not relevant to your practice.
Consider the purpose of collecting and organizing data, or if there are alternate methods that
would be equally or more relevant to your practice based on research.
Reflection Length Requirement: Two (2) pages not including the completed chart
Format Requirement: APA
Research Requirement: two (2) secondary outside resources should be cited in-text and in the
Reference Page (the articles provided in this unit do not count towards this requirement. You are
encouraged to cite them in your paper, yet they will not count towards the secondary resource
requirement).