Achievement Test Construction Techniques

Introduction

Achievement is the accomplishment or proficiency of performance in a given skill or body of knowledge. Therefore, it can be said that achievement implies the overall mastery of a pupil in a particular context. Testing is a systematic procedure for comparing the behaviour of two or more persons. Hence, an achievement test is an examination to reveal the relative standing of an individual in the group, with respect to his /her performance.

Achievement is the competence of a person in relation to a domain of knowledge. An achievement test is a test of knowledge or proficiency-based on something learned or taught. The purpose of an achievement test is to determine a student’s knowledge in a particular subject area.

Achievement tests assess the knowledge, abilities and skills that are at the centre of direct instruction in schools. Achievement tests may be standardized or non-standardized. Standardized tests are created by professional agencies and the same materials and administration procedures are used for all students. The non-standardized tests have neither the assessment materials tried out by a publisher nor has any student based data collected concerning how well the test is functioning.

However, achievement tests can be written to ascertain students’ level of learning within a course in a major or across their undergraduate education. For test results to be useful, they must follow basic measurement standards.

The Fundamentals of the Achievement Tests

The general fundamental purpose of achievement tests is to promote learning. Specifically,

  • To measure an individual’s achievement of course objectives.
  • To assess the group’s performance.
  • To evaluate the test and the items.
  • To evaluate and improve instruction and the curriculum.

However, achievement test results should accurately measure individual differences or achievement of a certain pre-specified mastery level and should always foster learning. To accomplish these purposes, a test must be valid and reliable. Any measuring instrument that measures the attainments or accomplishments of a pupil’s achievement must be valid and reliable. Validity is addressed when a test plan is formulated to accurately represent the course content and depth of learning achievement in a course. A test result must be reliable or repeatable to be confident that a student’s score is a true reflection of an examinees achievement.

Testing Factors

Several testing factors have been shown to contribute to learning.

First, the type of test students expected, oftentimes guide their study behaviour. If a multiple-choice test is planned, students typically will study only for recognition. If it is known that a test will emphasize factual information, a student will memorize facts that usually are forgotten quickly.

Second, test questions written above the rote level have a greater potential for promoting transfer and retention. Therefore, most tests should be written to include items to stimulate higher cognitive levels.

Because the assessment of students’ learning requires adequate and accurate sampling of course content, the multiple-choice test is recommended for achievement type testing. If the achievement test requires measuring the highest cognitive, combining both easy and multiple-choice items into a single test is commonly suggested.

However, in applying essay or multiple-choice tests the following could be considered.

Essay Examination

Uses: When measuring the highest cognitive levels (synthesis and evaluation levels of Bloom’s taxonomy).

When a response needs to be created.

When evaluating writing ability

Multiple Choice Tests

Uses: when measuring achievement at the knowledge, comprehension, application and analysis cognitive levels.

Essay

Advantages

  • Relatively short
  • The amount of time required to construct the items is reduced
  • Allows for creativity, originality and comprehension

Multiple Choice Tests

Objective scoring- once correct answers are decided.

  • Evaluation of validity is possible by comparing the test to the table of item specifications.
  • Evaluation of reliability is possible.
  • A thorough sampling of course content is possible.
  • Particular problems in the examination or in the instruction of learning can be revealed through item analysis resulting from test scores.

Essay

Disadvantages

  • Objective scoring is questionable and more difficult.
  • No generally acceptable criteria for demonstrating the validity and reliability of test.
  • Course sampling is limited
  • Time-consuming in evaluating responses

Multiple Choice Tests

Disadvantages

  • Time-consuming, in the construction
  • Difficult to construct items of the highest cognitive levels.
  • The examiner must have some training or knowledge in test construction and item analysis techniques to write valid and reliable tests.

How to Construct Valid & Reliable Classroom Achievement Test

When developing a test, to adequately and accurately measure the achievement of course content and to achieve reliable scores, certain guidelines have to be followed: Constructing a valid test that reflects course objectives is an integral part of planning the course. Thus “teaching to the test” is desirable and necessary. Also for assessing differences in learning, assigning grades or determining to master, there should be confidence in the test scores.

Achievement test scores are either norm-referenced or criterion-referenced. Norm-referenced tests emphasize individual differences, how students compare with each other, while criterion-referenced tests highlight how examinees’ performance compare to a specific standard or level of mastery, logically or empirically determined. To foster validity, a well-constructed test blueprint also referred to as a table of item specifications provides the necessary structure. Therefore, to measure the achievement of a unit, course or programme’s objectives, the test blueprint must be an accurate representation of the content and cognitive levels taught.

In summary, to construct a test blueprint in order to achieve a valid achievement test, the following steps could be used.

  • List important course content or topics
  • Identify appropriate cognitive levels using Bloom’s taxonomy of educational objectives for each of the course objectives.
  • Determine the number of items for the entire test and for each cell that is course content at a cognitive level.
  • Furthermore, addressing reliability, a reliability coefficient can range from 0.00 representing no consistency to 1.00 representing perfect consistency. Typically, a reliability coefficient of 0.70 or higher is considered necessary to place confidence in the scores of norm-referenced achievement tests. Hence, by adhering to the following the likelihood of constructing a reliable norm-referenced test with results reflecting a normal distribution is enhanced.
  • Write longer tests with well-constructed items.
  • Include items that are positive discriminators.
  • Write items that are moderately difficult.

In summary, to achieve a test with valid results, adhere to the test blueprint and ensure the items are written to reflect the appropriate cognitive levels. A test that promotes retention and transfer has items written at various levels of Bloom’s taxonomy or some other defensible learning taxonomy.

Be certain that items are written to accurately reflect the cognitive levels encouraged in the course for each important content area. In multiple-choice questions general item writing, for instance, an item or question contains three parts:

  • The stem in which the question is asked or the problem is stated.
  • The correct option.
  • The incorrect option is also called the distracters or foils.

Steps in Constructing an Achievement Test

the following steps could be followed which will make the test objective, realistic and valid.

1st Step

Selection of teaching objectives for measurement

The teaching objectives should be selected from the objectives of the subject. That is how much content has been studied, the need of the student, the importance of specific topics in the content.

Thus the reasons for selection include;

  • All levels of the cognitive domain (knowledge, comprehension, application, analysis, synthesis and evaluation).
  • All the levels of the affective domain (receiving, responding, valuing and conceptualization).
  • All skills of the psychomotor domain (drawing skill, computational skill,  constructional skill, observational skill, and problem-solving skill).

2nd Step

Assigning weight to selected levels: After the selection of the levels, weight is assigned to the levels, the tasks are done and the importance of the levels. For instance, such a table could be used= = scores percentage.

Selected teaching levels         Scores                Percentage

Knowledge                             40                                 40

Comprehension                      30                           30

Application                             20                          20

Skill                                         10                          10

Total                                       100                        100%

3rd Step

The weighting of Content: Content is used as the means of realizing objectives, therefore, questions have to be constructed on their basis. It’s, therefore, necessary to give weight to it. The weight should be given bearing in mind that there is distribution in the nature, importance and scope of each topic. Thus, the table should include Serial number, Topic, Number of items, Score, Percentage.

4th Step

Giving weight to the type of items: The number of items is determined by their types and relative marks. There is long answer, short answer, and objective types.

5th Step

Determining Alternatives:  According to the type of questions, you determine how many alternatives or options should be given.

6th Step

Division of section: This is necessary only when the scope and type of question are diverse and different. Also if the nature of the test seems heterogeneous, there is a need to make a separate section comprising each type of item.

7th Step

Estimation of time: Estimation of the total time the test is likely to take. This is estimated on the basis of type and number of items.

8th Step

Preparation of Blueprint: In a blueprint, one can see the topics, teaching objectives and types of questions, number of items and distribution of scores and their mutual relationship. It is the basis for test construction. It provides, in a nutshell, the view of the entire test.

9th Step

Preparation of score key: This improves the reliability of the test and therefore should be provided. There should be the provision of the procedure for scoring the answer script and how marks will be awarded to the test items.

You may like to read:-

Alternative Test Construction Techniques

What is Cumulative Record in Continuous Assessment

Record Keeping for Continuous Assessment

Counselling in Continuous Assessment

What is Continuous Assessment