Scale development refers to the process of creating a set of items that measure a construct. A construct is something that cannot be observed directly, such as job satisfaction, trust, burnout, motivation, psychological safety, or manager support. These constructs are important, but they cannot be measured in the same way as age, height, or salary. They have to be measured through items that represent the construct.
The quality of a scale depends on the quality of the process used to develop it. If the construct is not clearly defined, the items will be difficult to interpret. If the items are poorly written, respondents may answer them in different ways. If the items do not fit together, the final score may not represent the construct. For this reason, scale development should follow a clear process.
Define the construct
The first step is to define the construct. This means stating what the construct is, what it includes, and what it does not include. This step is important since the definition determines what items should be written.
For example, if the goal is to measure manager support, the construct needs to be defined clearly. Manager support could refer to emotional support, practical help, feedback, availability, or concern for employee development. These ideas are related, but they are not the same. If the definition is unclear, the items may measure several different things at once.
The definition should also specify whether the construct has one dimension or several dimensions. A construct with one dimension can be measured with one set of items. A construct with several dimensions may require separate sets of items for each dimension. For example, job satisfaction can be measured as general satisfaction with the job, or it can be divided into satisfaction with pay, coworkers, supervisors, promotion, and the work itself.
A clear definition gives the scale a boundary. It helps determine which items belong in the scale and which items should be excluded.
Decide how the items will be generated
After defining the construct, the next step is to decide how the items will be generated. Items can be developed deductively or inductively.
A deductive approach starts with theory. The researcher reviews the literature, defines the construct, and writes items that reflect that definition. This approach is useful when the construct is already understood and there is enough theory to guide item development.
An inductive approach starts with people's descriptions. The researcher asks people to describe the experience or behavior in their own words. These responses are then grouped into themes, and items are written from those themes. This approach is useful when the construct is newer or less clearly defined.
In both cases, the items should come from a clear source. They should reflect either a theoretical definition or a careful analysis of how people describe the construct.
Write more items than the final scale needs
The initial item pool should include more items than the final scale. Some items will be removed later. Some may be unclear. Some may overlap too much with other items. Some may fail to represent the construct well.
A useful goal is to write several items for each part of the construct. If the final scale is expected to have four to six items, the first item pool should contain more than that. This gives the researcher enough room to test the items and retain the strongest ones.
At this stage, the goal is to cover the construct well. The researcher should avoid writing only a small number of items and treating them as final too early.
Write simple and focused items
Items should be simple, direct, and easy for the target respondents to understand. The wording should match the population that will answer the scale. Items should avoid jargon, vague words, double negatives, and unnecessarily complex sentences.
Each item should ask about one idea. For example, the item "My manager is supportive and gives useful feedback" contains two ideas. A respondent may agree that the manager is supportive but disagree that the manager gives useful feedback. This makes the answer difficult to interpret.
A better approach is to separate the ideas:
"My manager is supportive when I need help."
"My manager gives useful feedback about my work."
Items should also avoid leading wording. The item should not push the respondent toward a specific answer. It should give respondents enough room to report their actual experience.
Items should also be consistent in perspective. A scale should avoid mixing items that refer to different targets, time frames, or types of responses unless there is a clear reason for doing so.
Check whether the items match the construct
After writing the items, the researcher should check whether the items represent the construct. This is usually done before the main survey.
One approach is to give judges or respondents the construct definitions and ask them to match each item to the correct construct. If an item is intended to measure manager support, people should be able to identify it as manager support. If many people place the item under a different construct, the item may be unclear or may need to be removed.
This step helps establish whether the items cover the intended content. It also helps identify items that look acceptable to the researcher but may be interpreted differently by respondents.
Test the items with the target population
After the content check, the items should be tested with a sample that reflects the target population. If the scale is meant for employees, it should be tested with employees. If it is meant for managers, it should be tested with managers. The sample should match the context in which the scale will be used.
At this stage, the researcher examines how the items perform. This includes looking at the mean, standard deviation, response range, and item-total correlations. Items should show enough variation. If almost everyone gives the same answer, the item may not help distinguish between respondents.
The researcher should also examine whether each item relates to the overall scale in a reasonable way. Items that do not relate well to the other items may not belong in the scale.
Reduce the items
After collecting data, the researcher reduces the scale by removing weaker items. This step should be based on both statistical evidence and the construct definition.
Factor analysis is often used to examine whether the items group together in the expected way. If the construct is expected to have one dimension, the items should mainly reflect one dimension. If the construct is expected to have several dimensions, the items should separate into those dimensions.
Items with weak loadings, strong cross-loadings, or poor item-total correlations may need to be removed. However, the researcher should not remove items only to improve statistics. The retained items still need to represent the construct. A scale can look strong statistically and still measure the construct too narrowly.
The goal is to keep items that are clear, useful, and representative of the construct.
Check reliability
Reliability refers to the consistency of the scale. In survey research, internal consistency is commonly used to assess whether the items in a scale work together. Cronbach's alpha is often reported for this purpose.
A reliability estimate around .70 is usually treated as a minimum for exploratory work. Higher reliability may be needed when the scale is used for important decisions. However, reliability alone is not enough. A scale can have high reliability if the items are very similar, even if the scale does not fully represent the construct.
Reliability should therefore be interpreted together with the definition of the construct, the factor structure, and the content of the items.
Confirm the structure
After the initial scale is developed, the structure should be tested again, ideally with a new sample. This step helps show that the scale does not work only in one specific dataset.
Confirmatory factor analysis can be used to test whether the items fit the expected structure. If the scale has one dimension, the model should reflect one dimension. If the scale has several dimensions, the model should reflect those dimensions.
This step gives stronger evidence that the scale has a stable structure and can be used beyond the original sample.
Show that the scale relates to other variables in expected ways
A scale should also relate to other variables in ways that make theoretical sense. For example, a burnout scale should relate to exhaustion and strain. A job satisfaction scale should relate to attitudes about the job. A manager support scale should relate to feedback, trust, or employee attitudes toward the manager.
This step helps show that the scale measures the intended construct. The scale should be related to constructs that are theoretically similar and less related to constructs that are theoretically different.
The researcher should identify these expected relationships before testing the scale. The pattern of relationships provides evidence for whether the scale behaves as expected.
Revise and test again
Scale development usually requires revision. Some items may need to be rewritten. Some dimensions may need more items. Some items may need to be removed. The definition of the construct may also need to be clarified if the items do not behave as expected.
A scale should not be treated as finished after one round of item writing. The process involves defining the construct, writing items, checking content, collecting data, reducing items, testing reliability, confirming the structure, and examining relationships with other variables.
The final scale should be clear, short enough to use, and supported by evidence. Each item should serve the construct. Each analysis should help determine whether the items are measuring what they are intended to measure.
References
Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1, 104–121.
Tay, L., & Jebb, A. T. (2017). Scale development. In S. Rogelberg (Ed.), The SAGE Encyclopedia of Industrial and Organizational Psychology.