Salary Survey Selection: How to Choose Compensation Surveys You Can Defend

Salary Survey Selection: How to Choose Compensation Surveys You Can Defend

Your market data is only as good as the survey behind it. Pick a survey full of the wrong companies, and every range you build on it inherits the error. Pick one that covers only a third of your jobs, and you will spend the year patching gaps with guesses.

Most comp teams choose surveys by habit. They renew what they bought last year, or they buy what a peer uses. That works until a leader asks why your engineers are paid at the 40th percentile of a market you never compete in.

This guide gives you a repeatable way to choose. You will score each survey on six criteria, set minimum thresholds, and blend sources so no single report drives your pay decisions. If you are new to the full process, start with our salary benchmarking guide and come back here.

TL;DR

• Score every survey on six criteria: participants, job coverage, data cut, sample size, data age, and methodology.
• Aim for a survey that covers at least 75% of the jobs you need to price.
• Look for at least five organizations behind each data point, with no single company above 25% of it.
• Use two or three surveys per job family, not one, and document why you chose each.

Start with the question, not the vendor

Before you open a single catalog, write down what the data must do. Surveys serve three different jobs, and each needs different data.

• Pricing benchmark jobs to build or refresh your salary ranges.
• Setting a pay position by job family, such as P50 for corporate roles and P65 for engineers.
• Checking offers and counteroffers in a hot hiring market.

A survey that excels at one can be weak at another. A broad cross-industry report suits corporate jobs like accounting and HR. A narrow industry survey suits roles that exist only in your sector, such as clinical research or pipeline engineering.

Also decide who your talent market is. List the companies you recruit from and the companies that hire from you. That list becomes your yardstick for every criterion below.

The six criteria that separate good surveys from noise

1. Participants

Check who submits data. The survey should include the employers you actually compete with for talent, in your industry or in the industries that hire your people.

Pearl Meyer, a compensation consultancy, advises that a survey should include the companies you recruit from and the companies that hire from you (Pearl Meyer). Ask the vendor for a participant list. Count how many of your named competitors appear.

2. Job coverage

Count how many of your benchmark jobs the survey can price. Pearl Meyer offers a rule of thumb: the survey should cover at least 75% of the positions you need to price.

Say you have 60 benchmark jobs. A survey that matches 38 of them covers 63%. That is below the bar, so plan to add a second source for the remaining jobs rather than forcing weak matches. Match quality matters as much as match count, which we cover in our guide to salary survey job matching.

3. Data cut

Confirm the survey reports the pay elements you need. Base salary is the minimum. If you manage total cash, you also need target bonus and actual bonus. If you offer equity, you need long-term incentive data.

Then check the filters. Can you cut by revenue size, headcount, industry, and geography? A national median is a poor guide for a 200-person company hiring in Boise. A survey that lets you filter to your size and region is worth more than a bigger one that does not.

4. Sample size and confidentiality rules

Thin data produces swings. Pearl Meyer recommends at least five participants behind each published data point, with no single company accounting for more than 25% of it.

Look at the number of organizations and the number of incumbents. A P50 built from 4,000 incumbents at three companies is weaker than one from 600 incumbents at 25 companies. When a cell shows too few organizations, the survey should suppress it. If a vendor publishes thin cells without warning, treat that as a red flag.

5. Data age and effective date

Every survey has an effective date, the day the pay data is true. Survey data is often 6 to 12 months old by the time you use it. Pearl Meyer notes that data should be no more than three months old at publication in line with antitrust guidance. Note that the DOJ withdrew its long-standing antitrust safe-harbor statements on information sharing in 2023 (Foley & Lardner), so confirm current practice with your legal counsel and your survey vendor.

You will almost always need to age the data to today. Our guide to aging market data shows the math. Choose a survey with a clear, recent effective date so the adjustment stays small.

6. Methodology and vendor support

Ask how the vendor handles bad submissions. Do they run outlier checks? Do they follow up on anomalies? Pearl Meyer flags that a survey with a single submission round and no follow-up on data errors may be lower quality.

Also check the practical side: a documented submission process, clear job-matching guides, and a named person who can explain results when your numbers look odd.

Build a scorecard

Turn the six criteria into a simple scorecard so the choice is not a gut call. Score each survey from 1 to 5 on each criterion, then weight the criteria by what matters to you.

To compute a weighted score, multiply each score by its weight and add the results. For Survey A: (4 × 0.25) + (5 × 0.25) + (4 × 0.15) + (3 × 0.15) + (4 × 0.10) + (4 × 0.10) = 4.1.

Do not buy on score alone. Survey C scores 3.9 overall but only a 2 on coverage. If it cannot price your core jobs, it cannot be your primary source, even though its participants are excellent. Set minimum thresholds, such as no score below 3 on coverage, and screen out failures first.

Blend sources instead of picking a winner

Rarely does one survey cover every job family well. Use a primary survey for each family and a second to validate it.

A common pattern looks like this.

• Corporate functions: one broad cross-industry survey plus one survey of companies your size.
• Technical roles: a specialist tech survey as primary, with a broad survey as a check.
• Sector-specific roles: an industry survey only, since a general survey will not match.

When you combine sources, weight them. If Survey A covers a job with 40 organizations and Survey B with 12, give A more weight. Record the weights in a short note. Auditors, executives, and employees who question their pay will all ask how you got the number.

Check public data as a sanity test too. The Bureau of Labor Statistics OEWS program publishes wage estimates by occupation and area. It will not match your jobs closely, but it can flag a survey result that looks far out of line.

Connect survey data to your job evaluation

Survey data prices your benchmark jobs. It does not tell you what to pay jobs that have no market match. That is where job evaluation comes in.

Score every job with a point-factor method so your internal hierarchy is clear, then use survey data to anchor the pay line. The market pricing vs job evaluation article explains how the two work together. Once you know your target percentile, our guide on lead, lag, or match shows how to choose it by job family.

Common selection mistakes

• Choosing on price. A cheap survey that covers half your jobs costs more in patchwork later.
• Buying the biggest survey. Large samples dominated by industries you do not compete with mislead you.
• Ignoring the effective date. Comparing a January survey to a September one without aging produces false gaps.
• Forcing job matches. A 70% match priced as a 100% match skews your range.
• Never reviewing the choice. Your talent market changes. Re-score your surveys every year before renewal.

Ready to turn market data into a defensible pay structure? Start free with PointFactors and see how point-factor scoring and market data work together.

FAQ

How many salary surveys should I use?

Use two or three per job family. One primary survey gives you the main data, and a second or third validates it. More than three adds cost and complexity without much gain.

How much does a salary survey cost?

Prices vary widely. Many vendors give free reports to participants who submit data, while premium surveys charge fees. Check each vendor's current pricing, and compare cost per benchmark job covered rather than total price.

Should I use a free or a paid survey?

Free or participant-only surveys can work well for broad roles, especially if you contribute data. Paid surveys often win on filters, sample size, and support. Use the scorecard to decide rather than assuming price equals quality.

What if no survey covers one of my jobs?

Use a close benchmark and adjust by job evaluation points. Score the unmatched job with your point-factor system and place it on the pay line between jobs that do have market data.

How often should I change surveys?

Review them annually and change only when the scorecard says so. Swapping vendors every year breaks your trend lines, so switch when coverage or participants clearly fall short.

How do I handle survey data from different effective dates?

Age each source to a common date using a documented growth rate before you blend them. Our aging market data guide walks through the calculation.

Is it a problem if my competitors do not participate in the survey?

It is a gap, not a deal breaker. Ask the vendor whether your key competitors submit data, and add a second source or a custom cut if too many are missing.

By Justin Hampton, founder and CEO of PointFactors.