Criteria for Selecting Off-the-Shelf Software: A Scorecard That Survives the Demo

23 October 20265 min readAshton

A magnifying glass over a row of plain software boxes on a conveyor, one box marked with a gold check

TL;DR: Selecting off-the-shelf software goes wrong when the demo sets the criteria. Decide the criteria first and weight them in writing: fit to your actual process, total cost over three years including your own time, integration with what you already run, the vendor's health and support behaviour, and how you leave if it fails. Then score every candidate, including the incumbent and the option of buying nothing, against the same card.

Key Takeaways

Selecting off-the-shelf software goes wrong when the demo sets the criteria.
Start with the failure pattern you can verify in your own records: Criteria invented during demos: Each vendor's strengths become the requirements, and the comparison measures salesmanship.
Work in sequence: Write the process the software must support, in your words, with the exceptions that occur monthly, not just the happy path. Then: Set and weight the criteria in writing before any demo: process fit, total three year cost, integration, support, vendor health, and exit.
Judge the result with your own numbers: Written criteria with weights exist before demos, and the same card scores every candidate.

Software selection usually starts with three demos and ends with whichever salesperson explained the company's own process back most flatteringly. The scorecard in this guide reverses the order: criteria and weights first, evidence second, demos last and scripted by you. It is the process we use when a service business asks whether a package will hold, before it becomes the system everyone works around.

What criteria actually decide whether packaged software works?

The order matters here. These problems tend to arrive together, but they do not cost the same, and fixing the visible one first usually means fixing it twice.

Criteria invented during demos. Each vendor's strengths become the requirements, and the comparison measures salesmanship.
Licence cost mistaken for total cost. Configuration, migration, training, and the workaround hours never appear in the quote.
Integration checked by assertion. The product integrates, in the sense that a file can be exported and a person can retype it.
Exit never discussed. Data export, notice periods, and what breaks on departure surface only when departure is already painful.

What does the selection process look like, step by step?

Write the process the software must support, in your words, with the exceptions that occur monthly, not just the happy path.
Set and weight the criteria in writing before any demo: process fit, total three year cost, integration, support, vendor health, and exit.
Script the demo: the vendor runs your scenarios with your sample data, including an exception and a failure.
Trial the finalist on real work with the real users, and score the trial against the same card.
Record the decision and the scores, so a future switch starts from evidence instead of archaeology.

If you are already part way down this list, start from the first step you cannot honestly check off. Systems fail at their earliest missing step more often than at their latest one, and repairs made downstream of a gap rarely hold.

What belongs in a total cost comparison?

Illustrative example only: two products tie on process fit, but one needs a staff member exporting and retyping data daily. At even 20 minutes a day, that hidden labour outweighs a modest licence difference within a year. The figures are illustrative; time your own integration test to get real ones.

The mistakes that most often undo this work:

Letting the most senior person's preference substitute for user trials. The daily user finds what the buyer cannot.
Scoring features as present or absent with no quality judgement. A present feature your team refuses to use scores zero in practice.
Comparing candidates bought at different times against moving criteria. Freeze the card for the whole selection.

How do you run the selection so the demo does not run you?

You know this is working when you can check the following against your own records, without asking anyone to prepare a report:

Written criteria with weights exist before demos, and the same card scores every candidate.
Total cost includes licences, setup, migration, training, and your own workaround time.
Exit terms are documented. data export format, notice period, and what happens to records on departure.

Second Brain builds and runs systems like these for service businesses, and the engagement always starts the same way: find where the work is actually being lost, in what order, and what evidence would prove a fix worked. If this guide's diagnosis matches your operation, that audit is the honest next step. If it does not match, you have saved yourself a purchase, which is also a good outcome.

FAQ

How many criteria should the scorecard have?

Five to seven, weighted. More than that and everything matters equally, which means nothing does. Process fit and total cost should almost always lead for a service business.

Should we score doing nothing as a candidate?

Yes. The current way, however creaky, is the baseline every purchase must beat on total cost and risk. Some selections honestly end there, and that is a successful process, not a failed one.

How do we judge vendor health?

Look for evidence: how long the product has existed, how it communicates changes and outages, whether support answers during your trial with substance, and how existing customers describe leaving or staying.

What if the team disagrees on the winner?

Return to the weighted card and the trial evidence. If the disagreement is about criteria, you are having the right argument late; record it, reweight openly, and rescore once.

Sources

Keyword basis: mined queue (SE Ranking, pulled 2026-10-23, US, desktop) entry "criteria for selecting off the shelf software", ESTIMATED volume 10, ESTIMATED difficulty 10, intent informational. Mined figures are estimates, unvalidated against our own GSC and PostHog data, and no mined figure is claimed as a result in this post.
Internal link targets: custom-vs-off-the-shelf-software, commercial-off-the-shelf-software-examples, off-the-shelf-software-cost (live, staged, or drafted in this same program, checked 2026-10-23).
No external statistics, market prices, or client claims are made in this post. Worked figures are labelled illustrative in the text, and guidance is stated as general practice from implementation and operations work.