PROCUREMENT
MSSP evaluation checklist for engineering teams running procurement
An MSSP evaluation settles four things: capability, response authority, data handling, and exit. Every other question hangs off one of the four. How to choose a managed security service provider is mostly a matter of getting specific answers on those axes before comparing price.
Two questions are missing from the widely ranked checklists on this topic. Who holds official first-line support for the underlying platform, and whether the provider changes production configuration at all or forwards technical work to the vendor.
Those two separate an operator from a billing intermediary, and neither appears on the checklists reviewed for this page.
The four things an evaluation has to settle
A long question list without a spine produces a long interview and no decision. Four axes carry the weight, and each has a recognizable good answer and a recognizable bad one.
| Axis | What it settles | A good answer contains | A weak answer sounds like |
|---|---|---|---|
| Capability | Whether the provider operates production or advises about it | A named change they would make in week one, and its tradeoff | A capability list and a partner badge |
| Response authority | Whether mitigation waits for your sign-off | A pre-agreed action scope and a named approver | “We follow your incident process” |
| Data handling | Where telemetry lives and who holds tuned logic | Jurisdiction, retention period, and export format | “Your data is always yours” |
| Exit | What migration costs when you leave | A documented handoff path, timeline, and price | “We have never had a client leave” |
Response authority and timing sit inside the contract, so that axis resolves into what the service-level agreement has to define. The provider landscape and delivery models give the market context that precedes any of this.
Questions to ask an MSSP that cannot be answered from a brochure
A provider that operates production answers in specifics and names tradeoffs. A provider that does not answers in categories. These six questions are hard to rehearse because each requires having read the client’s actual configuration.
- Which rules would you change in our current configuration in week one, and what does each change cost us?
- Who holds first-line support for the platform itself, and what do you add on top of that?
- Walk us through your last incident escalation, stage by stage, without naming the client.
- Which of our detection surfaces would you decline to cover, and why?
- Where are our logs stored, under which retention period, and in which jurisdiction?
- On termination, who transfers configuration and credentials, how long does it take, and what does it cost?
Question four does most of the work. A provider willing to name what it will not cover has read the estate. A provider that claims full coverage of everything has read the request for proposal.
Question two is covered in its own section below, because the answer is frequently wrong in a way that is easy to miss.
Before any of this, establish which delivery model the proposal describes, since two proposals with different category labels are not comparable until their scope is written on the same axes.
Operator or licence reseller
A share of providers in this market are billing intermediaries holding a vendor partner badge. A partner badge indicates a commercial relationship, not a technical capability. Neither of the two most visible checklists on this topic offers a test that separates the two, Meriplex’s 15 questions nor the msspproviders.io buyer checklist.
| Test | Operator | Intermediary |
|---|---|---|
| Asked to name a week-one configuration change | Names one and its tradeoff | Offers an assessment first |
| Asked who performs production changes | Their own engineers, under a change process | The client, or the platform vendor |
| Asked a configuration-level technical question | Answers it, including the limits | Routes it to the vendor |
| Asked what happens when vendor support declines a request | Describes a workaround they have built | Describes escalating harder |
None of these tests is about credentials. MSSP due diligence that stops at certifications and headcount will pass an intermediary with a good deck, because certifications attest to process rather than to depth on a specific platform.
An intermediary is not automatically the wrong choice. Where the client team holds its own engineering capability and needs commercial access rather than operational help, the arrangement is coherent. It is only a problem when it is bought as something else.
Who holds first-line support for the platform
Where a platform vendor provides official first-line support for its own service, a provider adds customer-side expert support on top of it and does not replace it. Neither of the checklists cited above addresses this, and the distinction changes who a client calls at the worst possible moment.
A provider claiming to replace vendor support has misdescribed the arrangement.
That misstatement is useful. It appears early, it is easy to check against the vendor’s own published support terms, and it disqualifies without a technical assessment.
Two things are being bought, and they are frequently conflated in a proposal. Official platform support, which comes from the vendor under the service contract, and expert support on the client’s side of the relationship, which is what the provider adds.
A single point of contact is a coordination promise, not a replacement of vendor support. It is a real and valuable thing to buy, and it is worth confirming which of the two the proposal is describing.
The check is quick. Read the platform vendor’s published support terms, then ask the provider to state where its own responsibility begins.
Data residency, retention, and who holds the detection logic
Client telemetry remains the client’s data under the terms signed. Two details decide whether that ownership is usable: the export format, and where the tuned detection logic lives.
A rule set exported in a proprietary format satisfies an ownership clause and cannot be loaded anywhere else. Ask for a named format, not a right.
For platforms operating in the European Union, jurisdiction and retention period are also data-protection questions rather than only commercial ones. Those decisions belong with qualified counsel.
Exit and handoff terms
Exit terms predict conduct during the contract. A provider confident in its work documents the handoff path before signature, including timeline and cost.
Vague exit language usually means configuration is being held as a retention tactic. It is one of the few procurement signals available before any work starts.
Agents and tooling licences deployed inside the client estate need a stated disposition at termination. Without one, the estate keeps components nobody is contracted to maintain.
What a certification proves and what it does not
ISO/IEC 27001 certifies the provider’s own information security management system. SOC 2 Type II attests to the operation of the provider’s controls over a period. Neither attests to the security of any individual client deployment.
Both of the checklists cited above list these as trust signals without stating the boundary, and Fortra’s expert survey on hiring an MSSP treats certification as a provider characteristic in the same way. The boundary is what makes them useful: a certification is a procurement filter that narrows a longlist, and it is not a guarantee about the estate being protected.
The evidence worth asking for sits behind the certificate. The scope statement, which names what the certification actually covers, and the most recent audit period.
A provider whose certification scope excludes the service being purchased has a valid certificate and an irrelevant one.
What to ask the references
Every checklist advises asking for references and stops there. A reference call yields signal only when it asks about a specific past event rather than about satisfaction.
- Describe the last incident they handled for you. Who acted, and how long did it take?
- Describe the last time they missed something. What changed afterwards?
- Has an invoice ever surprised you, and what caused it?
- What would you change about the scope if you were signing again?
- Who is your named contact, and how often has that person changed?
A reference unwilling to discuss a miss is a curated reference. That is worth knowing, and it is information about the provider rather than about the referee.
Red flags that disqualify
Each of these is a mechanism rather than a matter of taste. Any one of them is enough to stop an evaluation.
-
A quote before reading the traffic profile
Pricing issued without seeing the estate is a template applied to a logo.
Why it mattersThe scope will be renegotiated after signature, from a weaker position
-
A claim to replace platform vendor support
Official first-line support comes from the vendor under the service contract.
Why it mattersThe arrangement has been misdescribed before any work has started
-
No limits named anywhere
A provider that covers everything has not scoped anything.
Why it mattersGaps surface during an incident rather than during procurement
-
No documented handoff path
Undocumented exit converts configuration knowledge into a dependency.
Why it mattersMigration cost becomes unbounded and unpredictable
-
Ingestion pricing left undefined
Log volume grows with traffic, domains, and every new telemetry source.
Why it mattersOverage is usually the largest unbudgeted line in year one
Scoring the shortlist
Normalize every proposal onto the four axes before comparing price, because the categories vendors use are not comparable to each other. MSSP RFP questions that follow a vendor-supplied scorecard inherit that vendor’s weighting.
Weighting belongs to the buyer. A platform losing money to automated traffic weights capability and response authority heavily and treats breadth of service lines as close to irrelevant.
A proposal that leaves an axis blank has answered it. Record the blank as the answer and score accordingly.
What the evaluation should end with
A completed MSSP evaluation checklist produces four written answers per provider, not a preference. Capability, response authority, data handling, and exit, each stated specifically enough to hold the provider to later.
Certifications, references, and red flags narrow the list. The four axes decide it.
Put the two missing questions first.
Who holds first-line platform support, and who performs production changes. Providers differ sharply in how directly they answer both.
Frequently asked questions
What should an MSSP evaluation checklist cover?
Four axes: capability, response authority, data handling, and exit. Certifications, references, and pricing detail hang off those four. A checklist without that spine produces a long interview and no decision.
How do I tell whether a provider actually does the work?
Ask who performs production configuration changes, and ask the provider to name one change it would make in week one. An operator names the change and its tradeoff. An intermediary offers an assessment or routes the question to the platform vendor.
Does a vendor partner badge prove technical capability?
No. A partner badge indicates a commercial relationship with the platform vendor. Technical depth shows up only in configuration-level answers, which is why those questions belong early in an evaluation.
Does hiring an ISO/IEC 27001 certified provider make our platform compliant?
No. ISO/IEC 27001 certifies the provider’s own information security management system, and SOC 2 Type II attests to its control operation. Neither attests to your deployment. Ask for the certification scope statement and the audit period.
Who should I call first when the security platform itself breaks?
Where the platform vendor provides official first-line support, the vendor. A provider adds customer-side expert support on top of that and coordinates. A provider claiming to replace vendor support has misdescribed the arrangement.
How many providers should be in an evaluation?
Enough that the four axes produce contrast, which in practice means three to five. Beyond that, the questions repeat and the answers stop differentiating, and the evaluation cost rises without improving the decision.