concept
Recommendation stability
Whether the same inputs produce the same recommendation, and whether unstable outcomes trace to unstable requirement association.
- Question
- Are recommendation outcomes stable across repeated runs, and when they are not, is the instability in the recommendation or upstream in how requirements are associated?
Research components
Answers business questions
- Why did we disappear after follow-up questions?
- What must become true to survive evaluation?
- How do we measure whether our position improved?
- How do AI systems evaluate and recommend vendors?
Recommendation stability studies whether the same question and the same requirements produce the same recommendation across repeated runs. When outcomes move, the instability may sit in the recommendation itself or upstream in how the evaluator associates a requirement with a vendor.
The core question is how stable recommendation outcomes are, and where any instability originates. What remains open is how much variation is normal and what separates unstable requirement association from an unstable recommendation.
Evidence in this component
What we have run, and what we have seen.
Experiments
Observations
- Candidate · 2026-07-20Survival appeared in two modes, and their cause is confounded
Related components
Relevant business questions
- How do we measure whether our position improved?
This object references
- part-of → recommendation-survivability
- observes → requirements
Referenced by
- field-vs-membership-distinction (part-of)
- E-042 (part-of)
- E-045 (part-of)
Authors
Upstream Zero
Machine rendering