{"id":"M-4","type":"method","title":"Ingestion & consolidation for the requirement-led observatory","created":"2026-07-14","status":"frozen — Method v1.0 (2026-07-14)","authors":["Upstream Zero"],"edges":[],"pubState":"published","body":"**Method v1.0 — FROZEN 2026-07-14.** Future changes require observed\nevidence, protocol failure, or demonstrated contradiction — recorded as\nversioned revisions. Changelog: v0.1 (question-centered draft) → v0.2\n(rebuilt around the requirement spine) → v0.3 (falsification\nconsolidation, findings F-2/F-4/F-5/F-6 applied) → **v1.0 (frozen)**.\nAll prior drafts are preserved in the public git history.\n\n## 0. The ingestion path\n\nRaw Source → Observed Expression → Business Problem / Buying Role\nassignment → Requirement or Uncertainty assignment → source class and\nresearch confidence → Canonical-question consolidation →\nEvaluator-environment observation → Revision / split / merge. Stages\napply as evidence permits; this is the full model, not a mandatory\nconveyor.\n\n## 1. Observed expressions — admissibility\n\nVerbatim interaction artifacts only: raw prompts, search queries, buyer\nwordings, evaluator follow-ups, interview statements. Mandatory capture:\nverbatim text, timestamp, evaluator-environment (EE-), instrument\nversion (I-), source method, consent state for human sources.\nInadmissible: hearsay, recollected paraphrase, expressions invented to\nillustrate, model output generated on request as \"typical questions.\"\n\n## 2. Type-specific `observed` tests (one shared vocabulary)\n\nOne sourceClass vocabulary — observed · inferred · research-derived ·\neditorial — with object-specific admissibility:\n\n- **Business problem — observed:** directly stated or clearly documented\n  in admissible source material.\n- **Requirement — observed:** directly stated as a condition, criterion,\n  mandate, preference, or constraint (RFP, security questionnaire,\n  procurement checklist, direct buyer statement). Evaluator-asserted\n  requirements are recorded as narration-derived (C-0002): evidence that\n  the *evaluator applies* the requirement, not that buyers hold it.\n- **Gate behavior — observed:** behaviorally demonstrated through an\n  evaluator observation under protocol, a controlled test (flip test),\n  or a real procurement outcome.\n- **Canonical question — observed-commercial:** supporting observed\n  expressions consolidated under this method.\n- **Inferred (any type):** derived through this method with supporting\n  expressions preserved and a `provenance-shared` marker naming the\n  generating cluster.\n\nEditorial objects carry visible labels and never enter market-coverage\ndenominators.\n\n## 3. Raw permanence\n\nExpressions and requirement artifacts are immutable, content-hashed,\npermanently preserved; PII redaction on capture with declared\nredactions; no downstream step edits, deletes, or overwrites the raw\nlayer. Splits, merges, and re-classifications re-point edges only.\nCanonicalization never destroys observations.\n\n## 4. Assignment\n\nEach expression is assigned a business problem and, where determinable,\na buying role — with written rationale; `unassigned` where genuinely\nindeterminate, never guessed. Requirement-or-uncertainty assignment:\nobserved requirement if §2 evidence exists; inferred requirement if the\ncluster meets the pilot thresholds (§16); **otherwise the expression\nstops at an uncertainty attached to the business problem** — a\nrequirement is never retrofitted. The distinguisher: a requirement is a\nfalsifiable statement about what a solution must satisfy; an uncertainty\nis an open question about the problem or world.\n\n## 5. Assignment failure is evidence, not just noise (binding)\n\nCoder disagreement, assignment difficulty, unresolved multi-requirement\nambiguity, unassigned-pool growth, repeated requirement splits, repeated\nmerges, and instability across coders, markets, or evaluator\nenvironments are recorded **both** as instrument-health data **and** as\nobservations relevant to H-3. The two interpretations — instrument noise\nvs. hypothesis failure — remain visible as competing explanations on\nevery affected report until evidence distinguishes them.\n\n**Review triggers (thresholds set as pilot parameters, §16):** sustained\nbreach of any trigger opens a recorded review in which assignment\nfailure is weighed as evidence against, respectively: requirement\nidentifiability; requirement stability; the requirement/uncertainty\ndistinction; and the assumption that requirements are the durable\norganizing object. The architecture is not permitted to explain away its\nown refutation.\n\n## 6. Inferred requirements\n\nCreated only when: the cluster meets support thresholds (§16); the\nrequirement statement is entailed by the cluster (no scope beyond what\nthe expressions imply); method, coder, and rationale recorded;\n`provenance-shared` marker attached. **Anti-double-counting:** evidence\nshared between an inferred requirement and its cluster's canonical\nquestions counts once in any metric; stability or breadth claims for\ninferred requirements require evidence independent of the founding\ncluster (§16 defines independence).\n\n## 7. Requirement formation\n\nFormation — an uncertainty crystallizing into a requirement — is\nrecorded when time-separated evidence shows the transition; the\nrequirement enters `emerging` with a `formed-from` edge and the dated\nevidence. Formation events are publishable observations.\n\n## 8. Canonical-question consolidation\n\nA CQ is consolidated as a projection of its assigned\nrequirement-or-uncertainty × buying role × buyer stage × intent. The\nresolution test: two expressions share a cluster only if a complete\nanswer to one would fully resolve the other, for the same role at the\nsame stage. Default-to-separate under doubt. Canonical wording is\nentailed by its expressions — never beyond them; wording is versioned;\nmultilingual expression sets record the working language of the\ncanonical wording (§16.8). Competing canonicalizations persist as\n`contested` objects linked by `contests` edges; contests resolve only by\nnew evidence or a documented methodological ruling, and the disagreement\nhistory survives resolution.\n\n## 9. Ambiguity and multiplicity\n\nAn expression may evidence multiple requirements (edges with rationale;\nevidence weight divides, never duplicates). Genuinely unresolvable\nexpressions live in the visible `unassigned` pool — ambiguity is data\n(§5) and may itself evidence an unformed requirement.\n\n## 10. Evaluator-environment observation\n\nEvery expression and observation carries EE- and I- stamps; evaluator\nconfidence records are typed `expressed | behavioral`; distribution\nstatistics are computed from stamps, never asserted. Gate instances are\ncreated only from observed gate behavior under a declared protocol —\nflip tests (vary one requirement's satisfaction; observe\nrecommendation-set change), firing/inactivity across matched scenarios,\nsaturation checks, stage and role variation — or exist under an honest\nnon-observed source class per the gate policy parameter (§16.6).\n\n## 11. Baselines and exposure labeling\n\nBefore a market's requirement map or coverage is published: baseline\nevaluator observations captured; protocols and evaluator versions\nfrozen; raw outputs preserved; dates and sampling conditions recorded.\nEvery subsequent observation carries an exposure label:\n`pre-publication baseline` · `post-publication observation` ·\n`potentially exposed` · `exposure status unknown`. H-3 claims cite\nbaselines or declare their absence. Client Zero participation is\nmeasured, never hidden.\n\n## 12. Coverage denominators\n\nEvery coverage claim names its map version and validity date; historical\nmeasurements remain valid against their original map versions; history\nis never silently recalculated under a new denominator.\n\n## 13. Revision, split, and merge\n\nSplits and merges of requirements, gates, uncertainties, and CQs are\nRevision objects with reasons and impact; IDs never reused; superseded\nobjects remain published with forward pointers; expressions re-point;\nclass changes are revisions carrying their new evidence. Split/merge\nfrequency feeds §5.\n\n## 14. Safeguards against editorial invention\n\nThe unified class discipline with type-specific tests (§2); entailment\nrules (canonical wording ≤ expressions; inferred requirements ≤\nclusters); the audit trail (every derived object resolves to verbatim\nraw evidence or a declared gap); periodic laundering checks sampling\nderived objects for drift, with drift findings published as revisions;\n`provenance-shared` markers keeping the inference loop visible.\n\n## 15. Replication and coder discipline\n\nIndependent re-coding at the target agreement rate (§16.2); single-coder\nperiods declared on every affected object, with time-separated\nself-agreement substituting, honestly labeled; agreement rates published\nas instrument health *and* as §5 data.\n\n## 16. Pilot parameters — declared before the first pilot begins\n\nFrozen as parameters, not blockers; each must be declared and recorded\nbefore the pilot starts:\n\n1. N / M support thresholds — per confidence grade, for\n   observed-commercial CQ admission, and for inferred-requirement\n   creation.\n2. Coder-agreement targets and re-coding cadence.\n3. §5 review-trigger thresholds (disagreement rate, unassigned-pool\n   share, split/merge frequency).\n4. Redaction and consent policy details for human-derived material.\n5. Search-frequency admission gates.\n6. Gate-instance evidence policy (minimum observed behaviors).\n7. Independence criterion for the anti-double-counting rule.\n8. Working-language policy for canonical wording.","url":"/methods/M-4","machineUrl":"/objects/M-4","referencedBy":[],"_meta":{"site":"Upstream Zero — Commercial Evaluation Observatory","version":"0.1","note":"Claims are presented at their evidence tier; Narrated is the lowest. Verify by walking edges, not by trusting us."}}