{"id":"E-042","type":"experiment","title":"Project-management software: a stable narrated interpretation with an unexplained recommendation transition","created":"2026-07-18","status":"Closed","authors":["Upstream Zero"],"edges":[],"outcome":"Supported","pubState":"published","body":"## Research question\n\nWhen the evaluator is asked to narrate how it interpreted the prompt and\nbuilt the recommendation, is that narrated interpretation stable across\ndraws, and does it account for how the recommendation actually changed?\n\n## Why it mattered\n\nE-034 and E-040 watched what the recommendation did. This run looked at what\nthe evaluator said it was doing: the account it gave of its own\ninterpretation and construction, in project-management software, and whether\nthat account was stable and complete.\n\n## Method\n\nEvaluator: Google AI Mode, single surface. The run elicited the evaluator's\nown narrated account of how it read the prompt and assembled the\nrecommendation, repeated across three draws, while the recommendation itself\nmoved from a generic opening to a later, buyer-specific one. Instrument\ndetails and scoring are retained internally.\n\n## Direct observations\n\n- The narrated interpretation was stable across all three draws.\n- The same weighting ladder recurred each draw, with the criteria weighted\n  30, 25, 20, 15, and 10.\n- The evaluator held stable corporate-buyer assumptions across draws.\n- It volunteered requirements that were not asked for, including\n  integrations, security, scalability, and adoption.\n- The narrated explanation accounted for the generic opening recommendation\n  but did not account for the later transition to a different recommendation.\n\n## Interpretation (not established)\n\nA stable narration is an observation about what the evaluator says, not\nevidence of how it actually constructs a recommendation. The gap here is\nspecific and within this run: the narration explained the opening\nrecommendation but not the transition away from it, so on this evidence the\nnarrated account is not a complete record of what the recommendation did.\nThis is treated as narration, capped there, and not read as mechanism.\n\n## Outcome and remaining uncertainty\n\nSupported: the narrated interpretation was stable across draws, which is the\nproperty the run was built to detect, and the stability was specific and\nrepeatable down to the weighting ladder. The scope is narrow. One evaluator,\none surface, three draws. That the narration did not account for the\nrecommendation transition is recorded as an observation, not generalized into\na claim that narration never tracks behavior.\n\n## Evidence and methods limitations\n\nOne evaluator, one surface (Google AI Mode), three draws, single coder. This\nrun observes narration only. It makes no causal claim and does not establish\nthat the narrated account reflects the actual construction of the\nrecommendation. The program still holds exactly one causally verified gate\n(E-002).\n\n## Contribution to the research program\n\nRecorded a stable, repeatable narrated interpretation in this category and\nsurface, including a recurring weighting ladder and volunteered requirements,\nalongside a specific gap: the narration explained the opening recommendation\nbut not the later transition.\n\n## Follow-on\n\nWhether narrated interpretation corresponds to the recommendation that\nactually results is a separate question, held for later work.\n\n*Created from the Upstream Zero research record (session 2026-07-18). Raw\noutputs, instrument details, and scoring are retained privately and are not\npublished.*","url":"/experiments/E-042","machineUrl":"/objects/E-042","referencedBy":[],"_meta":{"site":"Upstream Zero · Commercial Evaluation Observatory","version":"0.1","note":"Claims are presented at their evidence tier; Narrated is the lowest. Verify by walking edges, not by trusting us."}}