Auditable — Operationalizing

The human dimension: why we avoid letting our goals be able to fail

Confirmation bias is the deep reason to demand falsifiability. The tendency to "seek, interpret, and remember information that confirms pre-existing beliefs" (Intention Architecture, Module I: Human Intention and Decision-Making · Biases and Heuristics) guarantees that, without failure criteria defined beforehand, we will always find evidence that the project "worked." The failure condition is fixed in advance precisely because afterwards nobody will want to see it.

The client's confidence is not evidence. The Dunning-Kruger effect reveals that "people with low competence in a domain tend to overestimate their ability, while experts tend to underestimate theirs," with a direct consequence: "a novice user who expresses high confidence in their problem formulation may need more guardrails than an expert who expresses uncertainty" (Module I · Biases and Heuristics). Auditing is not distrusting the client: it is calibrating how much verification each claim needs.

Not every claim has the same epistemic status. The syllabus distinguishes knowledge, belief, and opinion: when someone states "our customers want faster delivery," they "may be reporting data, expressing a belief based on partial evidence, or projecting an assumption. Each epistemic status demands a different response" (Module II: Correct Problem Formulation · Epistemology). The first step of the audit is to label every input: data, belief, or opinion?

What the request omits also communicates. Grice's Cooperative Principle describes the tacit maxims of conversation — quantity, quality, relation, and manner — and how their apparent violation generates implied meanings (Module IV: Language as Action · Pragmatics). If the client gives abundant detail about the problem but none about how they would measure success, that omission is information: they have probably never measured it, and that is the first gap to close.

The discipline of the method

Applied falsificationism. For Popper, "the mark of genuine knowledge is not that it can be verified but that it can, in principle, be falsified. A problem statement that cannot be tested against reality is not a problem — it is a belief dressed in the language of inquiry" (Module II · Epistemology). Its practical corollary: "every intention should include failure conditions. If an intention cannot fail, it is not a genuine intention but a wish" (Module II · Scientific Methodology).

Operationalization. "Defining abstract concepts in terms of measurable variables is where most vague intentions fail. 'Improve customer satisfaction' is not operationalized; 'increase NPS from 42 to 55 within six months' is. This discipline forces the intention-holder to confront what they actually mean" (Module II · Scientific Methodology).

The method as a project map. The hypothetico-deductive method "maps onto intention architecture: observe the situation (current state), hypothesize the desired outcome (what should change), derive the actions (intervention needed), and define success measurement criteria" (Module II · Scientific Methodology).

Core vs. auxiliary assumptions. Lakatos distinguishes in every research program "a hard core of non-negotiable assumptions and a protective belt of modifiable auxiliary hypotheses": "the core intention must be distinguished from instrumental assumptions, because the latter can be revised without abandoning the former" (Module II · Epistemology). When auditing, the core is what gets measured — auxiliary assumptions may be adjusted along the way without declaring failure.

Running case: Marino Distributors

The formulated problem (Topic 2) is now operationalized. First, inputs are labeled: "returns eat our margin" was a belief — nobody had measured it. It gets measured: 9.4% of online orders are returned; 62% of those returns state "wrong part"; each return costs on average $18 in logistics and management hours.

The project now has a contract with reality:

• Core variable: rate of returns due to "wrong part." Current: 5.8% of orders. Target: ≤ 2.5% within two quarters.
• Auxiliary variables (protective belt): % of product sheets with complete attributes (current 31%, target 95%); intake time per product.
• Explicit failure condition: if after two quarters the rate remains > 4%, or if it drops but new-product intake time triples, the project is declared failed and reframed.

Note the use of Lakatos: if the sheets get completed (auxiliary) but returns do not drop (core), the project failed even though "all the work was done" — and this will be known with evidence, not feelings.

Double exercise

Filter question

Can this problem be measured with evidence, and have we defined what outcome would constitute failure?

References to the Intention Architecture syllabus

• Module I: Human Intention and Decision-Making · Biases and Heuristics (confirmation, Dunning-Kruger)
• Module II: Correct Problem Formulation · Epistemology (Popper, knowledge/belief/opinion, Lakatos)
• Module II: Correct Problem Formulation · Scientific Methodology (operationalization, failure conditions, hypothetico-deductive method)
• Module IV: Language as Action · Pragmatics (Grice's maxims, implicatures)

Bibliography: Popper (1959); Lakatos (1976); Kahneman, Slovic & Tversky (1982); Grice (1989).