Incentives

Learning Objectives

By completing this book, you will be able to:

1. Explain why the classical view of incentives as simple levers of behavior is insufficient, and how behavioral economics reveals their real complexity.

2. Identify and anticipate the effect of motivational crowding out, by which extrinsic incentives destroy intrinsic motivation.

3. Distinguish between material, social, and moral incentives, and recognize in which contexts each type is most effective and in which it is counterproductive.

4. Apply Goodhart's Law to diagnose how systems of metrics and incentives corrupt the very behavior they claim to measure and reward.

5. Analyze how temporal discounting interacts with incentives to produce systematic misalignment between short-term rewards and long-term objectives.

6. Design incentive systems for interaction with AI that align the user's real behavior with their declared intention, anticipating perverse effects and motivational crowding out.

1. The Hidden Complexity of Incentives

Incentives are, in their most basic definition, rewards and penalties that shape behavior. Classical economics treats them as direct mechanisms: offer more reward for X, and you will get more X; penalize Y, and you will get less Y. It is a clean, elegant, and profoundly incomplete logic. Behavioral economics has demonstrated that the relationship between incentives and behavior is non-linear, context-dependent, and frequently paradoxical.

“A daycare center in Haifa, Israel, had a problem: parents were arriving late to collect their children. The obvious solution was to implement a fine for lateness. The result was exactly the opposite of what was expected: lateness increased. Before the fine, parents felt a moral obligation to be punctual—arriving late meant taking advantage of the caregivers. After the fine, parents reinterpreted the situation as an economic transaction: they were paying for the extra service of extended care. The fine did not add a cost; it replaced a social norm with a market norm. And when the daycare center eliminated the fine, lateness did not return to its original level. The moral norm had already been destroyed.”

This case, documented by Uri Gneezy and Aldo Rustichini, is emblematic because it demonstrates three principles the classical view cannot explain. First, incentives are not added to the existing context; they transform it. The fine was not added to the moral obligation; it replaced it. Second, the transformation is frequently irreversible: once a social norm becomes a market norm, eliminating the price does not restore the social norm. Third, the same incentive has opposite effects depending on the normative framework in which it operates.

For Intention Architecture, this means that every incentive you design—in an AI system, in an organization, in a policy—operates within an ecosystem of pre-existing norms. Before asking “what incentive should I implement,” you must ask “what norms are operating now, and how will this incentive transform them?” It is a second-order question (Module III), and omitting it is the source of most perverse incentives.

2. Motivational Crowding Out: When Paying Destroys

Motivational crowding out is the phenomenon by which extrinsic incentives reduce or destroy intrinsic motivation. You studied it briefly in Module I as the overjustification effect. Now you will examine it with the depth its impact on system design requires.

Bruno Frey, the Swiss economist, formalized the concept by distinguishing two mechanisms. The first is the perceived control effect: when a reward or punishment is perceived as controlling—as an attempt to manipulate behavior—people experience a reduction in autonomy, which undermines intrinsic motivation (this connects directly with Deci and Ryan's Self-Determination Theory, which you studied in Module I). The second is the signaling effect: the very existence of the incentive sends a signal about the nature of the task. If you are paid to do something, “it must be because it is not inherently valuable.” The reward communicates that the activity is a cost, not a benefit.

“A software company has developers who voluntarily contribute to an internal open-source repository out of professional satisfaction and technical pride. Management, wanting to ‘scale’ this practice, introduces a bonus of 500 euros per accepted contribution. High-quality contributions—the ones developers made out of passion—decline. In their place, trivial contributions designed to collect the bonus increase. The monetary incentive transformed an act of professional mastery into an administrative formality. The best developers, the ones who contributed out of intrinsic motivation, were the first to withdraw.”

Crowding out is not universal—it depends critically on the type of task and the type of incentive. Tasks involving creativity, complex judgment, or professional identity are the most vulnerable. Routine, mechanical, or unpleasant tasks—where there is no intrinsic motivation to displace—respond well to extrinsic incentives. The practical rule: the more interesting or meaningful the task, the more dangerous the monetary incentive.

2.1 Conditions of Crowding Out

Research has identified the conditions that maximize crowding out. The incentive is perceived as controlling (not as recognition). The task already generates intrinsic satisfaction. The incentive is contingent on a specific result (not on participation). The amount is large enough to reframe the activity as an economic transaction, but not large enough to compensate for the loss of intrinsic motivation. There exists a middle range—the “valley of death” of incentives—where the payment is too small to motivate extrinsically but sufficient to destroy intrinsic motivation.

“A consultant charges 3,000 euros a day. When a friend asks for professional help and offers him 200 euros, the consultant is offended. If the friend had offered nothing, the consultant would have helped gladly—out of friendship, a social norm. The 200 euros activated the market norm (where the value of his work is much greater) and deactivated the social norm (where helping a friend has no price). This is the valley of death: too little to be fair payment, enough to destroy the social framework.”

2.2 Crowding In: When the Incentive Strengthens

Crowding out is not the only possibility. Frey also identified crowding in: conditions under which extrinsic incentives reinforce intrinsic motivation instead of destroying it. This occurs when the incentive is perceived as recognition (not as control), when it confirms the individual's competence (rather than questioning it), and when it is unexpected or not contingent on a specific result.

“A researcher receives an unexpected prize for an article she published out of genuine intellectual interest. The prize does not undermine her motivation—it strengthens it. It confirms that her work has value for the community, it satisfies the need for competence (Deci and Ryan), and it does not reframe the activity as a transaction. Compare this with a system that pays 50 euros per published article: now the researcher calculates how many articles she needs to supplement her salary, and quality falls in cascade.”

The critical distinction is between incentives that inform and incentives that control. An informative incentive says “what you did has value”; a controlling incentive says “do this in order to get that.” The first strengthens autonomy; the second erodes it. For Intention Architecture, this distinction determines how an AI system should structure its feedback: feedback that recognizes the quality of the user's thinking strengthens their motivation; feedback that gamifies the interaction with points and rewards probably degrades it.

3. Social Incentives: The Force of the Group

Cialdini documented six principles of influence that are, in essence, six categories of social incentive: reciprocity, commitment and consistency, social proof, authority, liking, and scarcity. What classical economics treats as secondary factors, behavioral research demonstrates to be frequently the primary engines of behavior.

Social proof—the tendency to do what others do—is perhaps the most powerful social incentive and the most relevant for Intention Architecture. People do not decide in a vacuum; they decide by watching what others whom they perceive as similar to themselves are doing. Cialdini demonstrated this in the celebrated hotel towel experiment: a sign saying “most guests in this room reuse their towels” was significantly more effective than standard environmental messages. It was not the logical argument that changed behavior; it was knowing what others in the same situation were doing.

“An online learning platform discovers that showing the user ‘73% of people in your role complete this course in 3 weeks’ increases the completion rate 22% more than offering a free certificate on finishing. The certificate is a material incentive; the social comparison is a social incentive. In this context, knowing what your peers are doing is more motivating than receiving a tangible reward. The social incentive operates through identity: you do not want to be one of those who do not finish.”

3.1 Reciprocity as an Engine of Action

Reciprocity—the obligation to return what another has given you—is one of the oldest and most robust social mechanisms in the human species. Cialdini demonstrated that it operates even when the initial ‘gift’ is small, unsolicited, and comes from someone you do not like. The rule is automatic: if you give me something, I feel the obligation to give you something in return. It is an incentive that requires neither money nor authority—only a generous action.

For interaction with AI, reciprocity has a direct application: a system that offers something valuable first—an unsolicited analysis, a question that reveals an angle the user had not considered, a reformulation that improves the original request—generates in the user a willingness to invest more effort in the interaction. Reciprocity converts the transactional interaction (“give me what I ask for”) into a collaborative one (“we are both investing in the result”).

“An AI system that, before answering a vague request, offers: ‘Before I answer, I noticed your question could be interpreted two ways. Do you mean X or Y?’ is offering a free act of clarification—a cognitive ‘gift.’ The user, activated by reciprocity, tends to respond with more detail and precision than they would have offered spontaneously. The quality of the intention formulated improves not through demand but through reciprocity.”

3.2 The Limits of Social Incentives

Social incentives are not a panacea. They have important limits the intention architect must know. Social proof can produce informational cascades: if the majority does something incorrect, social proof reinforces the error. Reciprocity can be exploited manipulatively (crossing the ethical line you will study in Module VII). Social pressure can generate conformity at the expense of the quality of thinking—exactly the opposite of what Intention Architecture seeks.

The point of equilibrium is: use social incentives to facilitate behaviors the user already wants to perform but which inertia, procrastination, or lack of information prevent. Do not use them to push the user toward behaviors not aligned with their real intention. The difference between facilitation and manipulation is alignment with the user's authentic intention.

4. Perverse Incentives and Goodhart's Law

A perverse incentive is one whose structure of rewards produces exactly the opposite of what it intends. It is not a failure of implementation but a failure of design: the internal logic of the incentive contains the seeds of its own subversion. You already studied the cobra effect and Goodhart's Law in Module III as second-order causal phenomena. Now you will examine them as specific problems in the design of incentives.

Goodhart's Law—“when a measure becomes a target, it ceases to be a good measure”—operates through a specific mechanism: on fixing an indicator as a goal, people optimize for the indicator, not for the underlying result the indicator was trying to capture. The indicator was a proxy—a useful but imperfect approximation. On converting it into an objective, the proxy divorces itself from what it represented.

“A hospital measures the quality of care by the average waiting time in the emergency department. Doctors, incentivized to reduce this indicator, begin classifying the most complex patients as ‘non-urgent’ in order to remove them from the fast queue. The average waiting time falls—the indicator improves. But complex patients now wait longer, in a queue with no priority. The real quality of care worsens while the metric celebrates success. The indicator no longer measures what it claims to measure.”

4.1 Mechanisms of Metric Corruption

The corruption of metrics follows predictable patterns. The first is edge optimization: finding ways to satisfy the metric without satisfying the objective. The second is gaming: manipulating the inputs of the metric to alter its result. The third is the distortion of attention: concentrating all effort on what is measured at the expense of what is not measured but matters. Each mechanism is a rational response of the agent to the incentive—what is irrational is the design of the incentive, not the behavior of the agent.

“An AI platform measures user satisfaction with a thumbs-up/thumbs-down after each response. The product team optimizes to maximize the thumbs-up. Result: the model learns to give answers that sound good and confirm what the user wants to hear, instead of answers that are correct but uncomfortable. The satisfaction metric rises; the real utility of the system falls. The user is satisfied with answers that do not serve them. This is Goodhart's Law applied to the design of AI.”

For Intention Architecture, Goodhart's Law is an existential warning. Every time you define a metric of success for an intention—and every well-formulated intention must have criteria of success (Module II)—you are creating an incentive. And every incentive is susceptible to Goodhart. The solution is not to stop measuring (that is worse) but to measure multiple indicators that cannot be simultaneously optimized in a perverse way, and to be—always—willing to change the metrics when they show signs of corruption.

4.2 Defense Against Goodhart

There are strategies for mitigating Goodhart's Law, although none eliminates it completely. The first is to use batteries of metrics instead of single ones: if you simultaneously measure speed, quality, and client satisfaction, it is harder to optimize one at the expense of the others. The second is to rotate metrics: periodically changing what is measured prevents agents from building stable gaming systems. The third is to measure outcomes, not outputs: not how many reports the team produced but which decisions improved thanks to those reports. The fourth—perhaps the most important—is to monitor the unmeasured metrics: what is getting worse that nobody is measuring?

“An AI development team measures model accuracy, response time, and user satisfaction. But nobody measures the quality of the questions users formulate. Result: the model becomes excellent at answering mediocre questions, and users never improve their capacity to formulate. The unmeasured metric—the quality of the user's intention—degrades silently while everything else appears to be going well.”

5. Temporal Discounting and Long-Term Incentives

Temporal discounting—the systematic preference for immediate rewards over future ones—is the natural enemy of long-term incentives. You studied it in Topic 1 of this module as the intention-behavior gap: the conflict between the planner self and the doer self. Now you will examine it from the perspective of the incentive designer: how do you design rewards that operate in the future when the user's brain discounts the future automatically?

The problem is structural. Most valuable results—health, knowledge, relationships, professional competence—require sustained effort with a deferred reward. Destructive results—instant gratification, procrastination, shortcuts in quality—offer an immediate reward with a deferred cost. Temporal discounting guarantees that, in the absence of deliberate design, short-term incentives will win.

“A professional who uses AI in her work has two options: (a) invest 10 minutes formulating her request precisely, which will produce an excellent answer that saves hours of subsequent work; or (b) write a vague request in 30 seconds, obtain a mediocre answer, and spend an hour correcting it manually. Option (a) is objectively superior. But temporal discounting discounts the 10 minutes of investment against the 30 seconds of option (b). At the moment of decision, the 30 seconds win. Every time.”

5.1 Strategies of Temporal Design

There are four strategies for counteracting temporal discounting in the design of incentives. The first is to bring the reward closer: instead of promising a distant benefit, provide immediate feedback that anticipates the future value. The second is to fragment the cost: divide the required effort into small steps whose immediate cost is tolerable. The third is to make the cost of the short term visible: show what is lost with the fast option, not only what is gained with the patient one. The fourth is to pre-commit: use commitment devices (Topic 1) so that the decision to invest is made once, in a moment of clarity, and executes automatically thereafter.

“An AI tool that wants to incentivize precise formulations implements all four strategies simultaneously. Bring the reward closer: after each precise formulation, it shows the data point ‘your last precise request produced an answer you used without editing; your vague requests required an average of 3 iterations.’ Fragment the cost: it offers a template of three questions that takes 60 seconds to complete, not 10 minutes of free writing. Make the cost visible: ‘With your current formulation, I estimate a 40% probability that you'll need to reformulate. Do you want to invest 60 seconds now or take the risk?’ Pre-commit: it allows the user to activate a ‘guided formulation’ mode that operates automatically.”

6. Designing Incentives for Interaction with AI

Everything above converges on a practical question: how do you design the incentive ecosystem of an AI system so that it aligns the user's real behavior with their declared intention? The answer requires integrating the lessons of the five previous topics.

The first step is to map the existing incentives. When a user interacts with an AI system, they are already operating within an ecosystem of incentives: those of their organization (what is rewarded and what is punished), those of their temporal context (how much urgency they face), those of their professional identity (what it means to be good at what they do), and those of the system itself (how easy it is to take the fast path versus the precise one). Intervening without mapping these pre-existing incentives is a recipe for perverse effects.

“A sales team uses AI to generate commercial proposals. The organizational incentive is the number of proposals sent per week. The temporal incentive is the urgency of closing the quarter. The system's incentive is the ease of generating generic proposals with one click. The predictable result: mediocre proposals generated without reflection proliferate, the conversion rate falls, and the AI becomes a high-speed corporate spam machine. All the incentives push in the same direction: quantity over quality, speed over precision.”

Five design principles emerge from integrating the behavioral economics of incentives with Intention Architecture:

1. Map before intervening. Before designing any incentive, map the complete ecosystem: which norms are operating (market, social, moral)? What intrinsic motivation exists? What metrics are measured? What temporal discounting is at work? Without this map, any intervention is blind.

2. Preserve intrinsic motivation. If the user has intrinsic motivation to formulate their intentions well (curiosity, professionalism, the desire to solve real problems), protect it. Do not replace it with gamification, points, or rankings that displace it. Use recognition and information, not control.

3. Align incentives with intention, not with outputs. Do not incentivize the quantity of interactions with the AI, nor the speed of the response. Incentivize the quality of the formulation, the depth of the analysis, and the impact of the result. Measure outcomes: which decisions improved? Which problems were solved?

4. Anticipate Goodhart before it happens. For each metric you define, ask: “If someone wanted to maximize this metric without caring about the real result, how would they do it?” The answer will tell you exactly where your incentive system will fail.

5. Design for the long term with short-term tools. Use immediate feedback to make the value of careful formulation visible. Fragment the effort to lower the barrier. Pre-commit the user in their moments of clarity. But never forget that the objective is to develop the user's competence, not to create dependence on the system.

7. Practical Application: The Case of the Innovation Program

TechForward is a technology company with 2,000 employees launching an internal innovation program. The declared objective: that employees use AI tools to identify opportunities for improvement in their areas and formulate innovation proposals. The initial design includes a monthly prize of 5,000 euros for the best proposal, a public ranking of ‘innovators of the month,’ and a participation metric (number of proposals per department). Let us apply the complete incentive analysis.

7.1 Incentive Diagnosis

The monetary prize of 5,000 euros operates under a market norm. For employees with salaries of 60,000–120,000 euros a year, the prize is not negligible but neither is it transformative. It sits in an ambiguous zone: enough to activate the market norm (this is a competition, not a voluntary contribution) but not enough to compensate for the hours of work a serious proposal requires. Risk of crowding out: employees who already innovate out of intrinsic motivation may reinterpret innovation as a ‘paid extra task’ and calculate whether it is ‘worth it.’

The public ranking is a social incentive: status and recognition. But combined with the monetary prize, it produces a contradictory signal: do you innovate for money or for recognition? If the ranking is public but the prize is private, the social dimension is strengthened. If both are public, competition eclipses collaboration—and real innovation usually requires collaboration.

The participation metric (number of proposals per department) is an open invitation to Goodhart. Predicting the behavior is trivial: departments will generate superficial proposals to meet the quota. Quantity will rise; quality will fall. The AI system will be used to generate proposals rapidly—exactly the opposite of using AI to think better.

“After three months, TechForward has 450 proposals, most generated in under 20 minutes with AI, few with real analysis. The five best ‘innovators’ are people with free time who learned to write impressive proposals quickly, not people with deep insights into the real problems. The technical staff with the greatest knowledge of the actual problems do not participate—they consider the program ‘a writing contest, not an innovation contest.’ The program was a success on the metric and a failure on the objective.”

7.2 A Redesign Based on Incentive Theory

Eliminate the individual prize and create an implementation budget. Instead of 5,000 euros to the ‘winner,’ allocate 50,000 euros per quarter to implement the best proposals. The incentive is no longer to win a prize but to see your idea implemented—a far more powerful intrinsic incentive that activates the sense of agency and impact.

Replace the competitive ranking with narrative recognition. Instead of the ‘innovator of the month,’ publish innovation stories: what problem the person detected, how they analyzed it, how they used AI to go deeper, and what impact it had. The social incentive is recognition of competence (crowding in), not competition for status (crowding out).

Measure the quality of the analysis, not the quantity of proposals. The anti-Goodhart counter-indicator: not how many proposals a department generates but how many were implemented, and what measurable impact the implemented ones produced. An outcome metric, not an output metric.

Use AI as a tool of depth, not of speed. Configure the AI interaction so that, before generating a proposal, it guides the user through an analysis: what is the real problem? What evidence supports it? What second-order effects could the solution have? What trade-offs does it involve? AI as Socratic interlocutor, not as copywriter.

Design temporal incentives. Immediate feedback: after each interaction with the analysis AI, the user receives an assessment of the depth of their formulation. Fragmentation: the proposal is built over three 20-minute sessions, not as one document in a single sitting. Pre-commitment: the user enrolls in a 3-week ‘innovation cycle’ with weekly checkpoints, creating structure and social commitment with their group.

7.3 Derived Design Principles

Principle 1: Before designing an incentive, map the motivation that already exists and protect it. The most expensive incentive in the world does not compensate for the destruction of intrinsic motivation.

Principle 2: Every incentive transforms the normative framework. Introduce money into a social context and you will convert a contribution into a transaction. Sometimes this is desirable; frequently it is catastrophic.

Principle 3: Measure outcomes, not outputs. How many proposals says nothing. How many were implemented says something. What impact the implemented ones had says everything.

Principle 4: Goodhart's Law is inevitable but mitigable. Use batteries of metrics, rotate indicators, and—above all—ask constantly what is getting worse that nobody is measuring.

Principle 5: The best incentive is the one that disappears—the one that builds competence and intrinsic motivation until the behavior sustains itself, with no need of external reward.

8. Glossary of Key Terms

9. Final Reflection and Self-Assessment

Before moving on to the next topic (Framing), verify that you can design and diagnose incentive systems with behavioral rigor, not with naïve intuition.

10. Bibliography and Recommended Readings

Essential Readings

• Cialdini, R. (1984). Influence: The Psychology of Persuasion. William Morrow.

• Thaler, R. & Sunstein, C. (2008). Nudge: Improving Decisions About Health, Wealth, and Happiness. Yale University Press.

• Ariely, D. (2008). Predictably Irrational. HarperCollins.

Complementary Readings

• Gneezy, U. & Rustichini, A. (2000). “A Fine Is a Price.” Journal of Legal Studies, 29(1), 1–17.

• Frey, B. & Jegen, R. (2001). “Motivation Crowding Theory.” Journal of Economic Surveys, 15(5), 589–611.

• Sunstein, C. (2016). The Ethics of Influence: Government in the Age of Behavioral Science. Cambridge University Press.

• Deci, E.L. & Ryan, R.M. (2000). “The ‘What’ and ‘Why’ of Goal Pursuits.” Psychological Inquiry, 11(4), 227–268.

Next Topic: The next book of this module addresses Framing: how the presentation of information systematically shapes decisions. You will study with Thaler, Sunstein, and Kahneman attribute framing, goal framing, and risky choice framing. You will connect behavioral framing with the framing errors of Module II and with the speech acts of Module IV in order to build a complete understanding of how what we say—and how we say it—determines what people do.