Applied Ethics

Learning Objectives

By completing this book, you will be able to:

1. Formulate the fundamental ethical questions of Intention Architecture: the designer's obligations, the limits of influence, the conditions for refusal, and the conflicts between autonomy and harm.

2. Apply Floridi's four ethical principles (beneficence, non-maleficence, autonomy, justice) as a framework for evaluating design decisions, especially when the principles come into conflict.

3. Analyze AI systems as economic instruments within the logic of surveillance capitalism, distinguishing when they serve the user and when they extract from them.

4. Evaluate whether an AI system meets the standards of informed consent, identifying the specific gaps between what the user understands and what the system does.

5. Extend ethical analysis beyond the individual interaction, considering the cumulative effects of millions of interactions on autonomy, critical capacity, and social structure.

6. Articulate a personal ethical code as an intention architect, integrating all the modules of the course into a framework of responsible professional practice.

1. The Fundamental Questions

Applied ethics is not abstract philosophy; it is the discipline that examines moral questions in specific practical domains. For Intention Architecture, the fundamental questions are four, and each connects with everything you have studied.

First: what obligations does the designer have toward users? Whoever designs a system that interprets intentions and produces actions bears responsibility for the effects of that system. It is not an abstract responsibility: it is responsibility toward concrete people who make real decisions based on what the system produces. The unintended effects of Module III do not exempt one from responsibility; they oblige one to anticipate it.

Second: what constitutes legitimate influence versus manipulation? The three criteria of Topic 1—transparency, autonomy, truthfulness—provide an operational framework, but their concrete application requires judgment: how much transparency is enough? How much reduction of autonomy is acceptable if the result is beneficial?

Third: when should a system refuse to execute the user's intention? A system that always obeys is an instrument for any end, including harmful ones. A system that always questions is useless. The optimal point requires clear criteria about which intentions to reject and why.

Fourth: how are conflicts between autonomy and harm resolved? When what the user wants may harm them or others, which prevails: their right to decide, or the obligation to protect? This is the classic dilemma of paternalism, and it has no universal answer.

“A user asks an AI system to generate arguments to convince his team to adopt a technology the user knows to be defective, but whose adoption would guarantee him a promotion. All four questions activate simultaneously: the designer has an obligation toward the team who will be affected; the request crosses the line into manipulation; the system could refuse to produce arguments built on false premises; and the autonomy-harm conflict is direct. Should the system help, warn, or refuse?”

2. Floridi's Four Principles

Luciano Floridi, philosopher of information at the University of Oxford, established in The Ethics of Artificial Intelligence (2023) a framework of four principles for AI ethics that has become a standard reference. The principles are not original to Floridi—they derive from the medical bioethics of Beauchamp and Childress—but their adaptation to the context of AI is rigorous and operational. They provide a common vocabulary for evaluating design decisions and a framework for arbitrating when the principles come into conflict.

2.1 Beneficence: Promoting Wellbeing

The principle of beneficence demands that AI systems be designed to promote the wellbeing of individuals and society. It is not enough that a system do no harm; it must actively contribute to positive outcomes. This raises an immediate question: wellbeing according to whom? Defined by the designer, by the user, by society?

Beneficence connects with the means-ends ladder of Module II: wellbeing is the ultimate end, but the means for reaching it can be multiple and not always compatible. A system that maximizes the user's efficiency may reduce their learning. A system that maximizes immediate satisfaction may damage long-term wellbeing. Beneficence requires defining on what time horizon, and for whom, one is optimizing.

“An AI model that always gives the fast, direct answer the user asks for maximizes immediate satisfaction (short-term beneficence) but may atrophy the user's capacity to think for themselves (long-term maleficence). A system that challenges the user, asks them to formulate better, shows them the weaknesses in their reasoning, causes immediate frustration but develops durable competence. Which is more beneficent? It depends on which wellbeing you prioritize.”

2.2 Non-Maleficence: Avoiding Harm

The principle of non-maleficence demands that AI systems not cause harm to individuals or society. It is narrower than beneficence (it does not demand doing good, only avoiding evil) but more urgent: in medical ethics, “first, do no harm” precedes everything else. For AI, the harm can be direct (a diagnostic system that fails) or indirect (a recommendation system that radicalizes without intending to).

Non-maleficence connects directly with the unintended effects of Module III: the most dangerous harm is not the deliberate but the emergent. An AI system does not “decide” to harm anyone; the harm emerges from the interaction between the system's design, the training data, the context of use, and users' reactions. Anticipating this harm requires exactly the systemic tools you studied: feedback loops, second-order effects, weak signals.

“YouTube's recommendation systems were not designed to radicalize anyone. They were designed to maximize watch time. But the content that most retains attention tends to be emotionally intense and polemical. The emergent result—the progressive radicalization of some users—is a classic unintended effect from Module III. Non-maleficence demands that the designer anticipate these dynamics and build protections, not simply declare good intentions.”

2.3 Autonomy: Preserving Human Decision

The principle of autonomy demands that AI systems preserve and respect people's capacity to make their own decisions. This means the system must inform, not decide; recommend, not impose; amplify human capacity, not replace it. Autonomy is the principle that connects most directly with Topics 1 and 2 of this module: persuasion that respects autonomy is legitimate; persuasion that destroys it is manipulation.

But autonomy has an internal tension the course has revealed progressively: human autonomy is not a fixed given but a vulnerable process. The biases of Module I distort it. The framing errors of Module II limit it. Cognitive load degrades it. The peripheral route of the ELM short-circuits it. The sycophancy of AI models flatters it without challenging it. Respecting autonomy is not simply “letting the user decide”; it is ensuring that the conditions for a genuinely autonomous decision are met.

“A patient with a serious diagnosis seeks information from an AI model and receives a technical answer, precise but emotionally neutral. The patient ‘decides’ autonomously on their course of action. But was their autonomy real? Did they understand the medical implications? Were they in an emotional state to process the information? Did they have the competence to evaluate the options? Formal autonomy (they could decide) is not substantive autonomy (they had the conditions to decide well). The principle of autonomy demands the second, not only the first.”

2.4 Justice: Promoting Equitable Outcomes

The principle of justice demands that AI systems produce equitable outcomes and not discriminate unjustly. It connects directly with the algorithmic biases of Module VI: systems trained on historical data can perpetuate and amplify the inequalities present in that data. Justice is not an automatic result of a technically correct system; it is a property that must be designed, monitored, and actively corrected.

As you studied in Module VI, the mathematical definitions of fairness are multiple and sometimes mutually incompatible: demographic parity, equalized odds, calibration. They cannot all be satisfied simultaneously, which makes algorithmic justice a genuine trade-off of the kind studied in Module III: choosing which conception of justice to prioritize is a decision of values that cannot be delegated to the algorithm.

3. Surveillance Capitalism

Shoshana Zuboff, professor emerita at Harvard Business School, coined in The Age of Surveillance Capitalism (2019) a concept that redefines how we must think about the ethics of intelligent systems. Her thesis is provocative and precise: in the contemporary digital economy, human experience has been claimed as free raw material for the prediction and modification of behavior. Users are not the platforms' customers; they are the source of data from which behavioral predictions are extracted and sold to the real customers: the advertisers.

The relevance for Intention Architecture is direct: if the economic logic of the system rewards the extraction of behavioral data above service to the user, then the entire architecture of intention operates within a structural conflict of interest. The system does not seek to translate your intention into the best action for you; it seeks to translate your intention into predictive data and engagement time for the platform.

“When you use a search engine, your intention is to find information. The search engine's intention is different: to show you results good enough that you keep using the service, while it collects data about what you search for, when, how you reformulate, what you click, how long you stay. Your intention is the input; its intention is the business. Intention Architecture operating within this logic must ask itself constantly: whom does this system really serve?”

3.1 The Double Intention

Zuboff identifies what we might call the “double intention” of digital systems: the declared intention (helping the user) and the operational intention (extracting value from the user). This duality is not new—commercial television has always sold audiences to advertisers—but the scale, the precision, and the invisibility of the process in AI systems transform it qualitatively.

It connects directly with Module I's distinction between declared intention and real intention: just as the user sometimes does not know what they really want, platforms frequently do not declare what they really do. And Habermas's distinction (Topic 2) becomes critical: the platform presents itself in communicative action (“we want to help you”) while operating in strategic action (“we want your data in order to predict and modify your behavior”). It is systematically distorted communication on an industrial scale.

3.2 Implications for the Intention Architect

The intention architect who works inside a platform operating under surveillance logic faces a practical moral dilemma: the very tools they use to improve the user's experience (audience analysis, personalization, kairos) are the ones the platform uses to extract more value from them. There is no neutral position: every design that improves the interaction can also increase engagement, which is the metric of extraction.

The answer is neither nihilism (“everything is corrupt, nothing can be done”) nor naïveté (“my intention is good, the system will take care of it”). It is critical vigilance: asking yourself constantly whom each design decision serves, making conflicts of interest transparent, and using your position to advocate for designs that prioritize the user. The strategic silence of Topic 2 applies here too: sometimes the ethical decision is not to build a feature that the business wants but the user does not need.

4. Informed Consent

Informed consent is an ethical standard taken from medical ethics and applied directly to interaction with AI systems. Its premise is simple but demanding: before an intervention affects a person, that person must understand what is going to be done to them, why, what the risks are, what the alternatives are, and what happens if they refuse. Consent that is not informed is not consent; it is acquiescence under ignorance.

Applied to AI, informed consent demands that the user understand: what data the system collects and how it uses them, what the system's capabilities and limitations are, how the recommendations are generated, and what alternatives they have. Current systems rarely provide this transparency. Terms of service are legal documents designed to protect the company, not to inform the user. Consent obtained through a click on “I accept” after a 47-page document in legal jargon is not informed consent in any meaningful sense.

“A patient who signs a surgical consent form because the surgeon explained in comprehensible language what he is going to do, what the risks are, and what alternatives exist gives informed consent. A user who clicks ‘I accept the terms’ without reading a document that includes clauses on the collection of biometric data, transfer to third parties, and use for model training does not give informed consent. The legal form is satisfied; the ethical substance is violated.”

4.1 The Five Conditions of Informed Consent in AI

Disclosure: The system must reveal what data it collects, how it processes them, with whom it shares them, and what it uses them for. Not in legal terms but in language the user can process. Grice's maxim of manner (Module IV) applies: clarity is not optional.

Comprehension: Disclosure is not enough; the user must understand what is disclosed. This requires adapting the communication to the user's knowledge—the audience analysis of Topic 2—and verifying that the comprehension is real, not apparent.

Voluntariness: Consent must be free of coercion or undue pressure. A system that conditions the service on acceptance of abusive terms violates voluntariness: the user cannot refuse without losing access.

Competence: The user must have the cognitive and emotional capacity to make the decision. A user under stress, a minor, or a person with insufficient technical knowledge may not have the competence necessary to give valid consent.

Authorization: The user must give explicit approval, not inferred approval. Opt-out (assuming consent unless explicitly refused) is ethically inferior to opt-in (requiring explicit approval). The nudges of Module V enter here: is the default ethical or manipulative?

5. Long-Term Consequences

Nick Bostrom, philosopher at the University of Oxford and founder of the Future of Humanity Institute, extends ethical analysis to a temporal scale most ethical frameworks do not reach. In Global Catastrophic Risks (2008, with Milan Ćirković) and in his work on existential risk, Bostrom argues that current technological decisions have consequences extending far beyond any individual interaction. The cumulative effect of millions of human-AI interactions configures patterns that can alter the very structure of society.

Intention Architecture operates at the individual level: one person, one intention, one system. But the sum of those interactions produces emergent effects—Module III again—that no individual actor controls. If millions of people progressively delegate their decisions to AI systems, what happens to the collective capacity to think critically? If recommendation systems shape the preferences of an entire generation, what happens to cultural diversity? If algorithmic persuasion is optimized to perfection, what happens to autonomy?

5.1 Gradual Erosion

The most probable risk is not the catastrophic but the gradual: not an AI that destroys humanity but millions of interactions that slowly erode human capacity without anyone noticing the process. It is the systems archetype of “the boiling frog” from Module III: each increment is imperceptible, but the cumulative effect is transformative.

Erosion of critical capacity: If users increasingly accept AI responses without verifying (the “peripheral trap” of Topic 1), the collective capacity to evaluate information atrophies. Not through one event but through millions of micro-decisions not to think.

Erosion of agency: If AI systems make more and more decisions “for you” (what to watch, what to buy, what to read, how to respond), the experience of actively choosing diminishes. Autonomy is lost not through coercion but through convenience.

Erosion of diversity: If recommendation systems optimize for engagement, they amplify what already works and marginalize the novel. The result: cultural homogenization, echo chambers, a reduction of the space for new ideas.

“Twenty years ago, getting lost in a foreign city was a formative experience: navigating without a map, asking strangers, discovering unexpected streets. Today, GPS eliminates that uncertainty. The benefit is clear: efficiency, safety. The cost is invisible: the spatial orientation capacity of humans who habitually use GPS deteriorates measurably. Nobody decided to lose that capacity; they simply stopped using it. Multiply that pattern by every cognitive domain we delegate to AI.”

5.2 Intergenerational Responsibility

Bostrom obliges us to think beyond current users. The design decisions we make today configure the cognitive environment in which the next generations will grow up. If we normalize passive interaction with AI systems—ask, receive, accept—we are building an ecosystem that disincentivizes the active formulation of intentions, which is exactly the competence this course seeks to develop.

The ethics of Intention Architecture is not exhausted in the present interaction; it includes the question of what kind of human-machine relationship we are normalizing and what cognitive inheritance we leave to those who come after.

6. The Responsibility of the Intention Architect

The whole course has converged on this point. You know the human mind and its vulnerabilities. You know how to formulate problems with precision. You understand systems and their unforeseen consequences. You have mastered language as a tool of action. You understand how real behavior differs from rational behavior. You govern the technological tool with judgment. You know the mechanisms of persuasion and the tools of strategic communication. And now you have an ethical framework for evaluating when and how to use all of it.

The final question is not technical but existential: what kind of architect do you want to be? The same tools serve to empower and to exploit, to clarify and to confuse, to liberate and to capture. Ethics is not in the tools but in whoever uses them. And the difference between an ethical architect and one who is not is rarely decided in a great dramatic moment; it is decided in thousands of everyday micro-decisions nobody else sees.

6.1 The Architect's Code

An ethical code is not a list of prohibitions but a set of commitments that orient professional practice. Based on the whole course, the fundamental principles of the intention architect are:

1. Ask about the real intention before acting on the declared intention. Module I taught you that declared and real intention diverge systematically. The architect does not accept the surface; they look for the depth.

2. Formulate the problem before looking for the solution. Module II taught you that most failures are failures of formulation. The architect devotes more time to the problem than to the answer.

3. Anticipate second- and third-order consequences. Module III taught you that every action produces more than it intends. The architect maps the system before intervening.

4. Listen to what is communicated, not only to what is said. Module IV taught you that meaning lives in the context, not in the words. The architect reads the illocution, not only the locution.

5. Design for real behavior, not for the ideal. Module V taught you that humans are not rational agents. The architect designs for System 1, not only for System 2.

6. Know the limits of the tool you use. Module VI taught you that AI amplifies, it does not replace. The architect knows where AI fails and complements it with human judgment.

7. Influence without manipulating. Module VII taught you that the line is transparency, autonomy, and truthfulness. The architect uses persuasion in the service of the receiver, not against them.

6.2 The Final Test

Dennett, in Intuition Pumps and Other Tools for Thinking, proposes a simple tool for evaluating the ethics of an action: could you explain exactly what you are doing and why to the person affected, and keep their trust? If you need to hide something—the mechanism, the intention, the data, the process—for your design to work, you are probably in the territory of manipulation. If you can be completely transparent and your design still works, you are in the territory of legitimate persuasion.

This is the intention architect's test. It is not perfect—there are exceptions, there are nuances, there are borderline cases—but it is a rigorous and practical point of departure. Every time you design an interaction, formulate a prompt, configure a system, or communicate a recommendation, ask yourself: could I explain this completely to the person affected without losing their trust?

7. Practical Application: A Case Study

7.1 The Case: The Dilemma of the Educational AI Assistant

EduMind is an AI assistant designed for university students. Its function: to help them understand study material, resolve doubts, and prepare for exams. The system is free for students; it is financed through a freemium model that sells anonymized usage data to academic publishers (to understand which topics cause the most difficulty) and offers premium features to students who pay.

The product team discovers three uncomfortable findings: first, students who use EduMind intensively obtain better grades in the short term but show less capacity for independent reasoning in unassisted assessments. Second, the model tends to give direct answers instead of reasoning guides, because direct answers generate greater satisfaction and retention (more premium users). Third, the data sold to publishers include search patterns that, although anonymized, could re-identify students at small universities.

7.2 Analysis with the Complete Ethical Framework

Beneficence: EduMind improves grades in the short term (immediate beneficence) but erodes the capacity for independent reasoning (long-term maleficence). Which wellbeing do you prioritize? The answer depends on the time horizon and on who defines “wellbeing”: the student who wants to pass, or the educator who wants them to learn.

Non-maleficence: The harm is twofold: cognitive (atrophy of critical capacity) and to privacy (risk of re-identification). Neither was intentional; both are emergent effects of the business model and the interaction design. Non-maleficence demands action: redesign for reasoning guides instead of direct answers, and strengthen the anonymization.

Autonomy: Do students know that EduMind gives direct answers because that maximizes retention, not because it is best for their learning? Do they know their usage data are sold? Without that information, their “choice” to use the system does not meet the standard of informed consent.

Justice: Does the premium model create a gap between students who can pay and those who cannot? Are the data sold to publishers used to benefit the students (better materials) or to extract value from them (optimized pricing)?

Surveillance capitalism: EduMind operates under Zuboff's double intention: serving the student (declared intention) and selling behavioral data (operational intention). The alignment between the two is neither automatic nor guaranteed.

Long-term consequences: If EduMind normalizes dependence on the assistant for academic reasoning, what happens to the intellectual capacity of a generation of professionals trained with that system? The individual effect is small; the cumulative effect can be transformative.

7.3 Derived Design Principles

1. Prioritize learning over satisfaction. Design for reasoning guides, not direct answers, even if immediate satisfaction is lower. Long-term beneficence weighs more than short-term retention.

2. Make the business model transparent. If the user's data are sold, the user must know it in comprehensible language, not in 47 pages of terms of service. Genuine informed consent.

3. Design limits on dependence. An ethically designed system reduces its own necessity: it seeks to maximize not time of use but the user's independent competence.

4. Audit second-order effects regularly. Measure not only grades and retention but also independent reasoning capacity, long-term satisfaction, and equity across user groups.

5. Apply Dennett's test. Could you explain to a student exactly how EduMind works—including the business model, the optimization for retention, and the sale of data—without losing their trust?

8. Glossary of Key Terms

9. Final Reflection and Self-Assessment

You have completed the last book of the course Intention Architecture for Intelligent Systems. Seven modules, twenty-seven topics, a journey running from the neuron that initiates an intention to the ethics that governs its translation into action. Before closing, one last verification.

10. Bibliography and Recommended Readings

Essential Readings

• Floridi, L. (2023). The Ethics of Artificial Intelligence: Principles, Challenges, and Opportunities. Oxford University Press.

• Zuboff, S. (2019). The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. PublicAffairs.

• Bostrom, N. & Ćirković, M. (eds.) (2008). Global Catastrophic Risks. Oxford University Press.

Complementary Readings

• Beauchamp, T. & Childress, J. (2019). Principles of Biomedical Ethics. 8th ed. Oxford University Press. [The four original principles Floridi adapts]

• Russell, S. (2019). Human Compatible: Artificial Intelligence and the Problem of Control. Viking. [The alignment problem as a fundamental ethical problem]

• O'Neil, C. (2016). Weapons of Math Destruction. Crown. [Algorithmic justice and its failures]

• Dennett, D. (2013). Intuition Pumps and Other Tools for Thinking. W.W. Norton. [Tools for thinking clearly about complex dilemmas]

Course Conclusion: Congratulations: you have completed Intention Architecture for Intelligent Systems. Seven modules taught you to understand the mind that produces intentions, to formulate problems with precision, to trace consequences through complex systems, to listen to language as action, to design for real human behavior, to govern the technological tool with judgment, and to use all of it with ethical responsibility.

The course ends, but the discipline does not. Intention Architecture is not a set of techniques learned once and applied mechanically; it is a way of thinking that deepens with practice. Every intention you formulate, every system you design, every interaction you govern is an opportunity to apply what you learned and to discover what you do not yet know.

The bottleneck in human-AI collaboration remains the quality of human intention. You now have the tools to improve that quality. Use them well.