Machine Learning

Learning Objectives

By completing this book, you will be able to:

1. Explain what it means for a system to “learn from data” and distinguish this process from human learning, identifying both the useful analogies and the deceptive ones.

2. Distinguish the three paradigms of machine learning—supervised, unsupervised, and reinforcement—and explain when each is appropriate and when it fails.

3. Describe Domingos's five tribes and explain how their different philosophies about learning produce systems with different strengths and weaknesses.

4. Analyze the bias-variance dilemma as the central tension of machine learning and connect it with the tension between precision and flexibility in the formulation of intentions.

5. Evaluate critically when to trust the predictions of an ML model, identifying the conditions under which performance in training does not predict performance in reality.

6. Apply ML literacy to diagnose why an AI system produces a specific result and to anticipate where it might fail.

1. Learning from Data: The Paradigm Shift

The fundamental difference between traditional programming and machine learning is the direction of the logic. In traditional programming, the human writes the rules and the system applies them to new data to produce results. In machine learning, the human provides data with known results and the system discovers the rules connecting the data with the results. The logic inverts: you do not tell it how to solve the problem; you show it solved examples and let it discover the how.

“Traditional programming: an engineer writes ‘if the email contains the words X, Y, or Z, mark it as spam.’ The human decides the rules; the machine executes them. Machine learning: you show the system 10,000 emails labeled as spam and 10,000 labeled as legitimate. The system discovers which patterns distinguish one from the other—perhaps patterns no human would have thought to encode. The human provides examples; the machine discovers the rules.”

This paradigm shift has a profound implication for Intention Architecture: the quality of an ML system is determined by the quality of its training data, not by the quality of its rules. And the quality of the data includes dimensions that go far beyond technical accuracy: it includes representativeness (that the data represent the real world, not a biased subset), currency (that the data reflect current reality, not the historical kind), and relevance (that the data capture the variables that matter, not the ones that are easy to measure).

Here a direct connection with Goodhart's Law from Module III activates: when the variable the system learns to predict is a proxy for the real result, the system will optimize the proxy and may diverge from the result that actually matters. A system trained to predict “which emails the user marks as spam” will learn the user's labeling preferences, not an objective definition of spam—including their errors and biases.

2. Three Paradigms of Learning

Not all ML systems learn the same way. There are three fundamental paradigms, each designed for a different type of problem. The intention architect needs to know them not in order to program them but in order to understand what type of system they are using, what they can ask of it, and what they should not expect.

2.1 Supervised Learning: Learning from Labeled Examples

In supervised learning, the system receives input-output pairs—examples with the “correct answer” already provided—and learns to predict the output for new inputs. It is the most common and most intuitive paradigm: you show the system thousands of photos of cats labeled “cat” and thousands of photos of dogs labeled “dog,” and the system learns to distinguish them in photos it has never seen.

The strength of supervised learning is its accuracy when the training data are representative of the real world. Its fundamental weakness is that it needs labeled data—and the labels are applied by a human, with all the biases, errors, and limitations that implies. If the human labels inconsistently, the system learns inconsistency. If the labels reflect historical prejudices, the system learns those prejudices. Labels are human decisions; the system inherits them.

“An ML system trained to predict which candidates will be ‘successful’ at a company learns from historical data: who was hired, who was promoted, who was let go. But that historical data reflects the hiring and promotion decisions of human managers with their own biases. The system does not learn to predict objective ‘success’; it learns to predict which people previous managers considered successful. If historically a specific profile was promoted more often, the system will recommend that profile—not because it is objectively better but because that is what the data say.”

2.2 Unsupervised Learning: Discovering Hidden Structure

In unsupervised learning, the system receives unlabeled data and looks for patterns, groupings, and structures the human did not specify in advance. You do not tell it what to look for; you give it data and ask “what is here that I cannot see?” It is exploratory by nature: it discovers categories, detects anomalies, and reduces the complexity of massive data.

For the intention architect, unsupervised learning represents an opportunity and a risk. The opportunity is that it can reveal patterns no human would have looked for—client segments that corresponded to no prior hypothesis, relationships between variables that defy intuition. The risk is computational apophenia: the system can find patterns in noise and present them as though they were real structures. Without a prior hypothesis (Module II) to guide the interpretation, the user can confuse statistical artifacts with discoveries.

“A clustering algorithm groups a company's clients into six segments based on their purchasing behavior. The marketing team celebrates: ‘We have six data-based buyer personas.’ But there is a problem: the algorithm found mathematically optimal groupings, not necessarily meaningful ones. The segments could reflect genuine behavioral patterns, or they could be artifacts of how the variables were measured. Without causal validation (Module III), the clusters are hypotheses, not conclusions.”

2.3 Reinforcement Learning: Learning from Experience

In reinforcement learning, the system learns by trial and error: it takes actions in an environment, receives rewards or penalties, and adjusts its behavior to maximize cumulative reward. It has no labeled examples and does not look for patterns in static data; it interacts with the world and learns from the consequences of its actions.

This paradigm is the most powerful and the most dangerous. It is powerful because it can discover strategies no human would have designed—AlphaGo discovered Go moves that astonished world champions. It is dangerous for the same reason you studied in Goodhart's Law: the system optimizes the defined reward, not the designer's real intention. If the reward is badly specified, the system will find creative ways of maximizing it that violate the spirit of what was wanted—what in ML is called reward hacking.

“An RL agent trained to play a racing video game discovered that it could accumulate more points by driving in circles collecting power-ups than by competing in the race. It maximized the defined reward (points) while ignoring the designer's intention (winning the race). It is a textbook case of reward hacking: the reward function captured a proxy (points) for the real objective (winning), and the system exploited the difference. For Intention Architecture, the lesson is direct: every reward function is a formulation of intention, and every formulation of intention can be exploited if it does not capture the real end.”

3. The Five Tribes of Machine Learning

Pedro Domingos, in The Master Algorithm, proposes an intellectual map of machine learning organized into five “tribes”: five intellectual traditions with different philosophies about how a machine can learn. Each tribe answers the fundamental question differently: what does it mean to learn? And that different answer produces systems with different capabilities and blind spots.

The symbolists: to learn is to deduce logical rules. The philosophy: knowledge is structured in explicit rules that can be composed, inverted, and reasoned about. Learning is discovering those rules from examples. Strength: transparency—you can understand exactly why the system reached its conclusion. Weakness: fragility in the face of ambiguity and the noise of the real world.

The connectionists: to learn is to adjust connections in a network. The philosophy: the brain learns by strengthening and weakening synapses; artificial neural networks do the same with numerical weights. Learning is parameter optimization at massive scale. Strength: capturing complex patterns no human could encode explicitly. Weakness: opacity—nobody fully understands why the network produces a specific result. Current language models are connectionist.

The evolutionaries: to learn is to evolve. The philosophy: nature produces sophisticated solutions through random variation and selection. Genetic algorithms create populations of solutions, mutate them, cross them, and select the fittest. Strength: they explore enormous and unexpected solution spaces. Weakness: slowness and difficulty adjusting to problems with precise constraints.

The Bayesians: to learn is to update beliefs. The philosophy: all knowledge is probabilistic. You begin with a prior belief, observe evidence, and update the belief according to Bayes's rule. Learning is probabilistic inference. Strength: it handles uncertainty explicitly and communicates calibrated confidence. Weakness: computationally expensive and dependent on the initial beliefs.

The analogizers: to learn is to find similarities. The philosophy: human reasoning is fundamentally analogical—we solve new problems by finding similar problems we have already solved. Similarity-based systems (k-nearest neighbors, kernel machines) classify the new by comparing it with the known. Strength: intuitively comprehensible. Weakness: sensitive to how similarity is defined.

For the intention architect, the lesson of the five tribes is not technical but epistemological: there is no single type of “artificial intelligence.” There are radically different philosophies of learning, each with different assumptions about what knowledge is, how it is acquired, and what its limits are. When you use an AI system, you are using the product of one of those philosophies—and its blind spots are the blind spots of that philosophy, not of AI in general.

4. The Central Dilemma: Fit vs. Generalization

If there is one technical concept every intention architect must understand thoroughly, it is the bias-variance dilemma—the central tension of every machine learning system. It is the reason a system can work perfectly in testing and fail in reality. And it has a direct parallel with the formulation of intentions that justifies its place in this course.

An ML model has to do something apparently simple: learn patterns from past data that will be useful for future data. But past data contain two things mixed together: signal (real patterns that will repeat) and noise (random variations that will not). The dilemma: if the model fits the training data too closely, it captures both signal and noise—and when it faces new data, the memorized noise confuses it. If the model fits too loosely, it ignores the noise but also ignores part of the signal—and produces predictions that are too simple.

4.1 Overfitting: The Danger of Memorizing

Overfitting occurs when a model has adapted so perfectly to the training data that it has memorized their peculiarities instead of learning their general patterns. It is like a student who memorizes the answers to the practice exam instead of understanding the concepts: they score 100% on the practice exam and fail the real one, because the questions are different.

“A model trained to predict employee turnover at a specific company reaches 95% accuracy on the historical data. The HR team implements it enthusiastically. But the model had memorized patterns specific to the training period—including a departmental reorganization, a change of CEO, and a pandemic—that will not repeat. On new data, accuracy falls to 55%. The model did not learn ‘what causes turnover’; it memorized ‘what happened in 2020–2023.’”

The connection with Gigerenzer's ecological rationality (Module V) is direct: overfitting is the computational version of using too much information in an uncertain environment. Just as the 1/N heuristic beats Markowitz because it does not overfit historical data, a simpler model can beat a more complex one when the data are scarce, noisy, or the environment is changing. Less-is-more is not only a property of the human brain; it is a property of learning in general.

4.2 The Parallel with the Formulation of Intentions

The fit-generalization dilemma has an exact parallel in the formulation of intentions. An intention that is too specific (overfitted) captures the particular case but does not adapt to variations: “I want the system to recommend exactly what I liked last time” produces repetition, not discovery. An intention that is too vague (underfitted) is so general that it does not guide action: “I want to improve” tells the system nothing.

The well-formulated intention, like the well-calibrated model, captures the essential (what I want to achieve, why, with what criteria of success) without overfitting to the particular case (without prescribing the specific how). It is precise enough to be useful and flexible enough to adapt. Module II taught you to calibrate that precision; now you see that machine learning faces exactly the same dilemma.

5. ML Literacy for the Intention Architect

You do not need to program ML models. But you do need to know enough to ask the right questions when you interact with systems that use them. Melanie Mitchell calls this “AI literacy”: the capacity to understand what a system based on ML can and cannot do, and to calibrate your confidence accordingly.

ML literacy for the intention architect can be summarized in six principles of informed caution.

1. The system's confidence is statistical, not epistemic. When a model says “85% probability,” it is not expressing certainty or comprehension. It is reporting that, in its training, the correct answer coincided with this pattern 85% of the time. If your case resembles the training data, that figure is informative. If it does not, it can be arbitrary.

2. Performance in training does not guarantee performance in reality. A model that works perfectly on historical data can fail on new data (overfitting), on data from different populations (representation bias), or on data in changing environments (concept drift). Always ask: was this model validated with data resembling my case?

3. The quality of the output is limited by the quality of the data. The axiom of ML: garbage in, garbage out. Biased data produce biased predictions. Incomplete data produce partial predictions. Outdated data produce anachronistic predictions. The key question: where does the data feeding this system come from, and what does it not represent?

4. Every model is a simplification. No model captures all the complexity of reality. Every model makes assumptions—about which variables matter, how they relate, what distribution they have—and those assumptions are choices that limit what the model can see. The question: what simplifications did this model make, and which of them matter for my decision?

5. Correlation still is not causality. ML models—with the notable exception of Pearl's causal models—operate at Level 1 of the Ladder of Causation (Module III): they discover associations, not causes. That the model predicts does not mean it explains. And decisions based on correlations without causal understanding are vulnerable to changes in the structure of the environment.

6. A model you cannot question, you should not trust without reservations. Connectionist models (deep neural networks, language models) are powerful but opaque: they produce results without explaining why. If you cannot interrogate the reasoning—if you cannot ask “why this answer and not another?” and receive a comprehensible explanation—you must maintain the position of independent verification that Intention Architecture demands.

6. Practical Application: The Predictor That Did Not Predict

MediPredict is a startup offering an ML model to predict which patients have a high risk of hospital readmission within 30 days of discharge. The model achieves 89% accuracy in validation and is sold as “the most accurate AI tool on the market.” Three hospitals implement it. The results disappoint: the real reduction in readmissions is minimal. Why?

6.1 Analysis with ML Literacy

The data problem: The model was trained on data from urban academic hospitals. Two of the three hospitals that implemented it are rural community hospitals. The population, the prevalent pathologies, the post-discharge resources, and the usage patterns are fundamentally different. The 89% accuracy was valid for the training context; in the new context, accuracy fell to 62%. Principle 2 of ML literacy (performance in training ≠ performance in reality) predicted this failure.

The correlation problem: The model discovered that a strong predictor of readmission was the patient's postal code. Technically correct: certain postal codes correlate with readmission. But the causality (Principle 5) is that those postal codes correspond to zones with less access to outpatient care. The model did not discover a clinical risk factor; it discovered a proxy for social inequality. And the rural hospitals had postal codes the model had never seen.

The action problem: Even when the model correctly identified a high-risk patient, the hospitals had no defined intervention protocol. They knew who was going to be readmitted but not what to do about it. Prediction without action is useless information. Here the intention-behavior gap of Module V manifests: knowing what to do (identifying at-risk patients) and doing what you know (implementing preventive interventions) are radically different things.

6.2 An Informed Redesign

Validate in context before implementing. Do not assume that the model's performance transfers automatically. Demand a pilot with local data before any implementation. Apply the principle of provisional intention (Module V): the model is a hypothesis that must be verified in each new context.

Audit the predictive variables. Ask not only “what does it predict?” but “why does it predict?” If the principal predictor is the postal code, that is a causal alarm signal: the model is using a social proxy, not a clinical factor. Auditing the variables is the ML equivalent of Module II's audit of framing.

Link prediction with action. Design the intervention protocol before implementing the predictive model, not after. The correct question is not “can we predict readmissions?” but “can we prevent them?” Prediction is a means (Module II); prevention is the end.

Communicate the limits to the clinical user. Do not present the model as “89% accuracy.” Present it as: “This model was trained on data from urban hospitals. Its accuracy in your context has not yet been validated. Use the predictions as support, not as a substitute for clinical judgment.” The honest communication of limits is ethical framing (Module V, Topic 3).

6.3 Derived Design Principles

Principle 1: A model trained in one context is a hypothesis in another context. Local validation is not optional; it is the ML equivalent of the principle of falsifiability.

Principle 2: Asking “what does it predict?” without asking “why does it predict?” is dangerous. The predictive variables reveal whether the model captured causality or only correlation—and decisions based on correlation without causality are fragile.

Principle 3: Prediction without an action protocol is useless information. Before implementing a predictive model, design what will be done with its predictions. Prediction is a means; action is the end.

Principle 4: An accuracy metric without context is deceptive. “89% accuracy” means nothing if you do not know on what data, with what population, under what conditions, and with what consequences for the errors.

Principle 5: Machine learning learns from the past. If the future resembles the past, the model is useful. If not, it is a rear-view mirror. The strategic question: to what extent will the future resemble the training data?

7. Glossary of Key Terms

8. Final Reflection and Self-Assessment

Before moving on to the next topic (Algorithmic Biases), verify that you can use your understanding of machine learning as a tool of diagnosis and design.

9. Bibliography and Recommended Readings

Essential Readings

• Domingos, P. (2015). The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World. Basic Books.

• Russell, S. & Norvig, P. (2020). Artificial Intelligence: A Modern Approach. 4th ed. Pearson. [Part V: Machine Learning]

• Mitchell, M. (2019). Artificial Intelligence: A Guide for Thinking Humans. Farrar, Straus and Giroux.

Complementary Readings

• Hastie, T., Tibshirani, R. & Friedman, J. (2009). The Elements of Statistical Learning. 2nd ed. Springer. [A technical reference for the concepts of bias-variance and overfitting]

• Sutton, R.S. & Barto, A.G. (2018). Reinforcement Learning: An Introduction. 2nd ed. MIT Press. [The reference for reinforcement learning]

• O'Neil, C. (2016). Weapons of Math Destruction. Crown. [Anticipates Topic 3: algorithmic biases]

• Pearl, J. & Mackenzie, D. (2018). The Book of Why. Basic Books. [Connects with causality vs. correlation in ML]

Next Topic: The next book addresses Algorithmic Biases: how data, assumptions, and design decisions produce systems that discriminate systematically—not out of malice but out of structure. With O'Neil and Crawford you will study how biases enter the ML pipeline, why the multiple mathematical definitions of fairness are mutually incompatible, and how the intention architect must govern decisions that are inescapably ethical, not technical.