Machine Learning Fundamentals

Prediction tasks, features, and targets

A learning task specifies an input, an output, and the circumstances in which the output will be used. Statistical learning discovers predictive patterns from examples. Unlike a programmer-authored rule, the resulting behavior depends on the examples and fitting procedure. Both execute code; neither a successful fit nor executable code alone establishes useful predictions. [9]

A build-duration predictor might estimate one job's build-command runtime, in minutes, immediately before execution, to inform a waiting-time estimate. A job is one execution within a potentially larger build. Whole-build duration, setup time, and command runtime are different targets. The TravisTorrent schema distinguishes these measurements; choosing one is part of specifying the task. [10]

A feature is an input attribute; a feature vector collects one example's inputs. A label, or target, is its recorded answer. Parameters are fitted values shared across examples. The prediction is the model's computed answer, distinct from the observed target. These are the basic objects of supervised learning. [9]

Numeric inputs need a defined representation. One-hot encoding gives each category a binary column, avoiding an invented ordering such as compiler A < compiler B. Standardization subtracts a training-fitted mean and divides by a training-fitted standard deviation. Retain those values for later inputs. Preprocessing documentation explains both transformations. [5]

Recorded durations can contain measurement noise. Missing or canceled jobs also require a declared inclusion rule: an absent duration is not a measured zero. Otherwise the dataset silently changes what the predictor estimates. [12][10]

Training, fitted state, and inference

The training set contains examples used for fitting: determining model parameters from data. Inference applies the fitted function to another input. Saving a model preserves its learned state; restoring it requires the corresponding computational structure. An evaluation may compare predictions with labels and calculate loss without updating anything. [13]

Fitted preprocessing belongs with the predictor. A new build must use the retained encoding and scaling, rather than recomputing them from itself or an evaluation batch. Otherwise the same numerical coordinate can acquire a different meaning. [11]

Supervision, baselines, and loss

Supervised learning uses paired inputs and recorded targets. Target choice determines the task: predicting duration is regression, while predicting failure is classification. Encoding success as 0 and failure as 1 does not turn category prediction into duration regression. [9][16]

A loss assigns a numerical penalty to a prediction. Empirical risk minimization means minimizing average loss on observed examples. The training objective may add penalties or constraints. Comparing a feature-dependent model with a baseline on the same new cases tests whether its added complexity buys predictive value. [3][17]

Classification can produce a probability score rather than only a label. Logistic regression transforms a weighted input score through a sigmoid, mapping any finite score between zero and one. For failure labels, this estimates failure probability under the modeled distribution. A probability-shaped output is not automatically calibrated: its numerical confidence still needs checking against outcomes. [18]

Label ambiguity differs from measurement noise. A paper may reasonably fit several subject categories even when the dataset records only its author's chosen primary category. Exact-match scoring then penalizes another defensible answer. Annabell Schäfer's classification example illustrates why a deterministic scoring rule does not make the underlying target unambiguous. [20]

Model families and inductive assumptions

A hypothesis class is the set of functions available to the learner. Model capacity describes how flexibly that class can fit patterns. An inductive bias is a restriction or preference favoring some solutions. Examples constrain predictions where observations exist; assumptions determine much of what happens elsewhere. [2]

A hyperparameter controls fitting or the available model family rather than being learned by an ordinary fitting update. A tree's maximum depth is a hyperparameter; its selected split thresholds are fitted values. Learning rate is another hyperparameter. Choosing such settings uses validation evidence, not the final test results. [1][3][23]

Gradients and parameter updates

A gradient describes local sensitivity: how a small change in each parameter would change the objective. Gradient descent moves opposite that direction. The learning rate scales the move. Local information does not guarantee that a large step helps, and reducing training loss does not guarantee better predictions on future examples. [3]

For the dimensionless example $L(w)=(w-2)^2$, the gradient at $w=0$ is −4. A learning rate of 0.1 gives $w'=0-0.1(-4)=0.4$, lowering loss from 4 to 2.56. A rate of 1.25 instead reaches 5, where loss is 9. The direction was useful; the displacement was excessive. [24]

for x_batch, y_batch in training_batches:
    optimizer.zero_grad()
    predictions = model(x_batch)
    loss = loss_fn(predictions, y_batch)
    loss.backward()
    optimizer.step()

Backpropagation computes gradients through composed differentiable operations using the chain rule. The optimizer uses those gradients to change parameters. Clearing gradients matters because frameworks such as PyTorch accumulate them. The forward pass produces predictions; neither that pass nor loss calculation alone performs the parameter update. [25]

A minibatch is a subset processed for one update. Minibatch stochastic gradient descent uses its average gradient instead of the entire dataset's gradient. Smaller batches introduce more sampling variation; larger batches can exploit vectorized computation but change update frequency. An epoch is one pass through the training examples, potentially containing many updates. [24]

Structure learning and data-derived targets

Unsupervised learning fits structure without externally supplied answer labels. K-means clustering assigns observations to a chosen number of centroids—the groups' mean vectors—to reduce within-group squared distances. Compactness under those coordinates is its objective, not discovery of authoritative categories. [4]

Relative feature scaling changes distance and therefore can change clustering. Subtracting the same mean cancels in pairwise differences; dividing one coordinate by a larger scale reduces its contribution. The illustration changes only the source-size scale, retaining records and cluster count. Scaling is a modeling choice, not evidence of meaningful feature importance. [5]

These methods can be composed. A self-supervised model can supply frozen features for a supervised predictor; a linear probe trains only a final linear mapping to test what those features already contain. Embeddings and Representation Learning develops representation geometry, while Pretraining and Midtraining covers large-scale data-derived training. [30][29]

Cluster interpretation still requires domain judgment. A group of price-related agent traces might combine incorrect quotations with incorrect refund calculations, despite different causes. Ben Hylak's trace-analysis example shows why a coherent-looking cluster is not automatically a stable, actionable issue. [31]

Actions, rewards, and learning from interaction

Reinforcement learning improves action choices using rewards from interaction. The environment responds to actions; a scheduler's environment includes jobs and resources. Its observation exposes current allocations and waiting jobs. A state is sufficient information for predicting subsequent transitions; an observation may reveal only part of that state. [32]

A Markov decision process assumes that current state and action suffice to determine the next-state and reward distributions. Compressed observations can violate that sufficiency. DeepRM's scheduling formulation explicitly acknowledges partial observability; a job summary should not automatically be treated as a complete environment state. [32]

Rewards can also train generated behavior when outcomes are automatically checkable. Stefano Fiorucci's game example scores outcomes and invalid moves, illustrating that environment rules determine which behavior receives credit. Post-training and Alignment develops these methods for model behavior; a checkable score still needs to express the intended goal. [36][37]

Generalization and independent assessment

Generalization means performing well on new cases from a specified population or process. A sample contains finitely many such cases. Population risk is expected loss—the average under that process over repeated draws. Empirical risk averages the observed sample. Different samples yield different estimates even when the predictor is fixed. [38]

A lookup table can reproduce recorded job durations while having no answer for a new job. Low training loss therefore does not establish low population risk. The generalization gap separates those quantities. A fresh representative test sample estimates new-case performance; deterministic repetition of a prediction says nothing about whether it is correct. [38]

Leakage occurs when information crosses a boundary needed for realistic prediction or independent assessment. It includes preprocessing fitted on test data. During cross-validation, a fold is one partition used in repeated training–validation rounds. Fit transformations separately inside each training fold, then apply them unchanged to that round's validation fold. [11][39]

The split must match the claim. Holding out projects tests transfer to unseen projects. Predicting future builds of existing projects requires chronological separation. Related retries should not straddle a boundary that claims independent cases. Group separation and time separation address different dependencies; some tasks need both. [40]

Overfitting, regularization, and residual uncertainty

Underfitting means the fitted model misses useful structure, potentially because its family is too restrictive. High training loss can also reflect unsuccessful optimization. Overfitting means fitting sample-specific patterns that fail to transfer. Improving training loss while validation loss worsens is a warning, provided the losses and evaluation conditions are comparable. [26]

In a small language-model training exercise, Angelos Perivolaropoulos describes validation loss eventually rising despite continued training improvement. Validation loss was a useful signal, but task behavior still required inspection. The plotted example expresses that possible divergence without prescribing a universal curve. [6]

For squared-error prediction, bias describes how the average fitted prediction differs from the best prediction for the problem. Variance describes how fitted predictions change across newly sampled training datasets. Irreducible noise is outcome variability remaining beyond those differences. Prediction variance here is not randomness from repeatedly sampling outputs of one fixed model. [43]

The decomposition does not impose one universal relationship between capacity and test error. Some settings show error falling again after an initial rise as capacity increases. Model size alone therefore cannot diagnose overfitting or select a remedy; separated predictive evidence must decide. [2]

Distribution shift and prediction coverage

A data distribution describes how often inputs and outcomes occur together. A conditional distribution describes outcomes for a given input, or inputs for a given outcome. Distribution shift changes those patterns between training and deployment. [44]

Interpolation predicts within the observed input domain; extrapolation extends outside it. More frequent large projects can remain inside historical coverage. Unprecedented sizes require extrapolation. In multiple dimensions, each feature being individually within range does not establish coverage of their combination. [8]

An accuracy change alone does not identify a shift mechanism. Data Quality and Curation develops collection and coverage practices; Evals develops assessment under changed conditions. [44]

Prediction and causal effects

An intervention sets an action. A counterfactual describes its possible outcome. A causal effect compares interventions; confounding arises from shared causes of action and outcome. [46]

Predicting duration under historical worker assignment differs from estimating what additional workers would accomplish. Random assignment can replace an assignment mechanism driven by difficulty, making comparable treatment groups in expectation. The intervention and outcome must still be specified, and the experiment must address the intended population. Observational comparisons require additional identification assumptions. [47]

Feature importance describes a fitted predictor under a chosen attribution procedure. It does not identify what changing the corresponding real-world quantity would cause. Even changing the worker-count input and observing a smaller model prediction establishes only a computational response. Real-world effects depend on the assignment and outcome mechanisms, not just the predictor's accuracy. [49]

From predictive performance to system usefulness

A model test supports a bounded predictive claim. Integration tests establish that the surrounding software passes the right inputs and handles outputs correctly. Deployed-system evaluation asks whether the resulting workflow achieves its intended outcome. A duration forecast may be accurate while the scheduling rule that consumes it remains ineffective. [50]

Zed's edit-prediction work adds production acceptance and latency measurements because offline scores need not reflect editor preferences. Acceptance itself does not establish increased productivity. Evals develops intended-use criteria, uncertainty, offline versus live evidence, and failure investigation needed to connect model behavior with useful consequences. [22]

Open questions

1. 01

   Efficient assessment under repeated adaptation remains difficult: each revealed score can influence subsequent choices. Progress would preserve a defensible future-performance estimate under a declared query budget, without requiring a prohibitively large fresh dataset for every revision. [41]

2. 02

   Separating ambiguous labels from missing information remains a practical learning problem. More optimization cannot recover an author preference absent from the input. Progress would demonstrate that revised labels or additional available features improve independent agreement and task usefulness, rather than only the development score. [20][53]

3. 03

   Choosing model structure under a limited data budget remains uncertain because expressive capacity and generalization do not follow one universal curve. Progress would predict which structural assumptions transfer to new tasks under matched data and compute, then confirm those predictions independently. [2]

4. 04

   Learning policies from narrow experience leaves unvisited actions poorly understood. Exploration matters because it changes future training data, but informative actions can have costs. Progress would demonstrate improved held-out policy performance while explicitly bounding exploration costs and measuring coverage beyond familiar trajectories. [35]