Machine Learning Cheat Sheet

The scikit-learn workflow, choosing a model, regression and classification metrics, pipelines, cross-validation and tuning, ensembles, clustering, PCA, text, saving models and the PyTorch training loop.

Updated October 24, 2026

# → value shows what an expression returns. # name → value shows what a variable holds after that line. Examples use scikit-learn's built-in datasets, so they run offline.

The workflow#

Python

from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X, y = load_breast_cancer(return_X_y=True)
X.shape                                  # → (569, 30)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
len(X_train), len(X_test)                # → (426, 143)

model = make_pipeline(StandardScaler(), LogisticRegression())
model.fit(X_train, y_train)
round(model.score(X_test, y_test), 3)    # → 0.958
model.predict(X_test[:3]).tolist()       # → [1, 0, 0]

Every scikit-learn model has the same interface: fit(X, y), predict(X), score(X, y); classifiers add predict_proba(X); transformers have fit_transform and transform.

Choosing a model#

Task	Start with	Then try
Regression	`LinearRegression`, `Ridge`	`HistGradientBoostingRegressor`, `RandomForestRegressor`
Classification	`LogisticRegression`	`HistGradientBoostingClassifier`, `RandomForestClassifier`, `SVC`
Clustering	`KMeans`	`DBSCAN`, `AgglomerativeClustering`
Fewer features	`PCA`	feature selection
Anomalies	`IsolationForest`	`LocalOutlierFactor`
Text	`TfidfVectorizer` + `LogisticRegression`	embeddings, LLMs
Images, audio	pretrained neural network (PyTorch)	fine-tuning

Regression metrics#

Python

from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

actual = [100, 150, 200, 250]
predicted = [110, 140, 210, 230]
mean_absolute_error(actual, predicted)              # → 12.5
mean_squared_error(actual, predicted) ** 0.5        # → 13.228756555322953
round(r2_score(actual, predicted), 3)               # → 0.944

MAE is in the target's units; RMSE punishes large errors more; R² is the share of variance explained (1 is perfect, 0 is no better than the mean).

Classification metrics#

Python

from sklearn.metrics import confusion_matrix, f1_score, precision_score, recall_score

y_true = [1, 1, 1, 0, 0, 0, 0, 1]
y_pred = [1, 0, 1, 0, 0, 1, 0, 1]
confusion_matrix(y_true, y_pred).tolist()     # → [[3, 1], [1, 3]]
precision_score(y_true, y_pred)               # → 0.75
recall_score(y_true, y_pred)                  # → 0.75
f1_score(y_true, y_pred)                      # → 0.75

Confusion matrix rows are actual classes, columns predicted: [[TN, FP], [FN, TP]]. Use roc_auc_score and average_precision_score with probabilities from predict_proba(X)[:, 1].

Preprocessing with ColumnTransformer#

Python

import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder

df = pd.DataFrame({"age": [25, None, 40], "city": ["Pune", "Delhi", "Pune"]})
prep = ColumnTransformer([
    ("num", make_pipeline(SimpleImputer(strategy="median"), StandardScaler()), ["age"]),
    ("cat", OneHotEncoder(handle_unknown="ignore"), ["city"]),
])
prep.fit_transform(df).shape                  # → (3, 3)
prep.get_feature_names_out().tolist()         # → ['num__age', 'cat__city_Delhi', 'cat__city_Pune']

Put preprocessing and the model in one pipeline so cross-validation fits preprocessing on training folds only (no leakage).

Cross-validation and tuning#

Python

from sklearn.model_selection import GridSearchCV, cross_val_score

scores = cross_val_score(model, X, y, cv=5)
round(float(scores.mean()), 3)                # → 0.981

search = GridSearchCV(model, {"logisticregression__C": [0.1, 1, 10]}, cv=5)
search.fit(X, y)
search.best_params_                           # → {'logisticregression__C': 1}

Parameter names in a pipeline are stepname__parameter. Use RandomizedSearchCV when there are many settings, StratifiedKFold for classification, GroupKFold for grouped rows and TimeSeriesSplit for time order.

Trees and ensembles#

Python

from sklearn.ensemble import HistGradientBoostingClassifier, RandomForestClassifier
from sklearn.tree import DecisionTreeClassifier

tree = DecisionTreeClassifier(max_depth=3, random_state=0).fit(X_train, y_train)
round(tree.score(X_test, y_test), 3)          # → 0.916
forest = RandomForestClassifier(n_estimators=200, random_state=0).fit(X_train, y_train)
round(forest.score(X_test, y_test), 3)        # → 0.944
boost = HistGradientBoostingClassifier(random_state=0).fit(X_train, y_train)
round(boost.score(X_test, y_test), 3)         # → 0.951

Trees need no scaling. Key settings: max_depth, min_samples_leaf; forests: n_estimators, max_features; boosting: learning_rate, max_iter, max_leaf_nodes, early stopping.

Unsupervised learning#

Python

from sklearn.cluster import KMeans
from sklearn.decomposition import PCA
from sklearn.metrics import silhouette_score

X_scaled = StandardScaler().fit_transform(X)
km = KMeans(n_clusters=2, n_init=10, random_state=0).fit(X_scaled)
round(silhouette_score(X_scaled, km.labels_), 3)          # → 0.343
pca = PCA(n_components=0.95).fit(X_scaled)
int(pca.n_components_)                                    # → 10

Scale before k-means and PCA. Choose k with inertia (elbow) and silhouette; PCA(n_components=0.95) keeps 95% of the variance.

Text#

Python

from sklearn.feature_extraction.text import TfidfVectorizer

docs = ["great food", "slow service", "great service"]
tfidf = TfidfVectorizer()
tfidf.fit_transform(docs).shape               # → (3, 4)
tfidf.get_feature_names_out().tolist()        # → ['food', 'great', 'service', 'slow']

Options: ngram_range=(1, 2), stop_words="english", min_df=2. Compare embeddings with cosine similarity.

Saving a model#

Python

import joblib

joblib.dump(model, "model.joblib")
loaded = joblib.load("model.joblib")
(loaded.predict(X_test) == model.predict(X_test)).all()   # → np.True_

Save the whole pipeline, record library versions, and only load files you trust (loading can run code). PyTorch: torch.save(model.state_dict(), path) and model.load_state_dict(torch.load(path, weights_only=True)).

PyTorch training loop#

Text

model = nn.Sequential(nn.Linear(n_features, 32), nn.ReLU(), nn.Linear(32, n_classes))
loss_fn = nn.CrossEntropyLoss()                      # takes raw logits, integer labels
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
for epoch in range(epochs):
    model.train()
    for xb, yb in loader:                            # DataLoader(TensorDataset(X, y), batch_size=32, shuffle=True)
        optimizer.zero_grad()
        loss = loss_fn(model(xb), yb)
        loss.backward()
        optimizer.step()
    model.eval()
    with torch.no_grad():
        val_pred = model(X_val).argmax(dim=1)

Common pitfalls#

Fitting scalers, imputers or encoders on all data before splitting (leakage).
Judging imbalanced classification by accuracy alone.
Tuning on the test set; keep it for one final check.
Features that are computed from the target or unavailable at prediction time.
Forgetting random_state, which makes results impossible to reproduce.

Want the full explanations? Go to the Learn Machine Learning with Python course →