Beginner Machine Learning Interview Questions and Answers

Supervised learning trains on examples with known answers (labels) and learns to predict the answer for new examples: predicting house prices (regression) or spam vs not spam (classification). Unsupervised learning has no labels and finds structure in the data itself: grouping customers into segments (clustering), reducing many features to a few (PCA), or flagging unusual records (anomaly detection).

Both are supervised. Regression predicts a number (price, temperature, delivery time) and is measured with errors such as MAE or RMSE. Classification predicts a category (spam/not spam, which species) and is measured with accuracy, precision, recall and similar. Logistic regression, despite its name, is a classification model.

To estimate how the model performs on new data. A model scored on the data it learned from looks better than it really is, because it may have memorised that data. The test set is held back, never used for training or for choosing settings, and evaluated once at the end. Settings are chosen with a separate validation set or cross-validation.

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0, stratify=y)
print(len(X_train), len(X_test))

120 30

Overfitting is when a model learns noise and quirks of the training data, so it scores very well on training data and noticeably worse on new data. Detect it by comparing training and validation scores: a large gap means overfitting.

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)
for depth in [None, 3]:
    tree = DecisionTreeClassifier(max_depth=depth, random_state=0).fit(X_train, y_train)
    print(f"max_depth={depth}: train {tree.score(X_train, y_train):.3f}, test {tree.score(X_test, y_test):.3f}")

max_depth=None: train 1.000, test 0.881
max_depth=3: train 0.977, test 0.937

Prevent it with a simpler model or limits on complexity (like max_depth), regularisation, more training data, fewer noisy features, ensembles (random forests), and early stopping for neural networks.

The opposite: the model is too simple to capture the real pattern, so it does poorly on both training and test data (for example, a straight line fitted to clearly curved data). Fix it with a more flexible model, better features, or less regularisation.

If 99% of transactions are genuine, a model that always predicts "genuine" is 99% accurate and catches no fraud at all. For rare classes, report precision (of the cases flagged, how many were truly positive) and recall (of the true positives, how many were caught), plus the confusion matrix.

A spam filter flags 10 emails; 8 are really spam, and 20 spam emails exist in total.

• Precision = 8 / 10 = 0.8: of what we flagged, 80% was right.
• Recall = 8 / 20 = 0.4: we caught 40% of all spam.

from sklearn.metrics import precision_score, recall_score

y_true = [1] * 20 + [0] * 80               # 20 spam, 80 normal
y_pred = [1] * 8 + [0] * 12 + [1] * 2 + [0] * 78   # 8 spam caught, 2 normal emails wrongly flagged
print(precision_score(y_true, y_pred), recall_score(y_true, y_pred))

0.8 0.4

Which matters more depends on the cost of mistakes: a spam filter needs high precision (don't hide real emails); a disease screening test needs high recall (don't miss patients).

A table of actual vs predicted classes. For two classes it has four cells: true negatives, false positives, false negatives and true positives. Every common classification metric (accuracy, precision, recall, F1) is calculated from these counts, and the matrix shows which mistakes the model makes.

Scaling puts features on comparable ranges (for example, standardising to mean 0 and standard deviation 1). Models that use distances or gradient descent need it: k-nearest neighbours, SVMs, k-means, PCA, regularised linear and logistic regression, and neural networks. Otherwise the feature with the biggest numbers dominates. Tree-based models (decision trees, random forests, gradient boosting) do not need scaling.

Leakage is when information that would not be available at prediction time gets into training, so test scores look great but real performance is poor. Two common forms:

• Preprocessing leakage: fitting a scaler, imputer or feature selection on all the data before splitting. Fix: fit inside a pipeline on training data only.
• Target leakage: a feature that is caused by, or computed from, the target, such as "refund issued" when predicting complaints, or price per square foot when predicting price.

Models need numbers. One-hot encoding creates a 0/1 column per category and suits categories with no order (city, colour). Ordinal encoding maps ordered categories to numbers (small < medium < large). For categories with many values, options include grouping rare values, target encoding (carefully, inside cross-validation), or models with native categorical support.