Frequently asked machine learning, data scientist and ML engineer interview questions on model basics, evaluation, trees and boosting, unsupervised learning, neural networks, LLMs and deploying models, with hints and answers.
16 questions · Updated
Are there labels?
Supervised learning trains on examples with known answers (labels) to predict answers for new data: regression for numbers, classification for categories. Unsupervised learning has no labels and finds structure: clustering, dimensionality reduction (PCA) and anomaly detection.
Compare training and validation scores.
The model learns noise in the training data, so it scores much better on training data than on new data. Prevent it with simpler models or limits on complexity (max_depth), regularisation, more data, fewer noisy features, ensembles, cross-validated tuning and early stopping.
What does the scaler learn?
The scaler learns means and standard deviations. Fitting it on all data lets information from the test set into training (leakage), so test scores become optimistic. Fit preprocessing inside a pipeline so it only ever learns from training data, including in each cross-validation fold.
Which mistake is more expensive?
When missing a positive case is costly and a false alarm is cheap: disease screening, fraud detection, safety defects. Prioritise precision when false alarms are costly, such as a spam filter hiding real emails. The decision threshold lets you trade one for the other.
What does always predicting "not fraud" score?
Not necessarily: predicting "not fraud" every time also scores 98% and catches nothing. Check the fraud class's precision and recall, the confusion matrix, and average precision, and compare with that trivial baseline.
Precision = TP / (TP + FP); recall = TP / (TP + FN).
from sklearn.metrics import precision_score, recall_score
y_true = [1, 1, 1, 1, 0, 0, 0, 0]
y_pred = [1, 1, 0, 0, 1, 0, 0, 0]
print(precision_score(y_true, y_pred), recall_score(y_true, y_pred))0.6666666666666666 0.5Three cases are predicted positive and two of them are right (precision 2/3); two of the four real positives are found (recall 0.5).
Too simple versus too sensitive.
Bias is error from a model too simple to capture the pattern (underfitting); variance is error from sensitivity to the particular training sample (overfitting). More flexibility lowers bias and raises variance. Choose the complexity with the lowest validation error; more data reduces variance.
Which one produces exact zeros?
Both penalise large coefficients. L2 (Ridge) adds the sum of squared coefficients and shrinks all of them smoothly. L1 (Lasso) adds the sum of absolute values and sets some coefficients exactly to zero, selecting features. Elastic Net mixes both. Scale features first so the penalty treats them equally.
Parallel or sequential trees?
A random forest averages many deep trees trained independently on bootstrap samples with random feature subsets; it reduces variance and needs little tuning. Gradient boosting adds shallow trees one after another, each fixing the previous errors; it reduces bias, is usually more accurate on tabular data, and needs careful tuning of learning rate, number of trees and depth.
Every row is used for validation exactly once.
The data is split into k folds; the model trains k times, each time validating on a different fold, and the scores are averaged. Variants: stratified k-fold (keeps class proportions), group k-fold (keeps all rows of one customer or patient together), and time-series splits (always validate on later data than training).
There are no labels to check against.
Compare inertia across k and look for the elbow, compare silhouette scores, check that clusters are stable across random starts, and above all check that the clusters are meaningful and actionable. The methods often disagree.
Multiplying many small numbers.
In deep networks, gradients for early layers are products of many factors; with saturating activations like sigmoid they shrink towards zero and those layers stop learning. Fixes: ReLU-family activations, good initialisation, batch or layer normalisation, residual connections, and gradient clipping for exploding gradients.
Some layers behave differently during training.
model.eval() switches dropout off and makes batch normalisation use its stored statistics, so predictions are deterministic and correct. torch.no_grad() stops gradient tracking, saving memory and time. Forgetting eval() gives wrong, run-to-run varying validation scores.
Embeddings plus an LLM.
Documents are split into chunks, embedded and stored in a vector index. For each question, the most similar chunks are retrieved and sent to an LLM along with the question, with instructions to answer from that context. It gives answers grounded in current, private data without retraining, and makes sources citable. Evaluate retrieval quality and answer faithfulness separately.
Leakage, skew, drift.
Check for leakage in the offline evaluation (features unavailable at prediction time, preprocessing fitted on test data, the same customers in train and test), training-serving skew (features computed differently in production), data drift in the inputs, concept drift in the relationship to the target, and whether the test set represented production (time period, population).
Never switch all traffic at once.
Validate it offline against the current model on the same data, run it in shadow mode, release it to a small canary share of traffic, A/B test the business metric, monitor latency, errors, input drift and prediction distributions, and keep the previous version ready for instant rollback.