Data Analysis Interview Questions (pandas, NumPy, Statistics)

Where does the loop run?

Vectorisation means applying an operation to a whole array or column at once instead of looping in Python. The loop still happens, but inside compiled C code, so it is typically one or two orders of magnitude faster and the code is shorter: df["total"] = df["price"] * df["qty"] instead of a for loop over rows.

Labels or positions?

loc selects by label (index values and column names) and includes the end of a slice; iloc selects by integer position and excludes the end, like Python slicing. df.loc["Ravi", "salary"] versus df.iloc[1, 0].

Find, understand why, then decide.

First measure them with df.isna().sum() and understand why they are missing. Then choose: drop rows or columns (dropna, with subset/thresh) when few are affected and missing at random; fill (fillna) with a constant, the median, a group statistic, or forward fill for time series; or keep them with a flag column. Note that calculations like mean() skip NaN by default, and say which approach you used in the report.

Compare the number of rows in the result.

agg returns one row per group; transform returns a result aligned to every original row, so you can add group-level values as a column.

import pandas as pd

df = pd.DataFrame({"team": ["A", "A", "B"], "sales": [10, 30, 50]})
print(df.groupby("team")["sales"].agg("sum").tolist())
print(df.groupby("team")["sales"].transform("sum").tolist())

[40, 50]
[40, 40, 50]

Which rows survive when a key has no match?

An inner merge keeps only keys present in both tables; left keeps every row of the left table (missing matches become NaN); right keeps every row of the right table; outer keeps all rows from both. indicator=True shows which side each row came from, which helps find unmatched records. They correspond exactly to SQL's JOIN types.

Sort first, then take the head of each group.

import pandas as pd

df = pd.DataFrame({"region": ["N", "N", "N", "N", "S", "S"],
                   "rep": ["a", "b", "c", "d", "e", "f"],
                   "sales": [50, 80, 70, 20, 90, 40]})
top = df.sort_values("sales", ascending=False).groupby("region").head(3)
print(top.sort_values(["region", "sales"], ascending=[True, False])["rep"].tolist())

['b', 'c', 'a', 'e', 'f']

Region S has only two reps, so both are kept. An alternative is ranking within groups with groupby(...).rank() and filtering.

Speed versus flexibility.

apply runs a Python function per value or per row, which is flexible but slow on large data. Prefer built-in vectorised operations, .str and .dt methods, np.where/np.select, map with a dictionary, and groupby/merge. Use apply for logic that has no vectorised equivalent, and measure if performance matters.

Think about salaries.

For skewed data or data with outliers (incomes, prices, delivery times), because a few extreme values pull the mean but barely move the median. For roughly symmetric data the two are similar and the mean is fine. Reporting both, with the count, is often best.

Rules help, but investigation decides.

Visualise first (box plots, histograms), then apply a rule: the IQR rule (below Q1 − 1.5·IQR or above Q3 + 1.5·IQR) is robust; z-scores beyond ±3 assume roughly normal data. Then investigate: fix or remove data entry errors, keep genuine extremes (perhaps analysing them separately), or cap values (clip) when a model is sensitive to them. Document the decision.

Think of a third factor.

No. Correlation can come from a confounder that drives both (hot weather drives ice-cream sales and sunburn), from reverse causation, or from chance. Causal claims need a randomised experiment such as an A/B test, or a careful causal analysis that controls for confounders.

It is a probability calculated assuming the null hypothesis is true.

The probability of seeing a result at least as extreme as the observed one if the null hypothesis (no effect) were true. A small p-value means the data would be surprising under "no effect". It is not the probability that the null hypothesis is true, and it does not measure how large or important the effect is; report a confidence interval for that.

Metric, randomisation, sample size, duration, analysis.

Define one primary metric (purchase conversion) and guardrail metrics (refunds, page speed). Randomly assign users (not page views) 50/50. Choose the minimum effect worth detecting, α and power, and compute the sample size; convert it to a duration of at least one full week. Do not stop early when results look good. Analyse with a two-proportion z-test and a confidence interval for the uplift, check the split is balanced, and weigh practical significance against cost.

A trend in every group can flip when groups are combined.

A pattern that holds within every subgroup reverses when the subgroups are combined, because the groups differ in size or composition (a confounder). Example: a treatment can have a higher recovery rate for both mild and severe patients yet a lower rate overall if it was given mostly to severe patients. Compare within groups or adjust for the confounder rather than trusting pooled totals.

Load less, store smaller, or process in pieces.

Load only needed columns (usecols) and rows; process the file in chunks (chunksize) and combine partial aggregates; convert low-cardinality text to category and downcast numbers; parse dates once. If it is still too large, aggregate in a database with SQL first, or use tools designed for larger-than-memory data.

Each user's first month, then months since that month.

Assign each order a month (dt.to_period("M")), give each user a cohort with groupby("user")["month"].transform("min"), compute months since the cohort start, then count unique users per cohort and age with groupby([...])["user"].nunique().unstack(). Divide each row by its month-0 count to get retention percentages. The advanced interview lesson shows the full code.