Skip to content
Laviyx Learn

Try

Python for Data course menu

pandas apply, map and Custom Functions (and When to Avoid Them)

Use your own functions in pandas: Series.map with dictionaries and functions, Series.apply, DataFrame.apply across rows and columns, pipe for readable pipelines, and why vectorised code is preferred when it exists.

Intermediate7 min readUpdated
On this page

Sometimes no built-in method does exactly what you need. pandas lets you run your own function on every value, every row or every column. These tools are flexible, but they run Python code once per item, so they are slower than vectorised operations. Use them when a vectorised version does not exist.

Series.map: translate values#

map with a dictionary replaces each value with the matching entry; values not in the dictionary become NaN:

Python
import pandas as pd

grades = pd.Series(["A", "B", "A", "C", "E"])
points = grades.map({"A": 10, "B": 8, "C": 6, "D": 4})
print(points.tolist())
print(grades.map(str.lower).tolist())        # a function works too
Output
[10.0, 8.0, 10.0, 6.0, nan]
['a', 'b', 'a', 'c', 'e']

"E" is not in the dictionary, so it maps to NaN, which is a useful way to spot unexpected values.

Series.apply: a function per value#

Python
import pandas as pd

def delivery_band(minutes):
    if minutes <= 30:
        return "fast"
    if minutes <= 45:
        return "normal"
    return "slow"

times = pd.Series([22, 41, 58, 30, 45])
print(times.apply(delivery_band).tolist())
print(times.apply(lambda m: f"{m // 60}h {m % 60}m").tolist())
Output
['fast', 'normal', 'slow', 'fast', 'normal']
['0h 22m', '0h 41m', '0h 58m', '0h 30m', '0h 45m']

For a simple condition like this, the vectorised pd.cut or np.select (see Adding Columns) would be faster on big data; apply shines when the logic is genuinely complex.

DataFrame.apply: rows or columns#

On a DataFrame, apply passes whole columns by default (axis=0), or whole rows with axis=1:

Python
import pandas as pd

scores = pd.DataFrame({"maths": [92, 67, 78], "science": [88, 74, 81], "english": [75, 90, 69]},
                      index=["Asha", "Ravi", "Meera"])
print(scores.apply(lambda col: col.max() - col.min()))        # per subject
print(scores.apply(lambda row: row.idxmax(), axis=1))         # best subject per student

def summary(row):
    return f"avg {row.mean():.1f}, best {row.max()}"

print(scores.apply(summary, axis=1))
Output
maths      25
science    14
english    21
dtype: int64
Asha       maths
Ravi     english
Meera    science
dtype: object
Asha     avg 85.0, best 92
Ravi     avg 77.0, best 90
Meera    avg 76.0, best 81
dtype: object

With axis=1, each row is a Series whose labels are the column names.

Prefer vectorised code when you can#

Many apply calls have a faster vectorised equivalent:

With applyVectorised
df.apply(lambda r: r["a"] + r["b"], axis=1)df["a"] + df["b"]
s.apply(lambda x: x * 1.18)s * 1.18
s.apply(lambda x: x.strip().lower())s.str.strip().str.lower()
s.apply(lambda x: "big" if x > 100 else "small")np.where(s > 100, "big", "small")

Both give the same results:

Python
import numpy as np
import pandas as pd

df = pd.DataFrame({"a": np.arange(5), "b": np.arange(5) * 10})
with_apply = df.apply(lambda r: r["a"] + r["b"], axis=1)
vectorised = df["a"] + df["b"]
print(with_apply.equals(vectorised), vectorised.tolist())
Output
True [0, 11, 22, 33, 44]

On a DataFrame with a million rows, the row-wise apply is typically hundreds of times slower. Measure with %timeit in Jupyter if in doubt.

pipe: readable pipelines#

pipe passes the whole DataFrame to a function. It lets you chain your own cleaning steps like built-in methods:

Python
import pandas as pd

def clean_names(df):
    return df.assign(name=df["name"].str.strip().str.title())

def add_total(df, tax=0.18):
    return df.assign(total=(df["price"] * (1 + tax)).round(2))

raw = pd.DataFrame({"name": [" pen ", "NOTEBOOK"], "price": [20, 60]})
result = raw.pipe(clean_names).pipe(add_total, tax=0.05)
print(result)
Output
       name  price  total
0       Pen     20   21.0
1  Notebook     60   63.0

Each step is a small, testable function, and the chain reads top to bottom.

Common mistakes#

Practice#

Practice exercise

Shipping costs depend on both weight and zone: zone "local" costs ₹40 plus ₹10 per kg; "national" costs ₹80 plus ₹20 per kg, and orders over 5 kg get ₹50 off. Write a function and apply it row by row to the orders weight=[1.0, 6.5, 3.0], zone=["local", "national", "national"].

Show hint

Write def shipping(row) that reads row["weight"] and row["zone"], then use df.apply(shipping, axis=1).

Show solution
Python
import pandas as pd

def shipping(row):
    if row["zone"] == "local":
        cost = 40 + 10 * row["weight"]
    else:
        cost = 80 + 20 * row["weight"]
    if row["weight"] > 5:
        cost -= 50
    return cost

orders = pd.DataFrame({"weight": [1.0, 6.5, 3.0], "zone": ["local", "national", "national"]})
orders["shipping"] = orders.apply(shipping, axis=1)
print(orders)
Output
   weight      zone  shipping
0     1.0     local      50.0
1     6.5  national     160.0
2     3.0  national     140.0

This rule could be vectorised with np.where, but the function version is clearer, and for a few thousand rows the speed difference does not matter.

Key takeaway#

Next: Grouping and Aggregating.

Finished this lesson?

Related lessons

Python for DataAdd, Rename and Drop Columns in pandasCreate new pandas columns from calculations, conditions with np.where and np.select, and bins with cut; rename and reorder columns, drop columns and rows, and build several columns at once with assign.Beginner8 min readPython for DataCleaning Text Data in pandas with .str MethodsClean messy text columns in pandas: strip spaces, change case, replace characters, split columns, extract values with regular expressions, check text with contains and startswith, and standardise inconsistent labels.Intermediate7 min readPythonPython Lambda Functions: Short Anonymous Functions With ExamplesLearn Python lambda functions: syntax, when to use them, sorting with key=lambda, max and min by a field, map and filter, comparing lambdas with list comprehensions, and when a normal def is better.Intermediate6 min read