Python for Data course menu
pandas DataFrame Tutorial: Create, Inspect and Explore Tables
Learn the pandas DataFrame: creating one from a dictionary or list of rows, inspecting it with head, info, describe, shape and dtypes, selecting columns, and the index.
On this page
A DataFrame is a table: rows and named columns, where each column is a Series and every column shares the same row index. It is the object you will work with most in data analysis, much like a spreadsheet or a database table.
Creating a DataFrame#
From a dictionary of columns:
import pandas as pd
employees = pd.DataFrame({
"name": ["Asha", "Ravi", "Meera", "John", "Priya"],
"dept": ["Sales", "Engineering", "Engineering", "Marketing", "Sales"],
"salary": [52000, 78000, 81000, 46000, 61000],
"city": ["Mumbai", "Bengaluru", "Pune", "Mumbai", "Delhi"],
})
print(employees) name dept salary city
0 Asha Sales 52000 Mumbai
1 Ravi Engineering 78000 Bengaluru
2 Meera Engineering 81000 Pune
3 John Marketing 46000 Mumbai
4 Priya Sales 61000 DelhiOr from a list of rows (each row a dictionary), which is how data often arrives from APIs:
import pandas as pd
rows = [
{"product": "Laptop", "price": 55000, "stock": 12},
{"product": "Mouse", "price": 700, "stock": 150},
{"product": "Monitor", "price": 12000},
]
print(pd.DataFrame(rows)) product price stock
0 Laptop 55000 12.0
1 Mouse 700 150.0
2 Monitor 12000 NaNThe Monitor row has no stock, so pandas fills in NaN and the column becomes a float column (NaN is a float).
First look at a dataset#
These are the first things to run on any new dataset:
import pandas as pd
employees = pd.DataFrame({
"name": ["Asha", "Ravi", "Meera", "John", "Priya"],
"dept": ["Sales", "Engineering", "Engineering", "Marketing", "Sales"],
"salary": [52000, 78000, 81000, 46000, 61000],
"city": ["Mumbai", "Bengaluru", "Pune", "Mumbai", "Delhi"],
})
print(employees.shape) # (rows, columns)
print(employees.columns.tolist())
print(employees.dtypes)
print(employees.head(2)) # first rows (tail() for the last)(5, 4)
['name', 'dept', 'salary', 'city']
name object
dept object
salary int64
city object
dtype: object
name dept salary city
0 Asha Sales 52000 Mumbai
1 Ravi Engineering 78000 Bengaluruinfo() summarises columns, their non-missing counts and types; describe() summarises the numeric columns:
import pandas as pd
employees = pd.DataFrame({
"name": ["Asha", "Ravi", "Meera", "John", "Priya"],
"dept": ["Sales", "Engineering", "Engineering", "Marketing", "Sales"],
"salary": [52000, 78000, 81000, 46000, 61000],
})
employees.info()
print(employees.describe())<class 'pandas.core.frame.DataFrame'>
RangeIndex: 5 entries, 0 to 4
Data columns (total 3 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 name 5 non-null object
1 dept 5 non-null object
2 salary 5 non-null int64
dtypes: int64(1), object(2)
memory usage: 252.0+ bytes
salary
count 5.000000
mean 63600.000000
std 15501.612819
min 46000.000000
25% 52000.000000
50% 61000.000000
75% 78000.000000
max 81000.000000object is pandas' general type for text columns. A count lower than the number of rows in info() means missing values.
Selecting columns#
One column (with a string) gives a Series; a list of columns gives a smaller DataFrame:
import pandas as pd
employees = pd.DataFrame({
"name": ["Asha", "Ravi", "Meera"],
"dept": ["Sales", "Engineering", "Engineering"],
"salary": [52000, 78000, 81000],
})
print(employees["salary"])
print(type(employees["salary"]).__name__)
print(employees[["name", "salary"]])
print(employees["salary"].mean())0 52000
1 78000
2 81000
Name: salary, dtype: int64
Series
name salary
0 Asha 52000
1 Ravi 78000
2 Meera 81000
70333.33333333333Note the double brackets for several columns: the outer pair selects, the inner pair is a Python list.
The index#
Every DataFrame has a row index. By default it is 0, 1, 2, …, but a meaningful column can become the index, which makes lookups by that value easy:
import pandas as pd
employees = pd.DataFrame({
"name": ["Asha", "Ravi", "Meera"],
"salary": [52000, 78000, 81000],
})
by_name = employees.set_index("name")
print(by_name)
print(by_name.loc["Ravi", "salary"])
print(by_name.reset_index()) salary
name
Asha 52000
Ravi 78000
Meera 81000
78000
name salary
0 Asha 52000
1 Ravi 78000
2 Meera 81000reset_index() turns the index back into a normal column. Most pandas methods return a new DataFrame rather than changing the original; assign the result to keep it.
Common mistakes#
Practice#
Practice exercise
Create a DataFrame of four books with columns title, author, pages and price: ("Clean Code", "Martin", 464, 650), ("Fluent Python", "Ramalho", 1014, 1200), ("Python Crash Course", "Matthes", 552, 700), ("Atomic Habits", "Clear", 320, 450). Print its shape, the average number of pages, the title of the most expensive book, and a table of just titles and prices.
Show hint
df["pages"].mean(); for the most expensive title, find the index of the max price with idxmax() and look up the title with .loc.
Show solution
import pandas as pd
books = pd.DataFrame({
"title": ["Clean Code", "Fluent Python", "Python Crash Course", "Atomic Habits"],
"author": ["Martin", "Ramalho", "Matthes", "Clear"],
"pages": [464, 1014, 552, 320],
"price": [650, 1200, 700, 450],
})
print(books.shape)
print(books["pages"].mean())
print(books.loc[books["price"].idxmax(), "title"])
print(books[["title", "price"]])(4, 4)
587.5
Fluent Python
title price
0 Clean Code 650
1 Fluent Python 1200
2 Python Crash Course 700
3 Atomic Habits 450Key takeaway#
Next: loading real files, in Reading CSV, Excel and JSON.
Finished this lesson?