Select DataFrame rows

Summary: in this tutorial, you’ll learn how to select rows from a DataFrame object.

Select the first n rows

head is the built-in method to quickly select the first n rows from a DataFrame object. This is particularly useful if you’re wondering whether you have the right type of data inside the object.

By default, head returns the first 5 rows from the DataFrame.

import pandas as pd
url = 'https://www.fdic.gov/resources/resolutions/bank-failures/failed-bank-list/'
df = pd.read_html(url)
df[0].head()

Output :

If you want more than the first 5 rows, pass a number to head.

df.head(10) # Get first 10 rows from a DataFrame

Interestingly, you can pass a negative number to head. In this case, the method trims the last n rows and return the remaining part of the DataFrame, equivalent to df[:-n].

import pandas as pd
url = 'https://www.fdic.gov/resources/resolutions/bank-failures/failed-bank-list/'
df = pd.read_html(url)
# Trim 555 rows from the bottom of the object
# and return the remaining rows
df[0].head(-555)

We’ll get the first 7 rows as the output, 555 rows from the last rows has been trimmed out.

Select the last n rows

Contrary to the head to get rows from the top, Pandas has a built-in tail method to select from the last rows upwards.

import pandas as pd
url = 'https://www.fdic.gov/resources/resolutions/bank-failures/failed-bank-list/'
df = pd.read_html(url)
df[0].tail()

Similarly, passing a number n to the method gets us the last n elements from the DataFrame object. If a negative number is used, this method returns all rows except the first n rows.

import pandas as pd
url = 'https://www.fdic.gov/resources/resolutions/bank-failures/failed-bank-list/'
df = pd.read_html(url)
# Get the last 10 rows
df[0].tail(10)

Select rows by index

Just like how Python list slicing works, you can extract a range of rows from anywhere in the middle of the DataFrame using row indexes. For example.

import pandas as pd
url = 'https://www.fdic.gov/resources/resolutions/bank-failures/failed-bank-list/'
df_list = pd.read_html(url)
df = df_list[0]
# Get 5 rows from rows number 10
df[10:15]

An important thing to keep in mind that the method is that this will include row #10 and exclude row #15.
Similarly, df[:] will returns all rows in the DataFrame.

Select rows based on one condition

In order to select all rows that meet a specific condition, we’ll make use of Pandas Boolean Index.

If reading through the documentation doesn’t make sense to you, don’t worry, an example will clarify the concept.

In the example below, we need to select all rows that have Cert lower than 500.

# Create a DataFrame from FDIC data
import pandas as pd
url = 'https://www.fdic.gov/resources/resolutions/bank-failures/failed-bank-list/'
df_list = pd.read_html(url)
df = df_list[0]
# Select based on condition
low_cert = df['CertCert'] <= 500
df[low_cert]

We’ll get the output we wanted:

Select rows based on multiple condition

We can combine unlimited number of boolean indexes to filter data from DataFrame quickly and easily.

Let’s suppose we want to get all failed bank that meet two criteria : has Cert higher than 5000 and based in New York.

# Create a DataFrame from FDIC data
import pandas as pd
url = 'https://www.fdic.gov/resources/resolutions/bank-failures/failed-bank-list/'
df_list = pd.read_html(url)
df = df_list[0]
# Select based on multiple condition
low_cert = df['CertCert'] > 5000
ny_based = df['CityCity'] == "New York"
df[low_cert & ny_based]

And voila, the result comes.

Summary

• df.head(n) returns first n rows from a DataFrame.
• df.tail(n) returns last n rows from a DataFrame.
• df[x:y] returns rows from #x to #(y-1) (row number y is not included).
• You can select rows based on multiple condition using boolean indexes.
• df[criteria] with criteria being a boolean variable returns rows that met the criteria.

Author: Thijmen I’m currently a SysAdmin located in the Netherlands. Every day I try to keep around a hundred users happy with their network connections and overall, tech-related issues. I also spend my spare time fiddling with web-based applications.

See also  Google Colab Introduction