The first time a spreadsheet let me down, it was a file of about two million rows that Excel simply refused to open past its row limit. I needed the total revenue per region, a five second answer, and my tool could not even load the data. That is the moment most analysts meet pandas. It is the Python library that reads a file far too big for a spreadsheet, filters and groups it in a line or two, and hands you the answer in a fraction of a second.

You already know the ideas underneath pandas, even if the syntax is new. You filtered rows with WHERE in Part 6, summarised with GROUP BY in Part 8, and cleaned missing values in Part 9. pandas does all three, in code, on data that lives in memory instead of a database. This part gives you the handful of moves that cover most of the work: load a file, look at it, pick the rows and columns you want, group them, and fix the gaps.

What a DataFrame actually is

A DataFrame is a table held in your computer memory, with named columns and numbered rows, that you shape by writing code instead of clicking cells. Picture the spreadsheet you already know, then imagine driving it with typed instructions rather than a mouse. Each column of a DataFrame is a Series, which is a single labelled list of values that all share a type, numbers in one column, text in another, dates in a third. The DataFrame is the sheet; the Series is one column of it. Almost everything you do in pandas is either an operation on a whole DataFrame or an operation on one of its Series.

Every DataFrame carries a hidden column on the left called the index, the row labels pandas uses to identify each row. When you load a file, the index is just the row numbers 0, 1, 2 and up, and for your first weeks that is all it needs to be. The important habit is to picture the shape: rows down the side, named columns across the top, and a value where each row meets each column. Hold that picture and the code below stops being magic words and becomes a set of instructions about a grid you can already see in your head.

Load a file into a DataFrame

Almost every pandas session starts by reading a file, and for a CSV, the plain text table format you met in Part 5, the one function you need is read_csv. You import the library once, give it the customary short name pd, then point read_csv at your file and it hands back a DataFrame. From there, head shows you the first few rows so you can confirm the load worked before you do anything else. Here is the sample sales file this part uses, six rows of daily sales with one deliberate gap in the units column.

The sales.csv table used throughout this part. One units value is missing on purpose, so the cleaning section later has something real to fix.

import pandas as pd

sales = pd.read_csv('sales.csv')
sales.head()

Result: the first five rows of the table above, printed with the index 0 to 4 down the left. head defaults to five rows; pass a number, such as head(3), for fewer.

Common failure: a FileNotFoundError means pandas cannot find sales.csv where your code is running. The path is relative to your working folder, not to where the file sits in a file browser. Put the CSV in the same folder as your notebook, or pass the full path, and the error clears.

Look before you touch

Before you filter or sum anything, spend thirty seconds learning what you loaded. Three methods do this. head shows the first rows so you can eyeball the actual values. info reports the column names, how many non-null values each holds, and the type pandas gave each one, which is how you catch a number column that loaded as text. describe prints quick statistics, count, mean, min, max and the quartiles, for every numeric column at once, the same summary figures you met in Part 11. Running these three first is the habit that separates analysts who trust their data from those who get surprised by it later.

sales.info()
sales.describe()

Result: info lists five columns and reports units as having 5 non-null values out of 6, flagging the gap. describe shows revenue with count 6, mean about 1883, min 900 and max 4000; units with count 5, because the missing value is skipped.

Common failure: reading describe as if every column has the same count. units shows count 5 while revenue shows 6, and that single mismatch is your early warning that a value is missing. If you skip info and describe and jump straight to a sum, the gap stays invisible until a total looks wrong.

One pandas 3.0 detail is worth knowing here, because it changes what info prints. From version 3.0, text columns load as a dedicated string type called str rather than the older generic object type, which is faster and clearer to read. If you learned pandas on an older version or follow an older tutorial, you will see object where you now see str; the behaviour you care about is the same, but the label is cleaner.

Pick columns and filter rows

Selecting one column is the simplest move: name it in square brackets and pandas hands back that Series. Ask for sales revenue with sales followed by the column name in brackets and you get the six revenue values on their own. Selecting several columns takes a list of names, and because a list itself uses brackets, you end up with two sets, one for the selection and one for the list inside it. That doubled bracket confuses everyone once, so name it now: the outer brackets mean select, the inner brackets mean here is a list of columns.

Filtering rows is where your SQL WHERE knowledge pays off directly. You write a condition that produces a column of true and false values, then hand that to the DataFrame to keep only the true rows. The clearest way to do this is loc, the label based selector, which takes a row condition and, optionally, the columns you want. Asking for the rows where region equals North is one line, and it reads almost like the English sentence behind it.

# one column is a Series
sales['revenue']

# rows where region is North
north = sales.loc[sales['region'] == 'North']

# combine conditions with & for and, | for or
big_north = sales.loc[(sales['region'] == 'North') & (sales['revenue'] > 1500)]

Result: north holds the two North rows, revenue 2400 and 1000. big_north holds just the first, because only its revenue clears 1500.

Common failure: joining two conditions with the words and or or, as you would in plain Python. Inside a pandas filter you must use the symbols & and |, and each condition needs its own parentheses, or the comparison order goes wrong and pandas raises an error. Write it as (condition) & (condition), every time.

Summarise with groupby

groupby is the pandas version of the GROUP BY you learned in Part 8, and it works the same way in three beats: split the rows into groups by a column, apply an aggregate to each group, then combine the answers into one small result. To get total revenue per region you group by region, pick the revenue column, and call sum. pandas returns one row per region, indexed by the region name, exactly the summary a manager wants.

# total revenue per region
sales.groupby('region')['revenue'].sum()

# several aggregates at once
sales.groupby('region').agg(
    total_revenue=('revenue', 'sum'),
    orders=('revenue', 'count'),
    total_units=('units', 'sum'),
)

Result: the first line returns North 3400, South 2500, West 5400. The agg block adds an orders count and total_units per region, with South units summing to 8 because the missing value is skipped.

Common failure: forgetting that sum, like SQL, skips missing values rather than treating them as zero. South units reads 8, not some larger number, because the gap on 2026-06-03 is ignored. If you need the gap counted, fill it first, which the next section covers.

The grouped result from the agg block, sorted by revenue. South units reads 8 rather than higher because the missing value never becomes a zero.

Handle the missing values

Real data has gaps, and pandas marks a missing value as NaN, which stands for not a number. You met the idea in Part 9; here is how you act on it in code. First you find the gaps with isna, which turns the DataFrame into true and false and lets you count how many are missing per column. Then you make a decision, and there are only two honest choices: fill the gap with a value, using fillna, or drop the rows that carry it, using dropna. Which one is right depends on the column and the question, and that judgement matters more than the syntax.

# how many gaps per column
sales.isna().sum()

# option 1: fill the missing units with 0
filled = sales.fillna({'units': 0})

# option 2: drop rows missing a units value
kept = sales.dropna(subset=['units'])

Result: isna().sum() reports units 1 and every other column 0. filled keeps all six rows and turns the gap into 0. kept drops the 2026-06-03 South row, leaving five rows.

Common failure: filling a missing number with 0 out of habit. A zero is a real claim that zero units sold, which can be wrong and will drag an average down. If you do not know the value, dropping the row or filling with the column median is often more honest than a reflexive 0. Choose the fill to match what the gap actually means.

The five moves as one workflow

Put the pieces in order and you have the loop nearly every pandas task follows. Read the file, look at it, clean the gaps, filter to the rows that matter, group to a summary, and write the answer back out to a file or a chart. You will repeat this loop on data of every size, and the moves stay the same whether the file has sixty rows or sixty million. The diagram below is the path from a raw file to an answer someone can use.

flowchart LR
  A[read_csv loads the file] --> B[head, info, describe to inspect]
  B --> C[fillna or dropna to fix gaps]
  C --> D[loc filters the rows]
  D --> E[groupby summarises]
  E --> F[export to CSV or chart]

Because so much of this maps onto SQL, the fastest way to learn pandas when you already know queries is to translate. The table below lines up the query clauses you know against their pandas equivalents, so a filter, a projection, a sort and a group each have a familiar anchor. Keep it beside you for a week and the new syntax stops feeling foreign.

The same operations in both languages. Learning pandas as a translation of SQL you already know is faster than learning it cold.

Learn these five moves before anything fancier

pandas is a large library, and it is tempting to chase every clever method you see online. Resist that for now. Read a file, inspect it, filter with loc, summarise with groupby, and handle gaps with fillna or dropna. Those five moves answer the majority of questions a working analyst faces in the first year, and every fancier technique, merging tables, pivoting, time series, sits on top of this same foundation. Get these fluent, to the point where you type them without looking them up, and the rest of pandas becomes a series of small additions rather than a wall.

You can now take a file too big for a spreadsheet and pull a clean summary from it in a few lines. Part 16 keeps you in Python and turns those numbers into charts, drawing the same groupby result you just built as a bar and a line with Matplotlib, so your analysis stops being a table in a terminal and becomes a picture you can share. Keep the Data Analyst guide open as your map, and revisit Part 8 if the grouping logic under groupby needs a refresher.

This week, open a free Colab notebook, type the six rows above into a small CSV, and run the whole loop yourself: read it, call info and describe, filter to one region, group revenue by region, and fill the missing units. Doing it once by hand fixes the five moves in memory far better than reading them ever will.

References

• 10 minutes to pandas, pandas Documentation
• pandas.read_csv Reference, pandas Documentation
• pandas 3.0 released, pandas Blog

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts