pandas

A CSV file looks flat until you ask real questions of it: "Which team improved?", "Who is missing a snack?", "Can I attach badges from another sheet?" pandas gives Python spreadsheet-shaped thinking: labeled rows and columns, fast selection, grouped summaries, joins, and missing-data cleanup.

Series and DataFrame

pandas is a third-party package for table-shaped data. By convention, Python developers import it as pd.

A Series is one labeled column of data. A DataFrame is a table of labeled columns, like a spreadsheet with Python methods clipped onto it. An index is the row label pandas uses to line values up; it may be player names, dates, or plain row numbers.

import pandas as pd
 
scores = pd.Series([82, 91, 77], index=["Mina", "Ravi", "Noor"], name="score")
players = pd.DataFrame({
    "player": ["Mina", "Ravi", "Noor"],
    "mood": ["steady", "focused", "curious"],
    "score": [82, 91, 77],
})
 
print(scores.to_string())
print()
print(players.to_string(index=False))

Mina 82
Ravi 91
Noor 77

player mood score
Mina steady 82
Ravi focused 91
Noor curious 77

The Series is useful when one column is the whole story. The DataFrame is what you reach for when each row has several facts: player, mood, score, minutes, team, date.

Under the surface, pandas builds on the rectangular-array ideas from NumPy. The big difference is labels. NumPy is excellent when position is enough; pandas shines when column names and row labels carry meaning.

Loading CSV, Excel, and SQL-shaped data

read_csv() loads a CSV, a spreadsheet saved as plain text. Here, StringIO is a file-like wrapper around text, so pandas can read a string as if it were a small file.

from io import StringIO
import sqlite3
import pandas as pd
 
csv_text = """player,team,score,minutes
Mina,Gold,82,31
Ravi,Blue,91,28
Noor,Gold,77,24
"""
 
practice = pd.read_csv(StringIO(csv_text))
print(practice.to_string(index=False))
 
connection = sqlite3.connect(":memory:")
practice.to_sql("practice", connection, index=False, if_exists="replace")
gold_scores = pd.read_sql_query(
    "SELECT player, score FROM practice WHERE team = 'Gold'",
    connection,
)
print(gold_scores.to_string(index=False))
connection.close()

player team score minutes
Mina Gold 82 31
Ravi Blue 91 28
Noor Gold 77 24
player score
Mina 82
Noor 77

SQL is a language for asking questions of a database, software that stores tables for querying; databases get their full treatment in lesson 22.1. The key pandas idea is the same: read_sql_query() returns a DataFrame.

Excel has the same loading shape, but real .xlsx files usually need an extra Excel reader package in your project environment. The line looks like this on your own machine:

pd.read_excel("practice.xlsx", sheet_name="July")

Once data is a DataFrame, pandas does not care much whether it came from CSV, Excel, or SQL. You work with rows and columns.

Try it - select, filter, transform

The first time you run a pandas playground in this lesson, the browser downloads pandas once. After that, the package stays loaded for this page, and each Run still starts your own code fresh.

Selecting means choosing columns or rows. Filtering means keeping rows that pass a yes-or-no test. A Boolean mask is a column of True and False answers that pandas uses to decide which rows stay.

That new column is a transformation, a change that creates a more useful version of the data. You did not loop through rows by hand. pandas lined up the score and minutes columns, divided them value by value, and stored the result.

Here is the same selecting and filtering idea as a regular code example:

import pandas as pd
 
players = pd.DataFrame({
    "player": ["Mina", "Ravi", "Noor", "Isha"],
    "team": ["Gold", "Blue", "Gold", "Blue"],
    "score": [82, 91, 77, 88],
    "minutes": [31, 28, 24, 30],
})
 
strong_scores = players[players["score"] >= 85]
players["points_per_minute"] = players["score"] / players["minutes"]
 
print(players[["player", "score"]].to_string(index=False))
print(strong_scores[["player", "score"]].to_string(index=False))
print(players[["player", "points_per_minute"]].round(2).to_string(index=False))

player score
Mina 82
Ravi 91
Noor 77
Isha 88
player score
Ravi 91
Isha 88
player points_per_minute
Mina 2.65
Ravi 3.25
Noor 3.21
Isha 2.93

The double brackets in players[["player", "score"]] matter: one pair selects from the DataFrame, and the inner list names several columns.

groupby and aggregation

A groupby operation splits rows into piles, runs a calculation on each pile, and puts the summaries back together. Think of sorting scorecards into team piles, then writing one average on each pile.

An aggregation is a many-to-one summary, such as mean score, best score, count, minimum, or maximum.

import pandas as pd
 
players = pd.DataFrame({
    "player": ["Mina", "Ravi", "Noor", "Isha"],
    "team": ["Gold", "Blue", "Gold", "Blue"],
    "score": [82, 91, 77, 88],
    "minutes": [31, 28, 24, 30],
})
 
summary = (
    players.groupby("team")
    .agg(
        average_score=("score", "mean"),
        best_score=("score", "max"),
        player_count=("player", "count"),
    )
    .round(1)
    .reset_index()
)
 
print(summary.to_string(index=False))

team average_score best_score player_count
Blue 89.5 91 2
Gold 79.5 82 2

The shape changes. Four player rows become two team rows. That is the point of aggregation: fewer rows, more meaning.

Merging and joining tables

A merge combines two DataFrames by matching values in one or more columns. A join is the same table-combining idea; in pandas, merge() is the clearest starter tool when the matching label is a column.

This is like clipping two rows of cards together by matching names, not by trusting that the positions line up.

A missing value is pandas' table version of an empty seat: the value should exist, but you do not have it. NaN, short for "not a number," is one way pandas prints missing values.

import pandas as pd
 
players = pd.DataFrame({
    "player": ["Mina", "Ravi", "Noor", "Isha"],
    "team": ["Gold", "Blue", "Gold", "Blue"],
    "score": [82, 91, 77, 88],
})
 
badges = pd.DataFrame({
    "player": ["Mina", "Ravi", "Isha"],
    "badge": ["helper", "finisher", "comeback"],
})
 
joined = players.merge(badges, on="player", how="left")
print(joined.to_string(index=False))

player team score badge
Mina Gold 82 helper
Ravi Blue 91 finisher
Noor Gold 77 NaN
Isha Blue 88 comeback

A left merge keeps every row from the left DataFrame, players here. Noor has no matching badge row, so pandas leaves the badge empty.

Handling missing data

Use isna() to count the empty seats, fillna() to replace them, and dropna() when removing incomplete rows is the right choice.

import pandas as pd
 
attendance = pd.DataFrame({
    "player": ["Mina", "Ravi", "Noor", "Isha"],
    "snack": ["apple", None, "tea", None],
    "score": [82, 91, None, 88],
})
 
print(attendance.isna().sum().to_string())
print()
 
filled = attendance.copy()
filled["snack"] = filled["snack"].fillna("no snack")
filled["score"] = filled["score"].fillna(filled["score"].mean()).round(1)
 
print(filled.to_string(index=False))

player 0
snack 2
score 1

player snack score
Mina apple 82.0
Ravi no snack 91.0
Noor tea 87.0
Isha no snack 88.0

Missing data is not a side issue in real analysis. It changes averages, filters, joins, and charts. Your job is to choose a rule you can explain: fill snack with "no snack", fill score with the current mean, or drop rows only when incomplete data would mislead the result.

pandas gives you the table moves: load, select, filter, transform, summarize, merge, and clean. Next, visualization turns those tables into pictures so patterns and outliers can hit your eyes before they hide in the numbers.