The Data Workflow

Real data rarely arrives like a neat textbook table. It arrives with extra spaces, missing snacks, one score left blank, and someone typing "thirty" where a number should be. A data workflow is the repeatable path from raw data, the untouched input you start with, to a checked answer: inspect, clean, wrangle, explore, and explain.

Start by inspecting the mess

Cleaning means fixing values that are missing, inconsistent, or stored as the wrong type. Wrangling means reshaping data into the form your question needs. Think of the workflow like a kitchen prep line: wash the ingredients, trim the rough parts, arrange them for cooking, then taste before serving.

Here is the kind of tiny mess that scales up to real projects:

from io import StringIO
import pandas as pd
 
messy_csv = """player,team,score,minutes,snack
Mina, Gold ,82,31,apple
Ravi,Blue,91,28,
Noor,gold,,24,tea
Isha,Blue,88,thirty,apple
Sam,Gold,72,22,
"""
 
raw = pd.read_csv(StringIO(messy_csv))
 
print(raw.to_string(index=False))
print()
print(raw.isna().sum().to_string())

player team score minutes snack
Mina Gold 82.0 31 apple
Ravi Blue 91.0 28 NaN
Noor gold NaN 24 tea
Isha Blue 88.0 thirty apple
Sam Gold 72.0 22 NaN

player 0
team 0
score 1
minutes 0
snack 2

You already know NaN from pandas: it is pandas' printed sign for a missing value. The count at the bottom tells you where the empty seats are. It does not catch "thirty" yet, because that is present text in the minutes column, not an empty cell.

Clean and wrangle with a rule you can explain

Good cleaning is not "make the warning go away." It is choosing rules you would be willing to say out loud.

• Team names should have consistent spacing and capitalization.
• Missing snacks should become "no snack" because that is a real category here.
• Scores should be numeric; a missing score can use the overall sheet median, the middle value after sorting, or the average of the two middle values when there is an even count.
• Minutes should be numeric; a row with invalid minutes can stay in score summaries but should be set aside for rate calculations.

The .str accessor lets pandas apply string methods down a whole column. Coercion means converting a value into another type when possible; here, errors="coerce" turns impossible numeric values into missing values so you can handle them deliberately.

from io import StringIO
import pandas as pd
 
messy_csv = """player,team,score,minutes,snack
Mina, Gold ,82,31,apple
Ravi,Blue,91,28,
Noor,gold,,24,tea
Isha,Blue,88,thirty,apple
Sam,Gold,72,22,
"""
 
raw = pd.read_csv(StringIO(messy_csv))
clean = raw.copy()
 
clean["team"] = clean["team"].str.strip().str.title()
clean["snack"] = clean["snack"].fillna("no snack")
clean["score"] = pd.to_numeric(clean["score"], errors="coerce")
clean["minutes"] = pd.to_numeric(clean["minutes"], errors="coerce")
clean["score"] = clean["score"].fillna(clean["score"].median())
rate_data = clean.dropna(subset=["minutes"]).copy()
rate_data["points_per_minute"] = rate_data["score"] / rate_data["minutes"]
 
print(rate_data[["player", "team", "score", "minutes", "snack", "points_per_minute"]].round(2).to_string(index=False))

player team score minutes snack points_per_minute
Mina Gold 82.0 31.0 apple 2.65
Ravi Blue 91.0 28.0 no snack 3.25
Noor Gold 85.0 24.0 tea 3.54
Sam Gold 72.0 22.0 no snack 3.27

Notice what happened to Isha. Her score was fine, but her minutes were not usable for a points-per-minute calculation. Setting that row aside in rate_data keeps the rate honest without deleting her score from every later summary.

Try it - run the workflow loop

The first time you run this playground, the browser downloads pandas and Matplotlib once. After that, the packages stay loaded for this page, and each Run still starts your own code fresh.

Change one raw value, rerun, and watch both the summary and chart respond. This is the loop: inspect a change, clean by rule, explore again.

Explore before you claim

Exploratory data analysis (EDA) is the first pass of summarizing and plotting data before making claims. EDA is not proof. It is how you find better questions.

Start with small checks:

• How many rows and columns do you have?
• Which columns still have missing values?
• What are the group summaries?
• Does a chart show an outlier, a value far from the rest?

from io import StringIO
import pandas as pd
 
messy_csv = """player,team,score,minutes,snack
Mina, Gold ,82,31,apple
Ravi,Blue,91,28,
Noor,gold,,24,tea
Isha,Blue,88,thirty,apple
Sam,Gold,72,22,
"""
 
raw = pd.read_csv(StringIO(messy_csv))
clean = raw.copy()
clean["team"] = clean["team"].str.strip().str.title()
clean["snack"] = clean["snack"].fillna("no snack")
clean["score"] = pd.to_numeric(clean["score"], errors="coerce")
clean["minutes"] = pd.to_numeric(clean["minutes"], errors="coerce")
clean["score"] = clean["score"].fillna(clean["score"].median())
 
team_summary = (
    clean.groupby("team")
    .agg(
        players=("player", "count"),
        average_score=("score", "mean"),
        average_minutes=("minutes", "mean"),
    )
    .round(1)
    .reset_index()
)
 
print("rows, columns:", clean.shape)
print(team_summary.to_string(index=False))

rows, columns: (5, 5)
team players average_score average_minutes
Blue 2 89.5 28.0
Gold 3 79.7 25.7

Now add a chart. The chart does not replace the table; it gives your eyes another angle on the same cleaned data.

from io import StringIO
import pandas as pd
import matplotlib.pyplot as plt
 
messy_csv = """player,team,score,minutes,snack
Mina, Gold ,82,31,apple
Ravi,Blue,91,28,
Noor,gold,,24,tea
Isha,Blue,88,thirty,apple
Sam,Gold,72,22,
"""
 
raw = pd.read_csv(StringIO(messy_csv))
clean = raw.copy()
clean["team"] = clean["team"].str.strip().str.title()
clean["score"] = pd.to_numeric(clean["score"], errors="coerce")
clean["minutes"] = pd.to_numeric(clean["minutes"], errors="coerce")
clean["score"] = clean["score"].fillna(clean["score"].median())
 
plt.hist(clean["score"], bins=4, color="slateblue")
plt.title("Cleaned score distribution")
plt.xlabel("Score range")
plt.ylabel("Player count")
plt.show()

The moment you see the distribution, you can ask sharper follow-ups: Are scores clustered? Is one team carrying the average? Did cleaning rules change the story?

Work effectively in Jupyter

A Jupyter Notebook is an interactive document where code, output, charts, and notes live together. A cell is one runnable block in a notebook. A kernel is the Python process that runs those cells and remembers names between runs.

That memory is useful and dangerous. If you run cells out of order, your notebook can depend on a name that only exists because you ran an old cell yesterday. Keep notebooks honest with a small rhythm:

1. Question
2. Imports and raw data loading
3. Inspection
4. Cleaning rules
5. EDA summaries and charts
6. Short conclusion

Use markdown cells, notebook cells that hold formatted notes instead of Python code, to write the question and the cleaning rules in plain language. Restart the kernel and run all cells before trusting the result. A reproducible analysis can be rerun from the beginning and reach the same result; if the notebook cannot run top to bottom, it is not there yet.

When the work grows past exploration, move stable cleaning code into a .py file and keep the notebook as the lab notebook: focused question, visible outputs, and a short conclusion.

Section 21 gave you the working loop: arrays, tables, charts, and a clean analysis habit. Next, databases show where larger data lives and how SQL asks for just the rows your Python workflow needs.