Performance & Profiling

Slow code lies to your instincts. The line that looks expensive may be harmless, and the tiny helper you forgot about may run 20,000 times. Performance work means measuring before changing, so you fix the part that actually costs something.

Measure before you guess

Performance is how much time, memory, or other computer work a program needs. Optimization is changing code so it uses less of that cost. The professional order is boring and powerful: make it correct, measure it, then improve the measured hot spot.

A hot spot is the part of a program that costs enough to matter. Think of a profiler as an itemized receipt for a meal: you do not argue about the total by staring at the menu; you look at which items made the bill large.

Tiny timings are noisy, so you usually repeat the work many times. Python's timeit module is a standard-library tool for timing small pieces of code repeatedly.

import timeit
 
waitlist = [f"player-{number}" for number in range(1000)]
waitlist.append("Noor")
waitset = set(waitlist)
 
 
def find_noor_in_list():
    return "Noor" in waitlist
 
 
def find_noor_in_set():
    return "Noor" in waitset
 
 
list_time = timeit.timeit(find_noor_in_list, number=1000)
set_time = timeit.timeit(find_noor_in_set, number=1000)
 
print("list result:", find_noor_in_list())
print("set result:", find_noor_in_set())
print("both measured:", list_time > 0 and set_time > 0)

list result: True
set result: True
both measured: True

The examples below avoid exact seconds because timing varies by machine and browser run. In your own REPL, inspect list_time and set_time. The key habit is not "memorize this exact number." The habit is "measure the two choices under the same conditions."

Use cProfile for whole functions

Profiling is measuring where a running program spends its work. A profiler is a tool that records that evidence while the program runs. timeit answers "which tiny expression is faster?" A profiler answers "where did this whole task spend its time?"

Python ships with cProfile, a standard-library profiler that counts function calls and time spent in each function. You can run it from code or from the terminal.

import cProfile
import pstats
 
 
def badge_label(score):
    if score >= 90:
        return "badge earned"
    return "keep practicing"
 
 
def build_badge_report(scores):
    report = []
    for player, score in scores:
        report.append(f"{player}: {badge_label(score)}")
    return report
 
 
scores = [("Mina", 94), ("Ravi", 72), ("Noor", 88)] * 50
 
profiler = cProfile.Profile()
profiler.enable()
report = build_badge_report(scores)
profiler.disable()
 
stats = pstats.Stats(profiler)
function_names = {key[2] for key in stats.stats}
 
print(report[:3])
print("profile saw badge_label:", "badge_label" in function_names)
print("profile saw build_badge_report:", "build_badge_report" in function_names)

['Mina: badge earned', 'Ravi: keep practicing', 'Noor: keep practicing']
profile saw badge_label: True
profile saw build_badge_report: True

pstats is the standard-library helper module for reading cProfile data. This example checks the recorded function names through stats.stats instead of printing timing columns, because timing numbers change by machine and run.

In a terminal, you can also run a script like this:

$ python -m cProfile report.py

cProfile usually shows columns like ncalls, tottime, and cumtime. ncalls is how many times a function ran. tottime is time spent inside that function's own body. cumtime is cumulative time: the function plus the functions it called.

Start by sorting for the largest repeated cost, not the longest-looking source line. A slow program is often death by many small calls.

Try it - compare two lookup shapes

Run this, then change checked_in_set = set(checked_in) to checked_in_set = checked_in. The output stays the same. The performance shape changes because list membership may scan through the row, while set membership uses the set's guest-list-at-the-door shape.

Big-O in practice

You met Big-O as a growth forecast, not a stopwatch. It describes how work grows as input grows. That matters more than one timing run when your data changes size.

Common practical shapes:

• O(1) means roughly steady work, like checking membership in a set.
• O(n) means work grows with the number of items, like scanning a list.
• O(n^2) means nested growth, often from comparing every item with every other item.

Do not use Big-O as a costume for guessing. Use it to choose what to measure. If a list has ten names, an O(n) scan is fine. If it has ten million names and runs on every request, the shape matters.

Here are common Python performance pitfalls and easy wins:

• Repeated list membership in a loop -> build a set once, then use in.
• Rebuilding the same sorted list inside a loop -> sort once before the loop.
• Recomputing a value that never changes -> move it outside the loop.
• Building long strings with repeated + in a loop -> collect pieces and use "".join(...).
• Optimizing code that runs once at startup -> probably leave it readable.

checked_in = ["Mina", "Ravi", "Noor", "Ada"]
late_players = ["Noor", "Isha", "Mina"]
 
checked_in_set = set(checked_in)
 
for player in late_players:
    if player in checked_in_set:
        print(f"{player}: already checked in")
    else:
        print(f"{player}: needs desk help")

Noor: already checked in
Isha: needs desk help
Mina: already checked in

The easy win is not that sets are always better. Lists keep order and are perfect for many jobs. The win is choosing the data structure that matches the operation you repeat.

Where line_profiler fits

line_profiler is a third-party profiler that can show timing one source line at a time. Third-party means it is not in the standard library; it comes from PyPI, like the external packages you met in pip and virtual environments. Do not install it here. Just know the shape.

With line_profiler, you usually mark a function and run a command-line tool:

@profile
def build_badge_report(scores):
    report = []
    for player, score in scores:
        report.append(f"{player}: {badge_label(score)}")
    return report

Then the command workflow usually has this shape. kernprof is the command-line runner that line_profiler provides after installation.

$ kernprof -l -v report.py
Timer unit: <machine-dependent>
 
Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     2         1       <time>   <time>    <pct>  def build_badge_report(scores):
     3         1       <time>   <time>    <pct>      report = []
     4       150       <time>   <time>    <pct>      for player, score in scores:
     5       150       <time>   <time>    <pct>          report.append(f"{player}: {badge_label(score)}")
     6         1       <time>   <time>    <pct>      return report

The output points at individual lines inside the function. The <time> and <pct> placeholders stand in for machine-dependent timing values. Use line_profiler after cProfile tells you which function deserves attention. That order keeps you from measuring every line in a program when only one function is expensive.

A practical performance loop

Performance work has the same discipline as debugging and logging:

1. Name the user-visible problem: "The badge report takes too long for 50,000 scores."
2. Measure the current version with timeit or cProfile.
3. Find the hot spot.
4. Change one thing.
5. Measure again with the same setup.
6. Keep the clearer version unless the faster version wins enough to matter.

You now have the full Section 15 arc: find the bug, leave useful runtime evidence, then measure cost before changing code. Next, you will use that same discipline to prove behavior with tests instead of checking everything by hand.