Functional tools

What you'll learn

• Write small anonymous functions with lambda and know when a def is better
• Use functions as values: pass them to sorted, max, map, and filter
• Fix arguments in place with functools.partial and cache results with functools.lru_cache
• Dispatch on argument type with functools.singledispatch and reduce with functools.reduce

Concept

Functions in Python are values. You can store one in a variable, put it in a list, pass it to another function, and return it from one; decorators and closures already relied on that. The functional tools are the built-ins and the functools module that make working with functions as values convenient. They do not replace loops and classes; they shorten the places where you would otherwise write a tiny function only to hand it to something.

lambda creates a function without a name in one expression: lambda x: x * 2. It has no statements, no docstring, and no name in tracebacks, which is why the rule is short: use a lambda when the function is trivial and used once, right where it is written, most often as a key=. The moment it needs a second line, a name, or a test, write a def. Assigning a lambda to a name, square = lambda x: x * x, is the worst of both worlds; write def square(x). lambda is one of a small family of tools that take or return functions, and the table beside it is the whole family: what each one does, and the plainer thing to reach for when it does not fit.

map and filter are the lazy ancestors of comprehensions: map(str.upper, words) yields each word upper-cased, filter(None, items) yields the truthy ones. Comprehensions are usually clearer, but map shines when you already have the function, especially a method like str.strip, and both are iterators you can feed to join or sum. key= functions for sorted, min, and max are where most lambdas live, and the operator module supplies ready-made ones: itemgetter("price") and attrgetter("name") beat lambda p: p["price"] for both speed and readability.

functools.partial takes a function and some of its arguments and returns a new function waiting for the rest: partial(int, base=2) is a binary parser. It is the right tool for adapting a function to an interface that calls with fewer arguments. functools.lru_cache (and its unbounded form cache) remembers the return value for each set of arguments, turning an expensive pure function, a recursive one, or a repeated lookup into a dictionary hit; it requires hashable arguments and a function whose result depends only on them.

functools.singledispatch turns a function into one that picks an implementation based on the type of its first argument, registered with @name.register(type). It is polymorphism for functions when you cannot or should not add methods to the classes involved, such as built-in types or classes from a library, and it replaces the isinstance chain the polymorphism lesson warned about. One call goes in, and the type of the first argument decides which registered implementation runs.

functools.reduce folds a sequence into one value with a two-argument function: it calls f(acc, item) for every item, carrying the running result forward. It exists, and you should know it, but sum, max, "".join, and a plain loop cover almost every case more readably.

Code

lambda in its natural habitat, key=, and operator doing the same job with a name. map and filter beside the comprehensions that usually replace them.

# keys_and_maps.py
from operator import itemgetter, attrgetter

products = [{"name": "pen", "price": 1.5}, {"name": "book", "price": 12.0}, {"name": "ink", "price": 6.0}]

print(sorted(products, key=lambda p: p["price"])[0]["name"])
print(max(products, key=itemgetter("price"))["name"])
print([p["name"] for p in sorted(products, key=itemgetter("name"))])

words = ["  Ada ", "grace", " LINUS"]
print(list(map(str.strip, words)))
print(list(map(str.upper, map(str.strip, words))))
print(list(filter(None, [0, "", "x", None, 3])))
print([w.strip() for w in words if len(w.strip()) > 3])

pen
book
['book', 'ink', 'pen']
['Ada', 'grace', 'LINUS']
['ADA', 'GRACE', 'LINUS']
['x', 3]
['grace', 'LINUS']

partial and lru_cache: a pre-filled function, and a recursive function that becomes fast because every result is remembered.

# partial_cache.py
import functools
import time

parse_binary = functools.partial(int, base=2)
print(parse_binary("1010"), parse_binary("1111"))

log_error = functools.partial(print, "[ERROR]", sep=" ")
log_error("disk full")


@functools.lru_cache(maxsize=None)
def fib(n):
    return n if n < 2 else fib(n - 1) + fib(n - 2)


start = time.perf_counter()
print(fib(80))
print("fast:", time.perf_counter() - start < 0.5)
print(fib.cache_info().hits > 0, fib.cache_info().currsize)

10 15
[ERROR] disk full
23416728348467685
fast: True
True 81

singledispatch replacing an isinstance chain, with a default for unregistered types.

# dispatch.py
from functools import singledispatch


@singledispatch
def describe(value):
    return f"something of type {type(value).__name__}"


@describe.register(int)
def _(value):
    return f"an integer, {'even' if value % 2 == 0 else 'odd'}"


@describe.register(str)
def _(value):
    return f"a string of {len(value)} characters"


@describe.register(list)
def _(value):
    return f"a list of {len(value)} items"


for thing in (42, "hello", [1, 2, 3], 3.5):
    print(describe(thing))

an integer, even
a string of 5 characters
a list of 3 items
something of type float

reduce next to the built-ins that usually replace it. Both compute the same values; the second column is what you should write.

# reduce.py
import functools
import operator

numbers = [3, 7, 2, 9]

print(functools.reduce(operator.add, numbers), "==", sum(numbers))
print(functools.reduce(lambda a, b: a if a > b else b, numbers), "==", max(numbers))
print(functools.reduce(operator.mul, numbers))
print(functools.reduce(lambda acc, s: acc + [s.upper()], ["a", "b"], []), "==", [s.upper() for s in ["a", "b"]])

21 == 21
9 == 9
378
['A', 'B'] == ['A', 'B']

Try it yourself

Create functional.py with a list of employee dictionaries with name, department, and salary. Sort them by department then salary descending using itemgetter, find the highest-paid person with max and a key, use map to produce a list of "name (department)" strings, and filter to keep salaries above a threshold. Then write a cached slow_square(n) that sleeps briefly and show the second call of the same argument is instant. Finally write a format_value with singledispatch for int, float, and str.

Common mistakes

• Naming a lambda. f = lambda x: ... gives a nameless function a name badly; use def.
• Caching impure functions. lru_cache on a function that reads a file or the clock returns stale answers forever; cache only functions whose result depends solely on their arguments.
• Reaching for reduce. If sum, max, join, or a three-line loop expresses it, use that; reduce is for the rare fold that nothing else fits.

What's next

• Functions are values; lambda makes trivial one-expression ones for key=, and operator.itemgetter often reads better still.
• map and filter are lazy; comprehensions usually replace them. partial pre-fills arguments and lru_cache memoizes pure functions.
• singledispatch routes by type without isinstance chains; know reduce, and prefer the built-ins.

Your function toolkit is complete. Next you will make the with statement work for your own resources, with context managers.