Descriptors, metaclasses, and class creation

What you'll learn

• Explain how attribute lookup works and how property is built on the descriptor protocol
• Write a descriptor: a reusable, validated attribute defined once and used on many classes
• Hook into class creation with __init_subclass__ for registries and checks
• Understand what a metaclass is, write a minimal one, and know why you rarely need to

Concept

When you write obj.x, Python does more than read a dictionary. It looks on the class for x, and if what it finds has a __get__ method, it calls that method instead of returning the object. Anything with __get__, and optionally __set__ and __delete__, is a descriptor, and this protocol is how property, classmethod, staticmethod, and ordinary methods work. A method is a function, and functions have __get__, which is what turns Class.method into a bound method with self filled in when accessed through an instance. Every piece of "magic" you have used around attributes is a descriptor.

Writing your own descriptor pays off when the same validated attribute appears on several classes. A Positive descriptor that checks value > 0 in __set__ and stores the value under a private name can be declared once and attached as price = Positive() and quantity = Positive() on any class. __set_name__ tells the descriptor which attribute name it was assigned to, so it can choose where to store the data. This is the mechanism behind ORM fields, dataclass field machinery, and validation libraries; when you see field = SomeType() at class level doing something clever, it is a descriptor.

Classes are created at runtime too, when the class statement finishes, and you can hook that moment. __init_subclass__ is a classmethod on a parent that runs each time a subclass is defined, receiving the new class; a base class can use it to register every plugin automatically, or to check that a subclass defined a required attribute, all with no metaclass. It covers most of what metaclasses used to be for and is the tool to reach for first.

A metaclass is the class of a class. type is the default: class Foo: calls type("Foo", bases, namespace) to build the class object. A custom metaclass subclasses type and overrides __new__ to inspect or change the namespace before the class exists, which is how frameworks turn a class body full of field declarations into a schema. Enum and ABC use metaclasses. You should recognize metaclass= when you read it and understand what it does; you should write one perhaps once in a career, after __init_subclass__ and descriptors have proved insufficient.

Code

The descriptor protocol behind things you already use. A function's __get__ binds self; property is a descriptor whose __get__ calls your getter; the manual calls prove it.

# behind_the_curtain.py
class Circle:
    def __init__(self, r):
        self.r = r

    def area(self):
        return 3.14159 * self.r ** 2

    @property
    def diameter(self):
        return 2 * self.r


c = Circle(2)
print(type(Circle.__dict__["area"]).__name__, type(Circle.__dict__["diameter"]).__name__)
bound = Circle.__dict__["area"].__get__(c, Circle)
print(bound() == c.area(), bound.__self__ is c)
print(Circle.__dict__["diameter"].__get__(c, Circle), c.diameter)
print(hasattr(Circle.__dict__["diameter"], "__set__"), "<- property is a data descriptor")

function property
True True
4 4
True <- property is a data descriptor

A reusable validated attribute. Positive is written once, attached to two classes, learns its own name with __set_name__, and rejects bad values on assignment and in __init__.

# descriptor.py
class Positive:
    def __set_name__(self, owner, name):
        self.public = name
        self.private = "_" + name

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return getattr(obj, self.private)

    def __set__(self, obj, value):
        if not isinstance(value, (int, float)) or value <= 0:
            raise ValueError(f"{self.public} must be a positive number, got {value!r}")
        setattr(obj, self.private, value)


class Product:
    price = Positive()
    quantity = Positive()

    def __init__(self, price, quantity):
        self.price = price
        self.quantity = quantity


class Payment:
    amount = Positive()

    def __init__(self, amount):
        self.amount = amount


p = Product(9.99, 3)
print(p.price, p.quantity, p.__dict__)
p.quantity = 5
print(p.quantity, Product.price.private)
Payment(-1)

9.99 3 {'_price': 9.99, '_quantity': 3}
5 _price
Traceback (most recent call last):
  File "descriptor.py", line 37, in <module>
    Payment(-1)
    ~~~~~~~^^^^
  File "descriptor.py", line 30, in __init__
    self.amount = amount
    ^^^^^^^^^^^
  File "descriptor.py", line 13, in __set__
    raise ValueError(f"{self.public} must be a positive number, got {value!r}")
ValueError: amount must be a positive number, got -1

__init_subclass__ as a registry and a contract: plugins register themselves by being defined, and a subclass missing a required attribute is refused at definition time.

# init_subclass.py
class Exporter:
    registry = {}

    def __init_subclass__(cls, *, format_name=None, **kwargs):
        super().__init_subclass__(**kwargs)
        if format_name is None:
            raise TypeError(f"{cls.__name__} must declare format_name=...")
        if not hasattr(cls, "render"):
            raise TypeError(f"{cls.__name__} must define render()")
        Exporter.registry[format_name] = cls


class Csv(Exporter, format_name="csv"):
    def render(self, rows):
        return "\n".join(",".join(map(str, r)) for r in rows)


class Json(Exporter, format_name="json"):
    def render(self, rows):
        import json
        return json.dumps(rows)


print(sorted(Exporter.registry))
print(Exporter.registry["csv"]().render([(1, 2), (3, 4)]))


class Broken(Exporter, format_name="xml"):
    pass

['csv', 'json']
1,2
3,4
Traceback (most recent call last):
  File "init_subclass.py", line 28, in <module>
    class Broken(Exporter, format_name="xml"):
        pass
  File "init_subclass.py", line 9, in __init_subclass__
    raise TypeError(f"{cls.__name__} must define render()")
TypeError: Broken must define render()

A minimal metaclass, and type used directly to build a class. The metaclass rewrites the namespace to collect field declarations into a schema before the class exists.

# metaclass.py
class Field:
    def __init__(self, kind):
        self.kind = kind


class ModelMeta(type):
    def __new__(mcls, name, bases, namespace):
        fields = {k: v.kind for k, v in namespace.items() if isinstance(v, Field)}
        for k in fields:
            namespace.pop(k)
        namespace["_schema"] = fields
        return super().__new__(mcls, name, bases, namespace)


class Model(metaclass=ModelMeta):
    def __init__(self, **values):
        for key, kind in self._schema.items():
            value = values.get(key)
            if not isinstance(value, kind):
                raise TypeError(f"{key} must be {kind.__name__}")
            setattr(self, key, value)


class User(Model):
    name = Field(str)
    age = Field(int)


u = User(name="ada", age=36)
print(User._schema, u.name, u.age, type(User).__name__)

Dynamic = type("Dynamic", (), {"greet": lambda self: "hi from a class built with type()"})
print(Dynamic().greet(), type(Dynamic) is type)

{'name': <class 'str'>, 'age': <class 'int'>} ada 36 ModelMeta
hi from a class built with type() True

Try it yourself

Write fields.py with a Typed(kind) descriptor that enforces a type on assignment and a Bounded(low, high) descriptor that enforces a range, both using __set_name__. Attach them to a Thermostat class and show valid and invalid assignments. Then write a Command base class whose __init_subclass__ requires a name class attribute and registers subclasses in a dictionary; define three commands and dispatch on the registry. Leave metaclasses alone unless you can say why the other two tools were not enough.

Common mistakes

• Storing descriptor data on the descriptor. There is one descriptor per class, not per instance; store per-instance values on the instance under a private name.
• Reaching for a metaclass first. __init_subclass__ handles registries and checks; descriptors handle attributes; a metaclass is for rewriting the class namespace itself.
• Forgetting super().__init_subclass__(**kwargs). Without it, cooperative hooks in other base classes are skipped.

What's next

• Attribute access consults descriptors on the class; property, methods, classmethod, and staticmethod are all descriptors.
• A descriptor with __set_name__, __get__, and __set__ is a reusable validated attribute; __init_subclass__ hooks subclass creation for registries and contracts.
• A metaclass is the class of a class and rewrites the namespace at creation; recognize it, use it last.

You have seen how classes are built. Next you will see how the interpreter runs the code inside them: from source to bytecode to the evaluation loop.