Python Metaclasses

Posted on Edited on In Views:

In Python, a metaclass creates and defines the behavior of other classes, since classes themselves are objects.

Sources:

  1. Demystifying Python Metaclasses: Understanding and Harnessing the Power of Custom Class Creation
  2. Python Metaclasses by John Sturtz.

Types and classes

In Python 3, the definitions of types and a classes are euqal. So we can refer to an object’s type and its class interchangeably.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
n = 5
d = { 'x' : 1, 'y' : 2 }

class Foo:
	pass

x = Foo()

for obj in (n, d, x):
	print(type(obj) is obj.__class__)
# Output:
# True
# True
# True

The type of the built-in classes you are familiar with is also type:

1
2
3
4
5
6
7
8
>>> for t in int, float, dict, list, tuple:
...     print(type(t))
...
<class 'type'>
<class 'type'>
<class 'type'>
<class 'type'>
<class 'type'>

For that matter, the type of type is type as well (yes, really):

​

1
2
>>> type(type)
<class 'type'>

type is a metaclass, of which classes are instances.

In the above case:

  • x is an instance of class Foo.
  • Foo is an instance of the type metaclass.
  • type is also an instance of the type metaclass, so it is an instance of itself.
Python class chain

Defining Metaclasses

In order to define a custom metaclass, you need to define the following methods:

  1. __new__: This method is responsible for creating and returning the new class object. It takes four arguments:
    1. cls, the metaclass;
    2. name, the name of the new class being created;
    3. bases, a tuple of base classes for the new class; and
    4. attrs, a dictionary of attributes for the new class. You can use this method to customize the creation of new classes, by modifying the attrs dictionary or performing other custom processing.
  2. __init__: This method is responsible for initializing the new class object after it has been created by __new__. It takes the same arguments as __new__. You can use this method to perform any additional initialization that is required, such as setting default attribute values or performing other custom processing.

Here is an example of a custom metaclass definition that implements both __new__ and __init__ methods:

1
2
3
4
5
6
7
8
class MyMeta(type):
    def __new__(cls, name, bases, attrs):
        # customize the creation of new classes here...
        return super().__new__(cls, name, bases, attrs)

    def __init__(self, name, bases, attrs):
        # perform any additional initialization here...
        super().__init__(name, bases, attrs)

If you do not define the __init__ method in your custom metaclass, the default __init__ method of the type metaclass will be used instead. This default method does not do anything besides calling super().__init__(...) with the same arguments, so if you do not need any additional initialization logic in your metaclass, it is safe to omit the __init__ method.

Use Cases of Metaclasses

One common use case for metaclasses is to add additional methods or attributes to classes that are created with a certain metaclass.

This code example shows how metaclasses can be used to add additional methods or attributes to classes:

1
2
3
4
5
6
7
8
9
10
11
12
class MyMeta(type):
    def __new__(cls, name, bases, attrs):
        attrs['new_attribute'] = 'Hello, World!'
        attrs['new_method'] = lambda self: 'Hello from a new method!'
        return super().__new__(cls, name, bases, attrs)

class MyClass(metaclass=MyMeta):
    pass

obj = MyClass()
print(obj.new_attribute)  # Output: 'Hello, World!'
print(obj.new_method())   # Output: 'Hello from a new method!'

In this example, the MyMeta metaclass adds a new_attribute and a new_method to any class that uses it as a metaclass.

Enforcing constraints on the creation of classes

1
2
3
4
5
6
7
8
9
10
11
class MyMeta(type):
    def __new__(cls, name, bases, attrs):
        if 'required_attribute' not in attrs:
            raise TypeError('Class must define required_attribute')
        return super().__new__(cls, name, bases, attrs)

class MyClass(metaclass=MyMeta):
    required_attribute = 'some value'

class MyOtherClass(metaclass=MyMeta):
    pass  # Raises TypeError: Class must define required_attribute

In this example, the MyMeta metaclass raises a TypeError if a class created with it as a metaclass does not define required_attribute.

Implementing domain-specific languages (DSLs)

Just like how translators take words and phrases from one language and convert them to another language, metaclasses can take a custom syntax specific to a domain and convert it into Python syntax that can be executed. This enables developers to create a DSL specific to their application domain, making it easier to express concepts and perform operations.

This is how a DSL can be implemented using metaclasses in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
class DomainSpecificLanguage(type):
    def __new__(cls, name, bases, attrs):
        # Find all methods starting with "when_" and store them in a dictionary
        events = {k: v for k, v in attrs.items() if k.startswith("when_")}
        
        # Create a new class that will be returned by this metaclass
        new_cls = super().__new__(cls, name, bases, attrs)
        
        # Define a new method that will be added to the class
        def listen(self, event):
            if event in events:
                events[event](self)
        
        # Add the new method to the class
        new_cls.listen = listen
        
        return new_cls

# Define a class using the DSL syntax
class MyDSLClass(metaclass=DomainSpecificLanguage):
    def when_hello(self):
        print("Hello!")

    def when_goodbye(self):
        print("Goodbye!")

# Use the DSL syntax to listen for events
obj = MyDSLClass()
obj.listen("hello")  # Output: "Hello!"
obj.listen("goodbye")  # Output: "Goodbye!"

In this example, we define a new metaclass DomainSpecificLanguage that looks for methods in the class that start with "when_". These methods represent event handlers that will be triggered when a corresponding event is received.

The metaclass creates a new method called listen that can be used to listen for events and trigger the corresponding event handlers. This method is added to the class using the new_cls.listen = listen syntax.

Finally, we define a new class called MyDSLClass using the metaclass syntax. This class includes two event handlers: when_hello and when_goodbye. We can use the listen method to trigger these event handlers by passing in the name of the event we want to trigger.

Adding functionality to classes based on decorators or other annotations

Metaclasses can add additional functionality to classes that have been decorated1 or annotated in a certain way. This enables developers to customize the behavior of classes beyond what is possible with regular class definitions.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
class MyMeta(type):
    def __init__(cls, name, bases, attrs):
        for name, attr in attrs.items():
            if hasattr(attr, 'my_decorator'):
                # Decorate the method with some additional functionality...
                decorated_method = attr.my_decorator(attr)
                setattr(cls, name, decorated_method)
        return super().__init__(name, bases, attrs)

class MyClass(metaclass=MyMeta):
    @my_decorator
    def my_method(self):
        pass

def my_decorator(method):
    def decorated_method(self):
        # Add some additional functionality here...
        return method(self)
    decorated_method.my_decorator = True
    return decorated_method

In this example, the MyMeta metaclass looks for methods decorated with the @my_decorator decorator and adds some additional functionality to them.

  1. For python decorators, see my blog post.↩︎