Login With Github

Create Classes Dynamically in Python

0x00 Introduction

Classes in Python are existed as objects, so you can create classes at runtime dynamically, which also shows the flexibility of Python.

I'll introduce how to use type to create classes dynamically as well as some related usage and techniques in this article.

0x01 The class

What is a class? A class is an abstraction for a kind of things which have common characteristics in real life, and it describes the properties and methods that are common to the objects being created. In common compiled languages ​​(such as C++), classes are defined at compile time and cannot be created at runtime dynamically. So how does Python do it?

Look at the following code:

class A(object):
    pass

print(A)
print(A.__class__)

The execution results in Python 2 are as follows:

<class '__main__.A'>
<type 'type'>

The execution results in Python 3 are as follows:

<class '__main__.A'>
<class 'type'>

It can be seen that the type of the class A is type, which means that the type will be instantiated as the class, and the class will be instantiated as the object.

0x02 Use type to create a class dynamically

The definition for the parameters of type are as follows:

Type(name, bases, dict)
name: the generated name of the class
bases: a list of generated base classes, and its type is tuple
dict: the properties or methods contained in the generated class

Let's say you want to create the class A , you can use the following method.

cls = type('A', (object,), {'__doc__': 'class created by type'})

print(cls)
print(cls.__doc__)

The output is as follows:

<class '__main__.A'>
class created by type

It can be seen that the class created in this way is nearly the same as the statically defined class, and the former is even more flexible in use.

One of the usage scenarios for the method is:

You may need to pass a class as a parameter to some places, and there are variables which can be affected by the outside to be used in the class; of course, you can use global variables to solve the problem,  but it looks ugly. So at this point you can use the method of creating a class dynamically.

Here is an example:

import socket
try:
    import SocketServer
except ImportError:
    # python3
    import socketserver as SocketServer

class PortForwardingRequestHandler(SocketServer.BaseRequestHandler):
    '''process the request of port forwarding
    '''

    def handle(self):
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        sock.connect(self.server) # self.server is passed in during the time of the classes being created dynamically
        # connect to the target server and forward the data
        # The following code is omitted...

def gen_cls(server):
    '''create the subclasses dynamically 
    '''
    return type('%s_%s' % (ProxyRequestHandler.__name__, server), (PortForwardingRequestHandler, object), {'server': server})


server = SocketServer.ThreadingTCPServer(('127.0.0.1', 8080), gen_cls(('www.tutorialdocs.com', 80)))
server.serve_forever()

In the above example, since the target server address is passed in by the user, and the instantiation of the PortForwardingRequestHandler class is implemented in the ThreadingTCPServer, we can't control them. Therefore, using the method of dynamically creating a class can solve the problem very well.

0x03 Use the metaclass

A class is a template for an instance, and a metaclass is a template for a class. Classes can be created by metaclasses, and the default metaclass of a class is type, so all metaclasses must be the subclasses of type.

Here is an example for a metaclass:

import struct

class MetaClass(type):
    def __init__(cls, name, bases, attrd):
        super(MetaClass, cls).__init__(name, bases, attrd)
    
    def __mul__(self, num):
        return type('%s_Array_%d' % (self.__name__, num), (ArrayTypeBase,), {'obj_type': self, 'array_size': num, 'size': self.size * num})
    
class IntTypeBase(object):
    '''a type base class
    '''
    __metaclass__ = MetaClass
    size = 0
    format = ''  # strcut format
    
    def __init__(self, val=0):
        if isinstance(val, str): val = int(val)
        if not isinstance(val, int):
            raise TypeError('type error:%s' % type(val))
        self._net_order = True  # store network order data by default
        self.value = val
        self._num = 1
        
    def __str__(self):
        return '%d(%s)' % (self._val, self.__class__.__name__)
    
    def __cmp__(self, val):
        if isinstance(val, IntTypeBase):
            return cmp(self.value, val.value)
        elif isinstance(val, (int, long)):
            return cmp(self.value, val)
        elif isinstance(val, type(None)):
            return cmp(int(self.value), None)
        else:
            raise TypeError('type error:%s' % type(val))
        
    def __int__(self):
        return int(self.value)
    
    def __hex__(self):
        return hex(self.value)
    
    def __index__(self):
        return self.value
    
    def __add__(self, val):
        return int(self.value + val)
    
    def __radd__(self, val):
        return int(val + self.value)
    
    def __sub__(self, val):
        return self.value - val
    
    def __rsub__(self, val):
        return val - self.value
    
    def __mul__(self, val):
        return self.value * val
    
    def __div__(self, val):
        return self.value / val
    
    def __mod__(self, val):
        return self.value % val
    
    def __rshift__(self, val):
        return self.value >> val
    
    def __and__(self, val):
        return self.value & val
    
    @property
    def net_order(self):
        return self._net_order
    
    @net_order.setter
    def net_order(self, _net_order):
        self._net_order = _net_order
        
    @property
    def value(self):
        return self._val
    
    @value.setter
    def value(self, val):
        if not isinstance(val, int):
            raise TypeError('type error:%s' % type(val))
        if val < 0: raise ValueError(val)
        max_val = 256 ** (self.size) - 1
        if val > max_val: raise ValueError('%d is more than the maximum size %d' % (val, max_val))
        self._val = val
    
    def unpack(self, buff, net_order=True):
        '''extract data from buffer
        '''
        if len(buff) < self.size: raise ValueError(repr(buff))
        buff = buff[:self.size]
        fmt = self.format
        if not net_order: fmt = '<' + fmt[1]
        self._val = struct.unpack(fmt, buff)[0]
        return self._val
    
    def pack(self, net_order=True):
        '''return the memory data
        '''
        fmt = self.format
        if not net_order: fmt = '<' + fmt[1]
        return struct.pack(fmt, self._val)
    
    @staticmethod
    def cls_from_size(size):
        '''return the corresponding class from the integer size
        '''
        if size == 1:
            return c_uint8
        elif size == 2:
            return c_uint16
        elif size == 4:
            return c_uint32
        elif size == 8:
            return c_uint64
        else:
            raise RuntimeError('Unsupported integer data length:%d' % size)
        
    @classmethod
    def unpack_from(cls, str, net_order=True):
        obj = cls()
        obj.unpack(str, net_order)
        return int(obj)

class ArrayTypeBase(object):
    '''array type base class
    '''
    def __init__(self, val=''):
        init_val = 0
        if isinstance(val, int): 
            init_val = val
        else:
            val = str(val)
        self._obj_array = [self.obj_type(init_val) for _ in range(self.array_size)]  # initialization
        self.value = val
    
    def __str__(self):
        return str(self.value)
    
    def __repr__(self):
        return repr(self.value)
    
    def __getitem__(self, idx):
        return self._obj_array[idx].value
    
    def __setitem__(self, idx, val):
        self._obj_array[idx].value = val
        
    def __getslice__(self, i, j):
        result = [obj.value for obj in self._obj_array[i:j]]
        if self.obj_type == c_ubyte:
            result = [chr(val) for val in result]
            result = ''.join(result)
        return result
    
    def __add__(self, oval):
        if not isinstance(oval, str):
            raise NotImplementedError('%s is not supported by type %s' % (self.__class__.__name__, type(oval)))
        return self.value + oval
    
    def __radd__(self, oval):
        return oval + self.value
    
    def __iter__(self):
        '''iterator
        '''
        for i in range(self.length):
            yield self[i]
            
    @property
    def value(self):
        result = [obj.value for obj in self._obj_array]
        if self.obj_type == c_ubyte:
            result = [chr(val) for val in result]
            result = ''.join(result)
        return result
    
    @value.setter
    def value(self, val):
        if isinstance(val, list):
            raise NotImplementedError('ArrayType is not supported type list')
        elif isinstance(val, str):
            self.unpack(val)
        
    def unpack(self, buff, net_order=True):
        '''
        '''
        if len(buff) == 0: return
        if len(buff) < self.size: raise ValueError('unpack length error:%d %d' % (len(buff), self.size))
        for i in range(self.array_size):
            self._obj_array[i].unpack(buff[i * self.obj_type.size:], net_order)
            
    def pack(self, net_order=True):
        '''
        '''
        result = ''
        for i in range(self.array_size):
            result += self._obj_array[i].pack()
        return result
    
class c_uint8(IntTypeBase):
    '''unsigned char
    '''
    size = 1
    format = '!B'
    
class c_ubyte(c_uint8): pass
        
class c_uint16(IntTypeBase):
    '''unsigned short
    '''
    size = 2
    format = '!H'
    
class c_ushort(c_uint16): pass

class c_uint32(IntTypeBase):
    '''unsigned int32
    '''
    size = 4
    format = '!I'

class c_ulong(c_uint32): pass

class c_uint64(IntTypeBase):
    '''unsigned int64
    '''
    size = 8
    format = '!Q'

class c_ulonglong(c_uint64): pass

cls = c_ubyte * 5
print(cls)
val = cls(65)
print(val)

The outputs for the above code in Python 2.7 are as follows:

<class '__main__.c_ubyte_Array_5'>
AAAAA

The definition of the metaclass has been modified in Python 3:

class IntTypeBase(object, metaclass=MetaClass):
    pass

You can use the methods in the six base for compatibility:

import six

@six.add_metaclass(MetaClass)
class IntTypeBase(object):
    pass

The advantage of using metaclasses is that you can create classes in a more elegant way, such as the c_ubyte * 5 in the above code, which can improve code readability.

0x04 Rewrite the __new__ method

Each class that inherits from object will have a __new__ method, which will be called earlier than __init__ when the class is being instantiated. And the type it returns determines the type of the object that is created ultimately.

Let's look at the following code:

class A(object):
    def __new__(self, *args, **kwargs):
        return B()

class B(object):
    pass

a = A()
print(a)

The output is as follows:

<__main__.B object at 0x023576D0>

As you can see, though it instantiates A in the code, the returned object type is B. which is mainly attributed to the __new__ method.

The following example shows how to create a class dynamically in the __new__

class B(object):
    def __init__(self, var):
        self._var = var
        
    def test(self):
        print(self._var)

class A(object):
    def __new__(self, *args, **kwargs):
        if len(args) == 1 and isinstance(args[0], type):
            return type('%s_%s' % (self.__name__, args[0].__name__), (self, args[0]), {})
        else:
            return object.__new__(self, *args, **kwargs)
    
    def output(self):
        print('output from new class %s' % self.__class__.__name__)

obj = A(B)('Hello World')
obj.test()
obj.output()

The output is as follows:

Hello World
output from new class A_B

The example implements the process of creating subclasses for the two classes dynamically, and it's suitable for the scenes where you need to generate many subclasses from arranging and combining classes, which can avoid the pain of writing a bunch of subclass code.

0x05 Summary

You must use the type implementation to create classes dynamically. However, you can choose different methods depending on the scenes.

Actually, it's not friendly to static analysis tools, because the type has changed during the run time. Moreover, it will also reduce the readability of the code, so generally, it's not recommended to use such skilled code.

1 Comment

temp

When using code from "0x02 Use type to create a class dynamically" an error comes up:

Traceback (most recent call last): 
 File "C:/Data/Python/PyCharm/venv/Naslag_Classes_Dynamically_Create.py", line 66, in <module> 
   server = SocketServer.ThreadingTCPServer(("127.0.0.1",8080), gen_cls(("www.tutorials.dom", 80))) 
 File "C:/Data/Python/PyCharm/venv/Naslag_Classes_Dynamically_Create.py", line 63, in gen_cls 
   return type("%s_%s" % (ProxyRequestHandler.__name__, server), (PortForwardingRequestHandler, objec), {"server": server}) 
NameError: name 'ProxyRequestHandler' is not defined

How come the ProxyRequestHandler is not recognized?

By the way, I am not familiar with the socket concept, so just copied / pasted the code, to see how it executes and try learn from it.