Magic Methods

What are magic methods? They are special methods describing how a certain objects should behave. They are always surrounded by double underscores (e.g. __init__, __lt__). The double underscores indicate that these are magic methods and shouldn't be called directly by the programmer, they are normally called by the interpreter itself. Let's look at some common examples of these magic methods.

Notes

This tutorial requires a good understanding of Python classes. To read up on them check here: https://docs.python.org/3/tutorial/classes.html

The tutorial does not provide an exhaustive list of magic methods. If you want to read more check the documentation: https://docs.python.org/3/reference/datamodel.html#special-method-names

Construction and Initialisation

You have probably seen the most basic magic method, __init__. It's what we use to define the initialisation of an object. It is not the first method to be called when a new object is created however. The first method to be called is the __new__ method which creates a new instance and then calls the __init__ method which initialises the object according to the parameters passed to __new__ (which it passes to __init__). Together these 2 functions make up an object's constructor.

A third method related to object creation and deletion is the __del__ method which is called when an object's lifecycle ends. It does any cleanup that might be needed when an object is destroyed. It is the objects destructor.

Representing your class

It’s often useful to have a string representation of a class. Python provides a few methods we can implement in our class definition to customise how built in functions that return representations of our class behave. Here are a few of these functions:

• __str__(self) Defines behaviour for when str() is called on an instance of your class
• __repr__(self) Defines behaviour for when repr() is called on an instance of your class. The major difference between str() and repr() is the intended audience. repr() output should be machine readable (JSON for example) while str() should be human readable

Comparison methods

Python has quite a few magic methods designed to implement comparisons between objects using operators instead of method calls. They also provide a way to override the default Python behaviour for comparing objects (by reference). Here we list some of these magic methods and what they do:

• __cmp__(self, other) This is the most basic of the comparison magic methods. It can implement behaviour for all of the comparison operators (<, ==, !=, etc.). It should return a negative int if self < other, zero if self == other and a positive int if self > other. It's usually better do define each comparison we need, one by one rather than all at once but __cmp__ and this method isn't even used anymore since Python 3.
• __eq__(self, other) Defines the behaviour of the equality operator ==
• __ne__(self, other) Defines the behaviour of the inequality operator !=
• __lt__(self, other) Defines the behaviour of the less-than operator <
• __gt__(self, other) Defines the behaviour of the greater-than operator >
• __le__(self, other) Defines the behaviour of the less-than-or-equal-to operator <=
• __ge__(self, other) Defines the behaviour of the greater-than-or-equal-to operator >=

Example

In the following example we will implement a custom class called Area which will store the height and width of an image. We will implement it with all comparison magic methods to see how they all work.

Input:

class Area:
    
    def __init__(self, height, width):
        self.height = height
        self.width = width
    
    def __eq__(self, other):
        if isinstance(other, Area):
            return self.height * self.width == other.height *     other.width
        else:
            return False
        
    def __ne__(self, other):
        return not self == other
    
    def __lt__(self, other):
        if isinstance(other, Area):
            return self.height * self.width < other.height * other.width
        else:
            return False
        
    def __gt__(self, other):
        if isinstance(other, Area):
            return self.height * self.width > other.height * other.width
        else:
            return False
     
    def __le__(self, other):
        return self == other or self < other
    
    def __ge__(self, other):
        return self == other or self > other
    
a1 = Area(7, 10)
a2 = Area(35, 2)
a3 = Area(8, 9)
print('Testing ==')
print(a1 == 'hello')
print(a1 == a2)
print(a1 == a3)
print('Testing !=')
print(a1 != 'hello')
print(a1 != a2)
print(a1 != a3)
print('Testing <')
print(a1 < 'hello')
print(a1 < a2)
print(a1 < a3)
print('Testing >')
print(a1 > 'hello')
print(a1 > a2)
print(a1 > a3)
print('Testing <=')
print(a1 <= 'hello')
print(a1 <= a2)
print(a1 <= a3)
print('Testing >=')
print(a1 >= 'hello')
print(a1 >= a2)
print(a1 >= a3)

Output:

Testing ==
False
True
False
Testing !=
True
False
True
Testing <
False
False
True
Testing >
False
False
False
Testing <=
False
True
True
Testing >=
False
True
False

Arithmetic operators

Just like with comparison methods, python allows you to overload arithmetic operators as well. Here we will take a look at a few of them that you can overload (besides these you can overload pretty much any operator you’d want to use on an object):

Unary operators

• __pos__(self) Implements behaviour for when the unray + operator is called on our object
• __neg__(self) Implements behaviour for when the unray - operator is called on our object

Normal arithmetic operators

• __add__(self, other) Implements addition with the + operator
• __sub__(self, other) Implements subtraction with the - operator
• __mul__(self, other) Implements multiplication with the * operator
• __and__(self, other) Implements bitwise 'and' with the & operator
• __or__(self, other) Implements bitwise 'or' with the | operator
• __xor__(self, other) Implements bitwise 'xor' with the ^ operator

Reflected arithmetic operators

• __radd__(self, other) Implements addition operator + with operands swapped
• __rsub__(self, other) Implements subtraction operator - with operands swapped

Example

In this example we are going to create a class representing a Point with its 2 coordinates. We are going to implement the addition and subtraction operators with Points and we are also going to implement the multiplication operator with floats which will be commutative. Finally we are going to implement the unary - operator as well which will negate both coordinates of a Point. We will also implement the __str__ so it can be easily printed.

Input:

class Point:
    
    def __init__(self, x, y):
        self.x = x
        self.y = y
    
    def __add__(self, other):
        if isinstance(other, Point):
            return Point(self.x + other.x, self.y + other.y)
        else:
            raise ValueError('Addition with type {} is undefined.'.format(type(other)))
            
    def __sub__(self, other):
        if isinstance(other, Point):
            return Point(self.x - other.x, self.y - other.y)
        else:
            raise ValueError('Subtraction with type {} is undefined.'.format(type(other)))
            
    def __mul__(self, other):
        if isinstance(other, float) or isinstance(other, int):
            return Point(self.x * other, self.y * other)
        else:
            raise ValueError('Multiplication with type {} is undefined.'.format(type(other)))
            
    def __rmul__(self, other):
        return self * other
    
    def __neg__(self):
        return self * -1
    
    def __str__(self):
        return 'x: {}, y: {}'.format(self.x, self.y)
    
a = Point(2, 3)
b = Point(5, 7)
print(a + b)
print(b - a)
print(-(a - b))
print(a * 7)
print(2 * b)

Output:

x: 7, y: 10
x: 3, y: 4
x: 3, y: 4
x: 14, y: 21
x: 10, y: 14

Summary

We’ve seen quite a few different magic methods that we can overload and how they actually change the behaviour of our class. This was only a small subset of the different types of behaviours we can change in our object. Listed below we can see all the types of magic methods we can overload and some examples, this list still isn’t full so check the Python documentation to see all of them.

Construction and Initialisation

• __init__(self, ...)
• __del__(self)

Comparison

• __eq__(self, other)
• __ne__(self, other)

Unary operators and functions

• __pos__(self) unary + operator
• __neg__(self) unary - operator

Normal arithmetic operators

• __add__(self, other)
• __sub__(self, other)

Reflected arithmetic operators

• __radd__(self, other) addition operator + with operands swapped
• __rsub__(self, other) subtraction operator - with operands swapped

Augmented assignment

• __iadd__(self, other) addition with assignment using += operator
• __isub__(self, other) subtraction with assignment using -= operator

Type conversion

• __int__(self) type conversion to int with int() method
• __float__(self) type conversion to float with float() method

Representing your class

• __str__(self) string representation of class with str() method
• __repr__(self) machine-readable representation of class with __repr__() method

About the Author

Tamas works as a Data Engineer at Cambridge Spark, developing the companies data pipeline to process the data flowing through our systems. Prior to that, he has worked at Morgan Stanley for 2 years developing high throughput, realtime fraud prevention services in close collaboration with Data Scientists. He holds a BSc in Software Development and is pursuing an MSc at Birkbeck University of London in Advanced Computing Technologies.

Connect with Tamas

Thanks for reading! If you enjoyed the post, we’d appreciate your support by applauding via the clap (👏🏼) button below or by sharing this article so others can find it.

If you’d like to be the first to hear about our new content, including new tutorials, case studies and other useful resources, click here!

Further tutorials to practice your skills on:

- Deploying a Machine Learning Model to the web
- Six Data Science projects to expand your skills and knowledge
- Unit testing with PySpark
- Hyperparameter tuning in XGBoost
- Getting started with XGBoost