Table of Contents#
- Understanding
__new__and__init__in Python - Why Override
__new__in Subclasses? - Step-by-Step Guide to Using
__new__with__init__- 3.1 Basic Syntax of
__new__ - 3.2 Modifying Arguments Before
__init__ - 3.3 Controlling
__init__Execution - 3.4 Subclassing Immutable Types
- 3.1 Basic Syntax of
- Common Pitfalls and How to Avoid Them
- Advanced Use Cases
- Conclusion
- References
1. Understanding __new__ and __init__ in Python#
To grasp how __new__ can override __init__-like behavior, we first need to clarify the roles of these two methods.
What is __new__?#
__new__ is a class method responsible for creating (allocating memory for) a new instance of a class. It takes the class (cls) as its first argument, followed by any arguments passed to the class constructor (e.g., MyClass(arg1, arg2)). It returns the newly created instance.
If __new__ returns an instance of the class, Python will automatically call __init__ on that instance to initialize it.
What is __init__?#
__init__ is an instance method responsible for initializing an existing instance. It takes the instance (self) as its first argument, followed by the same arguments passed to the constructor. Unlike __new__, __init__ does not return a value—it modifies the instance in place.
Key Difference: Creation vs. Initialization#
__new__: Creates the instance (runs first).__init__: Initializes the instance (runs second, only if__new__returns an instance of the class).
Example: Basic __new__ and __init__#
class MyClass:
def __new__(cls, *args, **kwargs):
print(f"Creating instance of {cls.__name__}")
# Call the parent class's __new__ to create the instance
instance = super().__new__(cls)
return instance # Return the new instance
def __init__(self, value):
print(f"Initializing instance with value: {value}")
self.value = value
# Usage
obj = MyClass(42)Output:
Creating instance of MyClass
Initializing instance with value: 42
Here, __new__ creates the instance, returns it, and then __init__ initializes it with value=42.
2. Why Override __new__ in Subclasses?#
__init__ is sufficient for most initialization tasks, but there are scenarios where __new__ is necessary—especially in subclasses. Here are the most common use cases:
2.1 Subclassing Immutable Types#
Immutable types like str, int, tuple, and float cannot be modified after creation. __init__ runs after the instance is created, so it can’t alter the immutable value. To customize the value of an immutable subclass, you must use __new__ to modify the input before the instance is created.
2.2 Enforcing Singleton or Caching Behavior#
If you want a subclass to have only one instance (singleton) or cache instances (e.g., for reuse), __new__ can check for existing instances and return them instead of creating new ones. __init__ would run only once (when the first instance is created), avoiding redundant initialization.
2.3 Validating or Modifying Arguments Early#
__new__ runs before __init__, so it can validate arguments (e.g., ensuring a value is positive) or modify them (e.g., adding defaults) before the instance is created. This prevents invalid instances from ever existing.
2.4 Returning a Different Type#
__new__ can return an instance of a different class entirely. This is useful for factory patterns, where a subclass decides dynamically which type of object to create (e.g., returning a Square or Circle based on input).
3. Step-by-Step Guide to Using __new__ with __init__#
Let’s walk through practical examples to master __new__ in subclasses.
3.1 Basic Syntax of __new__#
To override __new__ in a subclass, define it as a class method with cls as the first parameter. Always call the parent class’s __new__ (via super()) to ensure proper instance creation, unless you’re intentionally bypassing it.
Syntax:
class SubClass(ParentClass):
def __new__(cls, *args, **kwargs):
# Custom logic here (e.g., validate args, modify inputs)
instance = super().__new__(cls, *args, **kwargs) # Create instance via parent
# Optional: Modify the instance before __init__ runs
return instance # Return the instance (triggers __init__)3.2 Modifying Arguments Before __init__#
Suppose you want a subclass of str that adds a fixed prefix to the string. Since str is immutable, __init__ can’t change the string after creation—so we use __new__ to modify the input first.
Example: PrefixedStr Subclass
class PrefixedStr(str):
def __new__(cls, value, prefix="[INFO] "):
# Modify the input value before creating the str instance
prefixed_value = f"{prefix}{value}"
# Call str.__new__ to create the immutable string instance
instance = super().__new__(cls, prefixed_value)
return instance
# Usage
s = PrefixedStr("Hello, World!")
print(s) # Output: [INFO] Hello, World!
print(type(s)) # Output: <class '__main__.PrefixedStr'> (still a subclass of str)Here, __new__ modifies value to include the prefix, then passes it to str.__new__ to create the instance. __init__ isn’t needed here because str’s __init__ does nothing (immutable types don’t require initialization beyond creation).
3.3 Controlling __init__ Execution#
__init__ runs only if __new__ returns an instance of the subclass. If __new__ returns an instance of a different class or None, __init__ is skipped. This allows you to conditionally skip initialization.
Example: Singleton Subclass
class SingletonSubclass:
_instance = None # Cache for the singleton instance
def __new__(cls, *args, **kwargs):
if cls._instance is None:
# Create a new instance if none exists
cls._instance = super().__new__(cls, *args, **kwargs)
return cls._instance # Return the cached instance
def __init__(self, value):
print("Initializing instance...")
self.value = value # Only runs once (when _instance is None)
# Usage
obj1 = SingletonSubclass(42)
obj2 = SingletonSubclass(99) # No "Initializing..." message
print(obj1.value) # Output: 42 (not 99, because __init__ didn't run for obj2)
print(obj1 is obj2) # Output: True (same instance)Here, __new__ returns the cached _instance after the first call, so __init__ only runs once.
3.4 Subclassing Immutable Types#
Immutable types like int or tuple require __new__ for customization. Let’s subclass int to create a PositiveInt that ensures values are non-negative.
Example: PositiveInt Subclass
class PositiveInt(int):
def __new__(cls, value):
# Validate: Ensure value is non-negative
if value < 0:
raise ValueError("PositiveInt must be non-negative")
# Create the int instance with the validated value
return super().__new__(cls, value)
# Usage
valid = PositiveInt(10)
print(valid) # Output: 10
invalid = PositiveInt(-5) # Raises ValueError: PositiveInt must be non-negativeIf we tried to use __init__ here, it would fail because int is immutable—__init__ can’t change the value after creation.
4. Common Pitfalls and How to Avoid Them#
Even experienced developers trip up with __new__. Here are key pitfalls and fixes:
Pitfall 1: Forgetting to Call super().__new__#
If __new__ doesn’t call the parent class’s __new__, Python won’t allocate memory for the instance, leading to a TypeError.
Bad:
class BadSubclass:
def __new__(cls):
return # Oops! No instance createdError:
TypeError: object.__new__() takes exactly one argument (the type to instantiate)
Fix: Always call super().__new__(cls, *args, **kwargs) to delegate instance creation to the parent class.
Pitfall 2: Returning a Non-Instance from __new__#
If __new__ returns a value that isn’t an instance of the subclass, __init__ won’t run. This is intentional in some cases (e.g., factories), but can cause bugs if accidental.
Example: Accidental Non-Instance Return
class MyClass:
def __new__(cls, value):
if value < 0:
return "Invalid value" # Returns a string instead of MyClass instance
return super().__new__(cls)
def __init__(self, value):
self.value = value # Never runs if value < 0
# Usage
obj = MyClass(-1)
print(obj) # Output: "Invalid value" (no __init__ called)Fix: Document intentional non-instance returns. For accidental cases, ensure __new__ returns super().__new__(cls, ...).
Pitfall 3: Mismatched Arguments Between __new__ and __init__#
If __new__ and __init__ expect different arguments, Python will throw a TypeError when __init__ is called.
Bad:
class MismatchedArgs:
def __new__(cls, a, b):
return super().__new__(cls)
def __init__(self, a): # __init__ expects 1 arg, but __new__ passes 2
self.a = a
# Usage
obj = MismatchedArgs(1, 2) # TypeError: __init__() takes 2 positional arguments but 3 were givenFix: Ensure __new__ and __init__ accept compatible arguments. Use *args and **kwargs to forward arguments dynamically:
class FixedArgs:
def __new__(cls, *args, **kwargs):
return super().__new__(cls)
def __init__(self, a, b):
self.a = a
self.b = b
obj = FixedArgs(1, 2) # Works!Pitfall 4: Modifying self in __new__ for Mutable Types#
For mutable types (e.g., list, dict), avoid modifying self in __new__—save changes for __init__. __new__ is for creation, not initialization.
Bad:
class BadList(list):
def __new__(cls, items):
instance = super().__new__(cls)
instance.extend(items) # Modify instance in __new__ (unnecessary)
return instance
# Better to use __init__ for mutable types:
class GoodList(list):
def __init__(self, items):
super().__init__(items) # Let __init__ handle initialization5. Advanced Use Cases#
Let’s explore more complex scenarios where __new__ shines.
Use Case 1: Factory Pattern with Dynamic Type Return#
Create a subclass that returns instances of other classes based on input.
class Shape:
def __new__(cls, shape_type, *args):
if shape_type == "circle":
return Circle(*args) # Return a Circle instance
elif shape_type == "square":
return Square(*args) # Return a Square instance
else:
raise ValueError(f"Unknown shape: {shape_type}")
class Circle:
def __init__(self, radius):
self.radius = radius
class Square:
def __init__(self, side):
self.side = side
# Usage
circle = Shape("circle", 5)
square = Shape("square", 10)
print(type(circle)) # Output: <class '__main__.Circle'>
print(type(square)) # Output: <class '__main__.Square'>Use Case 2: Caching Instances for Reuse#
Cache instances of a subclass to avoid redundant object creation (e.g., for performance).
class CachedSubclass:
_cache = {} # Maps arguments to instances
def __new__(cls, key):
if key in cls._cache:
return cls._cache[key] # Return cached instance
# Create and cache new instance
instance = super().__new__(cls)
cls._cache[key] = instance
return instance
def __init__(self, key):
self.key = key # Runs only once per key
# Usage
obj1 = CachedSubclass("foo")
obj2 = CachedSubclass("foo") # Returns cached instance
print(obj1 is obj2) # Output: True6. Conclusion#
__new__ is a powerful tool for customizing object creation in Python subclasses. By understanding its role as the "constructor" (vs. __init__’s "initializer"), you can:
- Subclass immutable types (e.g.,
str,int) by modifying values before creation. - Enforce singletons, caching, or factory patterns.
- Validate arguments early to prevent invalid instances.
- Dynamically return instances of other classes.
Remember to:
- Call
super().__new__(cls, ...)to ensure proper instance creation. - Match arguments between
__new__and__init__. - Use
__new__for creation logic and__init__for initialization.
With these techniques, you’ll inject custom code into the object creation process with confidence.
7. References#
- Python Official Docs:
object.__new__ - Real Python:
__new__and__init__in Python - PEP 367: New Metaclass Syntax (background on
__new__). - Fluent Python (Book by Luciano Ramalho) (Chapter 2: An Array of Sequences).