Implementing counters in Python

I am sure you’ve all been in a situation where you’d like to implement a counter to track website hits, database reads, API exceptions, etc. There are various ways this can be implemented with each one having their own pros and cons. I’ll briefly go over them and explain alongside, so just follow along.

Single Threaded Counter

This is the simplest and most efficient implementation considering you have a single threaded application. But in the modern world, this is a rarity so won’t be used often, but I’ll still mention it for the sake of it.

class SingleThreadCounter(object):
    def __init__(self):
    	self.value = 0
        
    def increment(self):
        self.value += 1

Pros: Very fast and efficient
Cons: If another thread executes the same code at the same time, you could end up with adding 1 to an old value

Thread-Safe Counter (for large number of reads)

This takes care of implementing a counter in a multi-threaded application. But it has it’s own merits and demerits.

import threading

class ReadCounter(object):
    def __init__(self):
        self.value = 0
        self._lock = threading.Lock()
        
    def increment(self):
        with self._lock:
            self.value += 1

Pros: Thread safe implementation.
Cons: You need to lock the counter each time you need to increment so in case where there are a lot of increments made, this may be slower.

Use this methodology if updates/increments are rare and reads are more frequent.

Using itertools.count() (for large number of writes)

itertools.count in Python is a counter like class that does not require locking.

import itertools
import threading

class WriteCounter(object):
    def __init__(self):
        self._number_of_read = 0
        self._counter = itertools.count()
        self._read_lock = threading.Lock()

    def increment(self):
        next(self._counter)

    def value(self):
        with self._read_lock:
            value = next(self._counter) - self._number_of_read
            self._number_of_read += 1
        return value

The increment code is quite simple in this case: the counter is just incremented without any lock. The GIL protects concurrent access to the internal data structure in C, so there’s no need for us to lock anything.

On the other hand, Python does not provide any way to read the value of an itertools.count object. We need to use a small trick to get the current value. The value method increments the counter and then gets the value while subtracting the number of times the counter has been read (and therefore incremented for nothing).

Pros: Thread safe implementation. Efficient for faster increments because it does not require locking.
Cons: There is no easy way to read the value of the counter. Need a hack to read the value which makes it slower.

Use this methodology if reads are rare and updates/increments are more frequent.

Performance Metrics