Flags (BitFields)

There are many different Global Satellite Navigation Systems (GNSS) in operation today:

• GPS

• GLONASS

• Galileo

• BeiDou

• QZSS

• IRNSS

GNSS receivers may understand or be configured to track one or more of these systems. If we wanted to build a data model of a GNSS receiver we would want to know which systems it can track. In Django we might do this using a collection of boolean fields like this:

from django.db import models


class GNSSReceiverBasic(models.Model):

    gps = models.BooleanField(default=False)
    glonass = models.BooleanField(default=False)
    galileo = models.BooleanField(default=False)
    beidou = models.BooleanField(default=False)
    qzss = models.BooleanField(default=False)
    irnss = models.BooleanField(default=False)
    sbas = models.BooleanField(default=False)

    class Meta:

        # we can create an index for all fields, which will speed up queries for
        # exact matches on these fields
        indexes = [
            models.Index(fields=[
                'gps',
                'glonass',
                'galileo',
                'beidou',
                'qzss',
                'irnss',
                'sbas'
            ])
        ]

Which would allow us to check for receiver compatibility and filter requirements like this:

receiver1 = GNSSReceiverBasic.objects.create(
    gps=True,
    glonass=True
)

receiver2 = GNSSReceiverBasic.objects.create(
    gps=True,
    glonass=True,
    galileo=True,
    beidou=True
)

# check for GPS and BEIDOU
assert not (receiver1.gps and receiver1.beidou)
assert receiver2.gps and receiver2.beidou

# get all receives that have at least GPS and BEIDOU
qry = GNSSReceiverBasic.objects.filter(Q(gps=True) & Q(beidou=True))
assert receiver1 not in qry
assert receiver2 in qry

# get all receivers that have either GPS or BEIDOU
qry = GNSSReceiverBasic.objects.filter(Q(gps=True) | Q(beidou=True))
assert receiver1 in qry
assert receiver2 in qry

This works pretty well. As our data scales though the waste of using an entire column for each boolean can add up. We can do better by using a single column as a bit field.

Python has a built-in enum.IntFlag type that is used to represent bit fields. Bit fields are useful for storing multiple boolean values in a single column. This is much more space efficient. EnumField supports enum.IntFlag types out of the box. We could rewrite our GNSS receiver model like this:

from django.db import models
from enum import IntFlag
from django_enum import EnumField


class Constellation(IntFlag):

    # fmt: off
    GPS     = 1 << 0  # 1
    GLONASS = 1 << 1  # 2
    GALILEO = 1 << 2  # 4
    BEIDOU  = 1 << 3  # 8
    QZSS    = 1 << 4  # 16
    IRNSS   = 1 << 5  # 32
    # fmt: on


class GNSSReceiver(models.Model):

    constellations = EnumField(Constellation, db_index=True)

And use it like this:

receiver1 = GNSSReceiver.objects.create(
    constellations=Constellation.GPS | Constellation.GLONASS
)

receiver2 = GNSSReceiver.objects.create(
    constellations=(
        Constellation.GPS |
        Constellation.GLONASS |
        Constellation.GALILEO |
        Constellation.BEIDOU
    )
)

wanted = Constellation.GPS | Constellation.BEIDOU

# check for GPS and BEIDOU
assert not (
    Constellation.GPS in receiver1.constellations and
    Constellation.BEIDOU in receiver1.constellations
)
assert (    
    Constellation.GPS in receiver2.constellations and
    Constellation.BEIDOU in receiver2.constellations
)

# we can treat IntFlags like bit masks so we can also check for having at
# least GPS and BEIDOU like this:
assert not wanted & receiver1.constellations == wanted
assert wanted & receiver2.constellations == wanted

# get all receives that have at least GPS and BEIDOU
qry = GNSSReceiver.objects.filter(constellations__has_all=wanted)
assert receiver1 not in qry
assert receiver2 in qry

# get all receivers that have either GPS or BEIDOU
qry = GNSSReceiver.objects.filter(constellations__has_any=wanted)
assert receiver1 in qry
assert receiver2 in qry

The bit field model is much more compact and it does better in space efficiency and query performance. We also get some additional lookup types for EnumField that represent flags: has_all and has_any.

The defined indexes are not necessarily used in our examples. They may be partially engaged for the has_all lookups but not for has_any. Our flag lookups perform optimized bit mask operations in the database. The bit field example will out perform the boolean example because it consumes much less memory while doing table scans with particular improvements in hard to index has_any queries.