'Get latest related item for each item in QuerySet

I have a simple model of an Observation made by a Sensor:

class Observation(models.Model):
    class ObservationType(models.TextChoices):
        PM25 = 'pm25_kal', 'PM2,5'
        PM10 = 'pm10_kal', 'PM10'
        RH = 'rh', _('Relative humidity')
        TEMP = 'temp', _('Temperature')

    date_time = models.DateTimeField()
    sensor = models.ForeignKey(Sensor, on_delete=models.CASCADE)
    obs_type = models.CharField(max_length=8, choices=ObservationType.choices)
    value = models.DecimalField(max_digits=6, decimal_places=3)

What I want to do, is get a list or QuerySet with the latest Observation of a certain type that should at least have been created within 24 hours, for each sensor. I solved the problem using a model method for my Sensor model and a custom QuerySet for my Observation model, to filter recent observations.

class ObservationQuerySet(models.query.QuerySet):
    def recent(self):
        return self.filter(date_time__gte=timezone.now() - timedelta(days=1))
def latest_recent_observation(self, obs_type):
    try:
        return self.observation_set.filter(obs_type=obs_type).recent().latest('date_time')
    except Observation.DoesNotExist:
        return None

I can loop over all sensors and get the latest_recent_observation() for each of them, but for larger datasets it is pretty slow. Is there any way to make this more efficient?

Edit: At this moment I'm using SQLite, but I might switch to MariaDB. Would that make this faster as well?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source