'Provide initial data for Django models from txt file

I have a .txt file with 3 columns (longitude, latitude, distance) and 40 million rows.
These are the first lines of the file.

-179.98 89.98   712.935
-179.94 89.98   712.934
-179.9  89.98   712.933
-179.86 89.98   712.932
-179.82 89.98   712.932
-179.78 89.98   712.931
-179.74 89.98   712.93
-179.7  89.98   712.929
-179.66 89.98   712.928
-179.62 89.98   712.927

Is there a way to provide these 40 million rows as initial data to this Django model?

from django.db import models

class Location(models.Model):

    longitude = models.FloatField()
    latitude = models.FloatField()
    distance = models.FloatField()


Solution 1:[1]

This is fully untested code, but you could try something like:

import os
from your_app.models import Location

with open('path/to/your_text_file.txt') as file:
    data = file.readlines()
    for line in data:
        coords_and_dist = line.split(' ')
        try:
           Location.objects.create(longitude=float(coords_and_dist[0]),
                                   latitude=float(coords_and_dist[1]),
                                   distance=float(coords_and_dist[2])
        except:
            pass

You may run into some performance and memory issues with 40 million lines, I have not run something like this on a text file that big.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Milo Persic