Bulk create model objects in django

django django-models

I have a lot of objects to save in database, and so I want to create Model instances with that.

With django, I can create all the models instances, with MyModel(data), and then I want to save them all.

Currently, I have something like that:

for item in items:
    object = MyModel(name=item.name)
    object.save()

I'm wondering if I can save a list of objects directly, eg:

objects = []
for item in items:
    objects.append(MyModel(name=item.name))
objects.save_all()

How to save all the objects in one transaction?

It seems the ball is rolling on implementing a fix for this code.djangoproject.com/ticket/19527

wondering for list.save_all ? You could almost answer yourself just paraphrase that wondering and use 2 first words from your topic question.

Amin Mir

as of the django development, there exists bulk_create as an object manager method which takes as input an array of objects created using the class constructor. check out django docs

Django's docs for bulk_create: docs.djangoproject.com/en/dev/ref/models/querysets/#bulk-create

But remember bulk_create has some limitations like it does not create primary keys if it is an AutoField which save() does automatically.

@HiteshGarg, Is that still true now days?

@RaydelMiranda, yes it is still true. It is right there in the documentation:

If the model’s primary key is an AutoField it does not retrieve and set the primary key attribute, as save() does, unless the database backend supports it (currently only PostgreSQL).

Using Django 3.0.x and I confirm that using bulk_create() does not trigger any signals. I wonder why.

daaawx

Use bulk_create() method. It's standard in Django now.

Example:

Entry.objects.bulk_create([
    Entry(headline="Django 1.0 Released"),
    Entry(headline="Django 1.1 Announced"),
    Entry(headline="Breaking: Django is awesome")
])

Changed in Django 1.10: Support for setting primary keys on objects created using bulk_create() when using PostgreSQL was added.

Thorin Schiffer

worked for me to use manual transaction handling for the loop(postgres 9.1):

from django.db import transaction
with transaction.atomic():
    for item in items:
        MyModel.objects.create(name=item.name)

in fact it's not the same, as 'native' database bulk insert, but it allows you to avoid/descrease transport/orms operations/sql query analyse costs

This slightly changed. Now transaction does not have commit_on_success anymore. You should use transaction.atomic() See: stackoverflow.com/questions/21861207/…

This method doesn't exactly scale, because you are practically making a create on every for loop pass which equals a db interaction per pass. bulk create seems to work faster.

Yeah this was answered a quite while ago, you should use bulk_create() now for this

Yagmur SAHIN

name = request.data.get('name')
period = request.data.get('period')
email = request.data.get('email')
prefix = request.data.get('prefix')
bulk_number = int(request.data.get('bulk_number'))

bulk_list = list()
for _ in range(bulk_number):
    code = code_prefix + uuid.uuid4().hex.upper()
    bulk_list.append(
        DjangoModel(name=name, code=code, period=period, user=email))

bulk_msj = DjangoModel.objects.bulk_create(bulk_list)

Ivan Klass

Here is how to bulk-create entities from column-separated file, leaving aside all unquoting and un-escaping routines:

SomeModel(Model):
    @classmethod
    def from_file(model, file_obj, headers, delimiter):
        model.objects.bulk_create([
            model(**dict(zip(headers, line.split(delimiter))))
            for line in file_obj],
            batch_size=None)

OmerGertel

Using create will cause one query per new item. If you want to reduce the number of INSERT queries, you'll need to use something else.

I've had some success using the Bulk Insert snippet, even though the snippet is quite old. Perhaps there are some changes required to get it working again.

http://djangosnippets.org/snippets/446/

MrJ

Check out this blog post on the bulkops module.

On my django 1.3 app, I have experienced significant speedup.

zEro

for a single line implementation, you can use a lambda expression in a map

map(lambda x:MyModel.objects.get_or_create(name=x), items)

Here, lambda matches each item in items list to x and create a Database record if necessary.

Lambda Documentation

You probably want to mention that the lambda has to be map ped over items: map(lambda name: MyModel.objects.get_or_create(name = name), items)

Ja, thats another way i try to say (:

Daniel Roseman

The easiest way is to use the create Manager method, which creates and saves the object in a single step.

for item in items:
    MyModel.objects.create(name=item.name)

+1. If name is unique and duplicate inputs are possible then it would be a good idea to use get_or_create.

How does this answer the question? Model.objects.create is equivalent to object = MoModel(..) object.save(). And this does not do it in one transaction...

Bulk create model objects in django

Follow WeChat

Want to stay one step ahead of the latest teleworks?

相似问题

Platform

Support

Contact US