Alen Piljic apiljic

syrusakbary / django_efficient_queryset.py

Created December 16, 2013 05:18

Django efficient queryset iterator (by dividing in chunks). Taked from https://djangosnippets.org/snippets/1949/

	import gc

	def queryset_iterator(queryset, chunksize=1000):
	'''''
	Iterate over a Django Queryset ordered by the primary key

	This method loads a maximum of chunksize (default: 1000) rows in it's
	memory at the same time while django normally would load all rows in it's
	memory. Using the iterator() method only causes it to not preload all the
	classes.