algorithm - Generate random permutation of huge list (in Python) -

i'd create random permutation of numbers [1,2,...,n] n big number. don't want store elements of permutation in memory, rather iterate on elements of particular permutation without holding former values in memory.

any idea how in python?

one possibility use encryption. since encryption reversible, i.e. one-to-one, given key same numbers encrypt in different order.

you need block cypher block size large enough include maximum n. use des in ecb mode n = 2^64 - 1. use aes in ecb mode n = 2^128 - 1. other sizes, either use hasty pudding cipher, has variable block size, or write own simple feistel cipher. assume need shuffle, not cryptographically secure shuffle.

if output greater n, re-encrypt until less n, 1-to-1 property ensures chain of large numbers unique.

there no need store entire array in memory, each number can encrypted needed. key , cipher algorithm needed. 1 slight complication block ciphers work on [0 ... n-1]; might need code deal extremes.