So, I’ve totally been writing this blog for almost a week, but so far I haven’t talked about how an actual algorithm works. Here’s a classic algorithm.
Let’s say you have an array of n elements a[0] through a[n-1]. You would like to generate a random permutation of this array. This means that any ordering of the array is equally probable, which means that if you look at any position in a, any element has an equal chance of being there: 1 in n (1/n). We assume that we have the function r(n) which generates a uniform random integer between 0 and n-1, inclusive. (In C, you could do this with random() % n, and in Java you’d do (int)(Math.random() * n).)
On two occasions while interviewing, I have been asked for an algorithm to permute an array. Both times the interviewer was amazed when I showed them this algorithm. Many people assume that a second storage array is necessary for remembering numbers you have picked, but you can do it in place with no memory. The algorithm (usually credited to Knuth, but see below) is quite simple:
- For i = 1 to n-1:
- Let j = r(i+1)
- Swap a[i] and a[j]
My guess as to why more people don’t know it is due to its interesting use of induction. Software engineers generally learn of induction in college, fear it, learn how to parrot back enough to pass the classes that use it, then promptly forget it. But recursion is ubiquitous and unavoidable. Whether explicitly recursing with a function calling itself, or implicitly in a loop, any time later steps assume that earlier steps have done their work over part of the data, you need induction. Instead of fearing induction, learn how it works and learn how you can rely on it to do more with your data.
So how does the algorithm use induction? Suppose we are in iteration i. Assume inductively that a[0]…a[i-1] are a random permutation of that part of the array. As a base case, a[0] starts as being a permutation of itself. Now consider what the array will look like after the swap. Element i has an equal chance of being in any position 0…i. What about the remaining elements? Each one has a (i-1)/i chance of standing still and a 1/i chance of being put in the ith spot. If it stands still, it had a 1/(i-1) chance of being there by the induction hypothesis. But multiplying this by the stand-still probability, we get a 1/i chance of it being there after the swap. Together with the swap probability, this means that it has a 1/i chance of being in any spot, which is the definition of a uniform permutation.
As a side note, Wikipedia gives a different version:
- For i = 0 to n-2:
- Let j = i + r(n-i)
- Swap a[i] and a[j]
This has the advantage that as soon as the i-loop visits a location, its value is finalized so you can output the permutation as it is being created. The disadvantage is that it’s “less inductive” in that, after the ith iteration, you don’t have a true permutation of the first i elements, but only part of a permutation of all n elements. Also, if you are simply generating a random permutation of the numbers 0…n-1 (or analogously the result of some function), you can replace a[i] with i (or f(i)) in the first version and you don’t need to initialize the array. This doesn’t work in the second because of its “future swaps”. This is the version that most people give. A proof of its correctness is given here.
What are some uses of random permutations? They can be good for generating test data, e.g. for a sorting function or any other function which should be able to deal with random data. They can be good as a starting point for a hill-climbing algorithm, e.g. with Traveling Salesman. And, of course, they’re great for asking about in interviews.
Update: I’m overwhelmed by the response! Thanks for all the replies, on here and Reddit. Here are some clarifications:
- I agree that random() % n is not perfectly uniform, but neither is (int)(Math.random() * n) due to the finite precision of the floating point number. Both of them are pretty close for small values of n. We could do better, but it’s outside the scope of this post. Unlike rand, random is supposed to have good entropy in the low order bits.
- Swapping i with r(n) is a simplification. This doesn’t give you a uniform distribution, however. As one reader mentioned, this page gives the lowdown, as do several commenters on reddit.
- Someone took issue with the first algorithm. However, they cited that same page, which is only about the i-with-rand(n) issue (and a[i] remaining stationary). If you can find a fundamental flaw with my proof, please tell!
Again, thanks, and I hope you keep reading!
Two modifications that occur to me (in the interest of simplifying code) would be:
For i = 0 to n-1:
* Let j = r(n)
* Swap a[i] and a[j]
and even better — getting rid of the potential boundary error in the for loop
For i = 0 to TIMESTHROUGH
* Let j = r(n)
* Let k = r(n)
* Swap a[i] and a[j]
The second one is obviously correct, as TIMESTHROUGH approaches infinity. Both would require some analysis to find out how bad their fall-off from the perfect solution is. How large a value of TIMESTHROUGH do you really need, for instance?
Of course, in actual (C++) code, I’d just write:
//where v is a container transversed by random access iterators
random_shuffle(v.begin(), v.end());
Should be this in the second algorithm
* Swap a[j] and a[k]
From an analytical point of view your solution is quite elegant. But from a practical one I would propose Ruby:
[1, 2, 4, 5].sort_by {rand}
Regards, polzr
Great blog, I like what you have so far!
That was a real eye-opener, and obvious once it’s explained. Time for me to re-read Kunth, I think…
random() % n is not uniform since it prefers lower numbers to higher depending on divisibility of n and RAND_MAX. If RAND_MAX would be 25 and n=10, you will get 3/26 probability of numbers 0 to 5 and 2/26 for numbers 6 to 9.
Otherwise I like the simplicity of permutaion algorithm.
the wikipedia is, if the phrase means anything at all, “more inductive”, not less. on each loop, you have an array, from which you pick one element at random. that leaves you with a smaller array remaining. the induction terminates when there is only one element (or none - either case is trivial).
look here - http://en.wikipedia.org/wiki/Mathematical_induction - and see how the empty array corresponds to the first step and the process of extracting one number corresponds to the second step.
[…] June 5th, 2007 [link][more] […]
well n be better be pretty small or the random number generator be really really beefy, because n! is super huge, bigger then most random gens cycle.
reddit +1
I have no clue what this means.. Can someone show an example and write back to me pls.
thx.
random() % n will only give you an uniform random integer if n happens to divide RAND_MAX
Random permutations are also great for implementing bogosort.
Hi there,
interesting stuff ! Keep up the good work
Here a ruby quickhack to visualize your algorithm on the CLI… (was just curious..)
#!/usr/bin/ruby -w
#
array = Array.new
0.upto(20) { |n| array.push(n) }
def permutate array
1.upto( array.length - 1 ) do |i|
j = rand( i + 1 )
array[i], array[j] = array[j], array[i]
end
array
end
def print array
array.each_with_index do |number, index|
printf(”%2i | %s\n”, index, (” “*number+”*”))
end
end
—
Result:
0 | *
1 | *
2 | *
3 | *
4 | *
5 | *
6 | *
7 | *
8 | *
9 | *
10 | *
11 | *
12 | *
13 | *
14 | *
15 | *
16 | *
17 | *
18 | *
19 | *
20 | *
“—”
0 | *
1 | *
2 | *
3 | *
4 | *
5 | *
6 | *
7 | *
8 | *
9 | *
10 | *
11 | *
12 | *
13 | *
14 | *
15 | *
16 | *
17 | *
18 | *
19 | *
20 | *
Cheers !
Just as a warning, doing random() % n is not guaranteed to produce random numbers in C. Many of the pseudo random number generators pretty much follow a set pattern for the lower bits, so using random() % n for small values of n can produce a visible pattern.
Only by using all the bits of the randomly generated value can you ensure truly pseudo-random numbers.
lol?
I’d like to mention another random-shuffling algorithm that is less efficient but particularly easy to understand:
1. Assign a random weight (e.g. floating point number) to each element
2. Sort by weight
3. Remove the weights
Here’s an example implementation in Erlang:
random_perm(L) ->
Weighted = [{random:uniform(), E} || E
> uniform random integer between 0 and n-1, inclusive. (In C, you could do this with random() % n
Except that random() returns uniformly (depending on libc) from 0..RAND_MAX which is 0..2147483647 on my system. However, this is prime (mersenne 31), so n never divides it (unless your list is really long) and so the modulus operation never produces a uniform random number.
Very cool. There’s an extensive comment thread on Reddit which you might be interested by…
Ah, you submitted it. *headsmack*
What are some uses of random permutations? They can be good for generating test data, e.g. for a sorting function or any other function which should be able to deal with random data. They can be good as a starting point for a hill-climbing algorithm, e.g. with Traveling Salesman. And, of course, they’re great for asking about in interviews.
And, of course, it’s good for card shuffling. Which not everyone gets right.
(I’m hoping HTML works…)
BTW, rand() % n isn’t a *good* way to get a random number from 0 to n-1 in C; many random number generators have less entropy in the lower bits than the upper ones. Read this (search for “lower”) and this.
Use this instead:
rand() / (RAND_MAX / N + 1)
Very cool! Not bad, you got in the del.icio.us popular links in ur first week
This was actually common on 8 bit systems.
Nice Blog! Hope you keep it up and you were already reddited.
haha, great post - thank you !
[…] This post on Algo Blog about random permutation shows a cool a little algorithm for randomizing an array which apparently impresses interviewers. I thought it might be fun to write it in Ruby. I wrapped the algorithm in a reusable function as well: def permute_array(a) 1.upto(a.length - 1) do |i| j = rand(i + 1) a[i], a[j] = a[j], a[i] end a end # alternate version def permute_array2(a) 0.upto(a.length - 2) do |i| j = rand(a.length - i) a[i], a[j] = a[j], a[i] end a end […]
I think in the first loop it is i = 0
but correct me if I am wrong
Nice to know for a future job interview
Nice post
Your first algorithm is wrong, see:
http://www.cigital.com/papers/download/developer_gambling.php
Very useful article mate. In fact it inspired me so much that I wrote the algorithm in Python. Here is my implementation:
import random
array = [2,3,-1,9,8,11,99] #The array to be permuted.
# This could be in a loop to generate the random permutations and print them as well.
for i in range(len(array)-1):
j=random.randint(0,i+1)
array[i], array[j] = array[j], array[i]
print array
Just for an afternoon break ;-). It is indeed a very useful piece of algorithm. Thanks for sharing it.
@ MrX_TLO
It also works for 16 bit systems, as well as 32 bit systems, if you have the time and the right quote.
Also, RAND_MAX only works with odd calculations.
There is no cow level.
Python module “random” comes with a shuffle method that does this in place permutation.
You can even pass your own random function as argument. Take a look at random.py in the python distribution, certainly funnier than C or Java.
[…] Algo Blog » Random Permutation (tags: algorithms programming) […]
@Simon
The article you linked shows that the following is wrong.
for (i is 1 to n)
Swap i with random position between 1 and n
This is a different algorithm than the one presented on this blog. The cigital article algorithm (for a size 3 list) randomly swaps item 1 into a random position from 1 to 3, and does the same for items 2 and 3.
The algorithm here first swaps item 1 between position 1 and 2, then item 2 into a position from 1 to 3.
Just try both of them out, you’ll see the difference immediately.
Hmm, I think this is a valid python implementation
import random
def permute(a):
for i in range(0,len(a)):
j = random.randrange(0,i + 1)
a[i],a[j] = a[j],a[i]
return a
hi
Some people have asked what use a random permutation is. I maintain a ski racing program which uses the Knuth algorithm to produce the start order.
alert(”WAL”);
Nice site
2955c6211c754ff9ef37…
2955c6211c75…
buy wholesale adderall…
news…
side effects of drug lexapro…
news…
buy percocet…
news…
buy viagra online…
news…
buy hydrocodone…
news…
discount phentermine…
news…
long term use of percocet…
news…
buy adderall without a prescription…
news…
ritalin…
news…
discount viagra…
news…
buy hydrocodone without a prescription…
news…
buy ambien…
news…
amoxicillin acne…
news…
ephedra yellow swarm…
news…
buy cialis…
news…
oxycodone online…
news…
online phentermine…
news…
buy vicodin without script…
news…
vaspro ephedrine…
news…
valium no prescription…
news…
fioricet cod…
news…
lexapro medicine…
news…
wellbutrin xl…
news…
hydrocodone apap…
news…
what is oxycodone…
news…
buy phentermine…
news…
biotek ephedrine…
news…
buy valium online…
news…
oxycodone extraction iv…
news…
percocet 93-490…
news…
adderall without a prescription…
news…
ritalin online…
news…
side effects of wellbutrin…
news…
pill propecia…
news…
zoloft common side effects…
news…
ephedra pills…
news…
tramadol side effects…
news…
adderall…
news…
phentermine without prescription…
news…
hydrocodone no prescription…
news…
xanax side effects…
news…
health risks of ephedrine…
news…
free viagra…
news…
ephedra…
news…
buy amoxicillin without prescription…
news…
generic percocet…
news…
natural herb reverse impotence from propecia…
news…
buy valium…
news…
meridia no prescription…
news…
buy fioricet…
news…
nice article…
nice article…
viagra hgh…
news…
propecia and online drugs stores…
news…
prescription weight loss medications - meridia…
news…
fioricet free shipping…
news…
soma and addiction…
news…
nice article…
nice article…
nice article…
nice article…
side effects of percocet…
news…
long-term side effects of ritalin…
news…
hydrocodone…
news…
nice article…
nice article…
Your Honours save us from…
Justice Brian Sully at his Supreme Court retirement ceremony gave an all-embracing spray to law reformers (”unstable obsessives”), bureaucrats (”all…
ephedra liquid gel products…
news…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
Schaffner EMC Increases Attenuation Performance…
… household appliance, test and measurement, power supply and electric data processing industries. In addition, the filter’s low leakage…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
New High-Tech Products for Underwater…
The Intova IC700 Underwater Digital Camera 7MP has Image resolution up to 2816 x 2112 with a 4x digital zoom….
Checking the Checkers - The…
The armed Securitas security guard in question was able to get state certification to work in Portland from the Department…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…
nice article…