Quantcast
Channel: User Fixed Point - Mathematics Stack Exchange
Viewing all articles
Browse latest Browse all 36

Answer by Fixed Point for How to test a computer language random number generator

$
0
0

There are many many tests. First these are all statistical tests. You first decide on a $p$-value, your cutoff, usually 5% for normal everyday use (gaming, scientific simulations stuff) and 1% for very stringent applications like cryptographic applications. Then you run one of these statistical tests which returns a $p$-value which you compare with your cutoff. The smaller the returned $p$-value the better. If the returned $p$-value is less than your cutoff then consider the test "passed" meaning your random number generator is "probably" good. If the returned $p$-value is larger than your cutoff then your random number generator has failed.

There are no tests to tell you with certainty that your RNG is truly random. There are only tests to tell you if your RNG is bad. If you fail a test then your RNG has obvious patterns in it. If it doesn't fail then you don't know, it may have other patterns in it. So if your RNG passes a test then you can have some faith that it is probably good. The more test you pass obviously the more confidence you have on your RNG.

I looked into this very carefully a few years ago and there are about 40 tests more commonly employed and they are kind of split up in two different "suites" which have become a defacto standard. If you change the -1 to 0 then you can treat them like binary bits, your RNG is spitting out bits.

  • The basic tests include counting zero and one to see if they are roughly equal.
  • You can also count the subsequences 00,01,10,11 to see if they roughly occur quarter of the time each.
  • You can also count the subsequences 000,001,010,100,011,101,110,111 to see if they roughly occur 12.5% each.
  • You can also count all subsequences of length four.....and so on.

  • You can take a subsequence of length 9, form a 3x3 matrix and determine its binary rank which will tell you if the vectors are linearly independent or not.

  • You take a subsequence of length 16, form a 4x4 matrix and determine its binary rank....
  • You can take a subsequence of length 25, for a 5x5 matrix and ....

  • You can also pick a length (powers of two are faster) and take the DFT to look at the spectrum to see if there are any other periodicities.

And so on. The two batteries are the Diehard tests and the suite developed by NIST to test cryptographic strength RNG. I like the one by NIST. They have extensive documentation which is a pretty cool document in its own right. And they provide the suite to you. Get on a linux machine. Download it. Compile it and run pointing to your file with the random bits in it. You literally just pick the tests and it runs them and give you all the $p$-values. It is super easy, has a ton of tests, and is optimized and fast. The documentation provides complete theory and details behind the tests and how is the $p$-value computed and to be interpreted and how to use the tests. Very well done indeed. If you have any math interests, I would recommend taking a good look at the entire document, learned some really cool stuff.

The more tests your RNG passes the better and I would very very impressed if it passes all of them. You should expect to pass most and fail a few. Even the best of the best RNGs fail one test or another. So just because you fail a few, your RNG is not automatically crap. Lastly, you just keep coming up with more and more tests but you stop after a while. After a while you are reasonably sure if your RNG is "good" or not. There is no one definitive test. Also, coming up with a good RNG is very very difficult so instead of cooking up one of your own you might want to get any one of the million libraries out there to do random number generation for you.


Viewing all articles
Browse latest Browse all 36

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>