Re: FFT test with few kbits

From: Ernst Lippe (ernstl-at-planet-dot-nl_at_ignore.this)
Date: 02/29/04


Date: Sun, 29 Feb 2004 21:02:47 +0100

On Fri, 27 Feb 2004 21:00:14 +0000, Paul Pires wrote:

> Take a moment to look at the common sense. Say this test
> is deployed, say it routinely passes good random streams.

But what evidence do you require before you would say that good random
streams routinely pass this test?

This would be easy, if we had a proof that Christiano's statistic
(asymptotically) follows the KS-distribution, but I hope that you can
agree, that we currently don't have such a proof.

Basically, that leaves just one other option: Monte Carlo
simulations. Essentially, what you are doing then is approximating the
empirical distribution of a statistic. Of course this is a valid
approach for any statistic. The only slightly tricky point is to
determine the number of simulations that you need. When someone has
done these simulations for a given value of N, you can of course use
these results in other cases where the sample size is also equal to
N. But what should you do when want to use this test with a different
value for N? The most prudent approach (which AFAIK is also followed
by statistical packages such as Diehard and NIST's) is to assume that
the test should only be used for the values of N for which simulations
have been performed. When you believe that we can be more lenient,
i.e. that we can also use the test even for values of N for which
there are no simulation results, could you explain why and under which
conditions? (One of the reasons why I would be cautious about
extrapolations is that the Fourier components show some curious
patterns that depend on the prime factors in N, and I would not be
surprised if the prime factorization also influenced the distribution)

>From a cryptographic point of view, of course, we are only interested
to see if the test can be used as a discriminator. But one of the
problems at the current stage is that it cannot be used in
isolation. For any other statistical test it would be reasonable to
reject a RNG based on the sole fact that it consistently fails that
test. At the current stage, we can't do that with Christiano's test,
because we must also show that a good RNG would not have failed as
well, something that is not necessary for statistical tests with a
known distribution (it does not matter if that distribution was
computed from its analytical solution or by simulations).

> I think it would be more usefull at this point to characterize the test rather
> than "prove" it. Make a matrix of tested outputs of many generators tested
> with many popular tests and also your proposed test. Post the results.

That's a good idea of course.

Ernst Lippe



Relevant Pages

  • Re: 11 pts opposite weak NT
    ... run on a dealer using normal weak NT constraints (ie 4432,4333,5332 shapes ... Your have done enough simulations that the difference between your two ... On second thought I doubt that it matters, to the HC distribution, ... extrapolate that the missing lower right block -- the most relevant ...
    (rec.games.bridge)
  • Re: Question for the statisticians
    ... "According to the first part of the Central Limit Theorem, ... sampling distribution of means is a normal distribution if the ... If one were to run 30 separate 50 hand simulations of the same ...
    (rec.games.bridge)
  • Re: Question for the statisticians
    ... about the Central Limit Theorem: ... sampling distribution of means is a normal distribution if the ... not apply) to the issue of bridge simulations. ...
    (rec.games.bridge)
  • Re: Normally distributed integer numbers
    ... >> distribution normal. ... > NT is the number of simulations done, ... > What are Nmin and Nmax referring to, ...
    (sci.stat.math)
  • Re: RNGs an their accuracy
    ... and assign each result to one of the 36 possible dice rolls. ... I've been doing enormous amounts of number-crunching of lottery data, particularly 6/49 format, to investigate small but persistent effects that don't appear to be predicted by current mathematical theory. ... it seems to be more effective to combine lots of extremely short simulations from a crap RNG than it is to have a very long simulation using the Mersenne Twister. ...
    (rec.gambling.craps)