Re: diehard and ent results quesion

From: Bryan Olson (fakeaddress@nowhere.org)
Date: 02/20/03 

From: Bryan Olson <fakeaddress@nowhere.org>
Date: Thu, 20 Feb 2003 20:23:40 GMT

Douglas A. Gwyn wrote:
> Bryan Olson wrote:
>
>>The obvious question, open at least in this thread: is there a
>>simple computation for entropy, under the same assumptions, that
>>is unbiased?
>
> Usually for small samples (where this matters) I instead use
> Kullback's information statistic to discriminate between
> hypotheses.

That's not the situation here. If we knew of a statistic that
should differ between a given generator and a random source,
then we'd be in the market for likelihood estimation.

Here, we're given the stream and asked if it's random. We may
be able to find a statistic that lets us confidently reject the
hypothesis that it's random, but no matter how many tests we
run, we can never reject that it's non-random.

That said, there's every reason to ignore ent's so-called
"entropy" statistic. It's labeling in the output has misguided
many people. It's computed solely from the frequency counts,
and as Cristiano pointed out, the Chi-square test does a much
better job of that. The fact that it's biased is just one more
defect. 

-- 
--Bryan

Relevant Pages

  • Re: diehard and ent results quesion
    ... Douglas A. Gwyn wrote: ... > Bryan Olson wrote: ... >> simple computation for entropy, under the same assumptions, that ...
    (sci.crypt)
  • Re: diehard and ent results quesion
    ... Bryan Olson wrote: ... > You mean given the samples, estimate the entropy of the source? ... > appears with near-equal frequency. ... And is not the chi-square test better? ...
    (sci.crypt)
  • Re: A REAL LIVE Reverse Entropy Machine-Warning
    ... entropy sparse languages (highly redundant languages)- something you can ... easily prove whenever you communicate in a poorly known second language. ... Well Douglas was built on a ATARI ST using Basic in the end of the ...
    (sci.crypt)
  • Re: true random number generator
    ... Douglas A. Gwyn wrote: ... > I could give it a precise definition, ... > entropy pool be consumed at a rate considerably ... > below half the estimated entropy, ...
    (sci.crypt)
  • Re: true random number generator
    ... Douglas A. Gwyn wrote: ... > Entropy is always relative to a state of knowledge; ... unforeseen bias in a typical hardware ...
    (sci.crypt)
Loading