the entropy of a discrete real variable熵的离散变量.pdf

Entropy 2012, 14, 1522-1538; doi:10.3390/ OPEN ACCESS entropy ISSN 1099-4300 /journal/entropy Article The Entropy of a Discrete Real Variable Scott Funkhouser SPAWAR Systems Center Atlantic, Joint Base Charleston, North Charleston, SC 29406, USA; E-Mail: scott.funkhouser@ Received: 12 June 2012; in revised form: 3 August 2012 / Accepted: 6 August 2012 / Published: 17 August 2012 Abstract: The discrete Shannon entropy was formulated only to measure indeterminacy effected through a set of probabilities, but the indeterminacy in a real-valued discrete variable depends on both the allowed outcomes and the corresponding probabilities . A fundamental measure that is sensitive to both and is derived here from the total differential entropy of a continuous real variable and its conjugate in the discrete limit, where the conjugate is universally eliminated. The asymptotic differential entropy recovers plus the new measure, named , which provides a novel probe of intrinsic organization in sequences of real numbers. Keywords: Shannon entropy; information entropy; information theory 1. Introduction Let be a discrete variable with generic outcomes and corresponding probabilities such that (1) Expressed in terms of the natural logarithm, the Shannon entropy attributed to in connection with is [ 1]