An Entity of Type: Thing, from Named Graph: http://dbpedia.org, within Data Space: dbpedia.org

In information theory, the entropy of a random variable is the average level of "information", "surprise", or "uncertainty" inherent in the variable's possible outcomes. The concept of information entropy was introduced by Claude Shannon in his 1948 paper "A Mathematical Theory of Communication", and is also referred to as Shannon entropy. As an example, consider a biased coin with probability p of landing on heads and probability 1 − p of landing on tails. The maximum surprise is for p = 1/2, when there is no reason to expect one outcome over another. In this case a coin flip has an entropy of one bit. The minimum surprise is for p = 0 or p = 1, when the event is known and the entropy is zero bits. When the entropy is zero bits, this is sometimes referred to as unity, where there is no un

Property Value
dbo:abstract
• In information theory, the entropy of a random variable is the average level of "information", "surprise", or "uncertainty" inherent in the variable's possible outcomes. The concept of information entropy was introduced by Claude Shannon in his 1948 paper "A Mathematical Theory of Communication", and is also referred to as Shannon entropy. As an example, consider a biased coin with probability p of landing on heads and probability 1 − p of landing on tails. The maximum surprise is for p = 1/2, when there is no reason to expect one outcome over another. In this case a coin flip has an entropy of one bit. The minimum surprise is for p = 0 or p = 1, when the event is known and the entropy is zero bits. When the entropy is zero bits, this is sometimes referred to as unity, where there is no uncertainty at all - no freedom of choice - no information. Other values of p give different entropies between zero and one bits. Given a discrete random variable , with possible outcomes , which occur with probability the entropy of is formally defined as: where denotes the sum over the variable's possible values. The choice of base for , the logarithm, varies for different applications. Base 2 gives the unit of bits (or "shannons"), while base e gives "natural units" nat, and base 10 gives units of "dits", "bans", or "hartleys". An equivalent definition of entropy is the expected value of the self-information of a variable. The entropy was originally created by Shannon as part of his theory of communication, in which a data communication system is composed of three elements: a source of data, a communication channel, and a receiver. The "fundamental problem of communication" – as expressed by Shannon – is for the receiver to be able to identify what data was generated by the source, based on the signal it receives through the channel. Shannon considered various ways to encode, compress, and transmit messages from a data source, and proved in his famous source coding theorem that the entropy represents an absolute mathematical limit on how well data from the source can be losslessly compressed onto a perfectly noiseless channel. Shannon strengthened this result considerably for noisy channels in his noisy-channel coding theorem. Entropy in information theory is directly analogous to the entropy in statistical thermodynamics. The analogy results when the values of the random variable designate energies of microstates, so Gibbs formula for the entropy is formally identical to Shannon's formula. Entropy has relevance to other areas of mathematics such as combinatorics. The definition can be derived from a set of axioms establishing that entropy should be a measure of how "surprising" the average outcome of a variable is. For a continuous random variable, differential entropy is analogous to entropy. (en)
dbo:thumbnail
dbo:wikiPageID
• 15445 (xsd:integer)
dbo:wikiPageLength
• 56791 (xsd:nonNegativeInteger)
dbo:wikiPageRevisionID
• 1056548280 (xsd:integer)
dbp:backgroundColour
• #F5FFFA00 (en)
dbp:borderColour
• #0073CF00 (en)
• 6 (xsd:integer)
dbp:id
• 968 (xsd:integer)
• p/e035740 (en)
dbp:title
• Entropy (en)
• Shannon's entropy (en)
dbp:wikiPageUsesTemplate
dct:subject
rdf:type
rdfs:comment
• In information theory, the entropy of a random variable is the average level of "information", "surprise", or "uncertainty" inherent in the variable's possible outcomes. The concept of information entropy was introduced by Claude Shannon in his 1948 paper "A Mathematical Theory of Communication", and is also referred to as Shannon entropy. As an example, consider a biased coin with probability p of landing on heads and probability 1 − p of landing on tails. The maximum surprise is for p = 1/2, when there is no reason to expect one outcome over another. In this case a coin flip has an entropy of one bit. The minimum surprise is for p = 0 or p = 1, when the event is known and the entropy is zero bits. When the entropy is zero bits, this is sometimes referred to as unity, where there is no un (en)
rdfs:label
• Entropy (information theory) (en)
owl:sameAs
prov:wasDerivedFrom
foaf:depiction
foaf:isPrimaryTopicOf
is dbo:knownFor of
is dbo:wikiPageDisambiguates of
is dbo:wikiPageRedirects of