Markovian discrimination in spam filtering is a method used in CRM114 and other spam filters to model the statistical behaviors of spam and nonspam more accurately than in simple Bayesian methods. A simple Bayesian model of written text contains only the dictionary of legal words and their relative probabilities. A Markovian model adds the relative transition probabilities that given one word, predict what the next word will be. It is based on the theory of Markov chains by Andrey Markov, hence the name. In essence, a Bayesian filter works on single words alone, while a Markovian filter works on phrases or entire sentences.

Property Value
dbo:abstract
  • Das Markow-Filter (nach Andrei Andrejewitsch Markow) ist ein Spamfilter basierend auf einem Hidden Markov Model und stellt eine Weiterentwicklung des Bayes-Filters dar. Das Markow-Filter errechnet dabei die Wahrscheinlichkeit, mit der die Wortketten des überprüften Textes zu Wortketten typischer Spamtexte passen. Während bei einem Bayes-Filter die Wahrscheinlichkeit einzelner Wörter errechnet wird, zieht das Markow-Filter Wortketten zur Ermittlung der Wahrscheinlichkeit heran und gewichtet die einzelnen Kombinationsmöglichkeiten. Ähneln die Wortketten des überprüften Textes denen typischer Spamtexte, so gilt der überprüfte Text als Spam. (de)
  • Markovian discrimination in spam filtering is a method used in CRM114 and other spam filters to model the statistical behaviors of spam and nonspam more accurately than in simple Bayesian methods. A simple Bayesian model of written text contains only the dictionary of legal words and their relative probabilities. A Markovian model adds the relative transition probabilities that given one word, predict what the next word will be. It is based on the theory of Markov chains by Andrey Markov, hence the name. In essence, a Bayesian filter works on single words alone, while a Markovian filter works on phrases or entire sentences. There are two types of Markov models; the visible Markov model, and the hidden Markov model or HMM.The difference is that with a visible Markov model, the current word is considered to contain the entire state of the language model, while a hidden Markov model hides the state and presumes only that the current word is probabilistically related to the actual internal state of the language. For example, in a visible Markov model the word "the" should predict with accuracy the following word, while ina hidden Markov model, the entire prior text implies the actual state and predicts the following words, but doesnot actually guarantee that state or prediction. Since the latter case is what's encountered in spam filtering,hidden Markov models are almost always used. In particular, because of storage limitations, the specific typeof hidden Markov model called a Markov random field is particularly applicable, usually with a clique size ofbetween four and six tokens. (en)
dbo:wikiPageID
  • 3517732 (xsd:integer)
dbo:wikiPageRevisionID
  • 544240988 (xsd:integer)
dct:subject
http://purl.org/linguistics/gold/hypernym
rdf:type
rdfs:comment
  • Das Markow-Filter (nach Andrei Andrejewitsch Markow) ist ein Spamfilter basierend auf einem Hidden Markov Model und stellt eine Weiterentwicklung des Bayes-Filters dar. Das Markow-Filter errechnet dabei die Wahrscheinlichkeit, mit der die Wortketten des überprüften Textes zu Wortketten typischer Spamtexte passen. Während bei einem Bayes-Filter die Wahrscheinlichkeit einzelner Wörter errechnet wird, zieht das Markow-Filter Wortketten zur Ermittlung der Wahrscheinlichkeit heran und gewichtet die einzelnen Kombinationsmöglichkeiten. Ähneln die Wortketten des überprüften Textes denen typischer Spamtexte, so gilt der überprüfte Text als Spam. (de)
  • Markovian discrimination in spam filtering is a method used in CRM114 and other spam filters to model the statistical behaviors of spam and nonspam more accurately than in simple Bayesian methods. A simple Bayesian model of written text contains only the dictionary of legal words and their relative probabilities. A Markovian model adds the relative transition probabilities that given one word, predict what the next word will be. It is based on the theory of Markov chains by Andrey Markov, hence the name. In essence, a Bayesian filter works on single words alone, while a Markovian filter works on phrases or entire sentences. (en)
rdfs:label
  • Markow-Filter (de)
  • Markovian discrimination (en)
owl:sameAs
prov:wasDerivedFrom
foaf:isPrimaryTopicOf
is dbo:wikiPageDisambiguates of
is foaf:primaryTopic of