数据挖掘第六章.ppt

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * Chapter 5: Mining Frequent Patterns, Association and Correlations: Basic Concepts and Methods Basic Concepts Frequent Itemset Mining Methods Which Patterns Are Interesting?—Pattern Evaluation Methods Summary * Interestingness Measure: Correlations (Lift) play basketball ? eat cereal [40%, 66.7%] is misleading The overall % of students eating cereal is 75% 66.7%. play basketball ? not eat cereal [20%, 33.3%] is more accurate, although with lower support and confidence Measure of dependent/correlated events: lift Basketball Not basketball Sum (row) Cereal 2000 1750 3750 Not cereal 1000 250 1250 Sum(col.) 3000 2000 5000 * Are lift and ?2 Good Measures of Correlation? “Buy walnuts ? buy milk [1%, 80%]” is misleading if 85% of customers buy milk Support and confidence are not good to indicate correlations Over 20 interestingness measures have been proposed (see Tan, Kumar, Sritastava @KDD’02) Which are good ones? * Null-Invariant Measures * Data Mining: Concepts and Techniques * Comparison of Interestingness Measures Milk No Milk Sum (row) Coffee m, c ~m, c c No Coffee m, ~c ~m, ~c ~c Sum(col.) m ~m ? Null-(transaction) invariance is crucial for correlation analysis Lift and ?2 are not null-invariant 5 null-invariant measures Null-transactions w.r.t. m and c Null-invariant Subtle: They disagree Kulczynski measure (1927) * Analysis of DBLP Coauthor Relationships Advisor-advisee relation: Kulc: high, coherence: low, cosine: middle Recent DB conferences, removing balanced associations, low sup, etc. Tianyi Wu, Yuguo Chen and Jiawei Han, “Association Mining in Large Databases: A Re-Examination of Its Measures”, Proc. 2007 Int. Conf. Principles and Practice of Knowledge Discovery in Databases (PKDD07), Sept. 2007 Which Null-Invariant Measure Is Better? IR (Imbalance Ratio): measure the imbalance of two itemsets A and B in rule implications Kulczynski and Imbalance Ratio (IR) togeth