番茄花园-3TypicalWorkonAutomaticRelationExtraction.ppt

3 Typical Work on Automatic Relation Extraction 自动关系抽取的三种重要方法 武文娟 2009.06.04 Outline DIPRE,1998 KnowItAll, 2005 Open IE, 2007 1 DIPRE: Dual Iterative Pattern Expansion Sergey Brin, Extracting Patterns and Relations from the World Wide Web, In : Proceedings of the International Workshop on the Web and Databases, 1998. 1 DIPRE: Dual Iterative Pattern Expansion 首次利用迭代方法发现数据实体间的模式和关系，并成功的发现了作者/作品数据对。 Input: 5本书的样本集(author, title) Output: 自动扩展到了15,000本书 有些书是最大的网上书店亚马逊也没有的。 1.1 Idea 1.2 Algorithm Pattern generation 1.3 Experiments Corpus A repository of 24 million web pages 147G 1.3 Experiments: Initial sample 1.3 Experiments: 3 Patterns in First Iteration 1.3 Experiments: 4047 new pairs in First Iteration 1.3 Experiments: review 1.4 Conclusion DIPRE： 半监督关系学习方面的最初的工作 利用了关系和模板之间的对偶性，在Web这样的大规模语料库上，通过少量的sample作为种子，以迭代的方法，不断地抽取新的模板和实例。 Outline DIPRE,1998 KnowItAll, 2005 Open IE, 2007 KNOWITALL Oren Etzioni etc. University of Washington Unsupervised Named-Entity Extraction from the Web: An Experimental Study AAAI 2005 Introduction 以前的工作：HMM, CRF 小规模的语料库 需要提供种子数据 KNOWITALL: an unsupervised, domain-independent system that extracts information from the Web 关键挑战: 保证准确率：a novel generate-and-test architecture 提高召回率： Pattern Learning (PL) Subclass Extraction (SE) List Extraction (LE) 1 Flowchart of the main components in KnowItAll Information Focus 唯一领域相关的输入是一组predicate，用来指定所关注的领域。 通用的抽取模板 Extraction Rules 通用的抽取模板，结合predicate的标签，生成相应领域的抽取规则 Class1 = ‘city’，规则即为 “cities such as ” NPList “towns such as ” NPList Keywords: “cities such as ” , “towns such as ” （提交给搜索引擎） Discriminator 用来确认某个抽取到的信息是否validate 利用PMI Training discriminator: Bootstrapping The result of training A set of discriminator, eg. Discriminator: I is a city Learned threshold T: 0.000016 Conditional probabilities P(PMI T | class) = 0.83 P(PMI T | ?class) = 0.08 An Example Predicate: city Bootstrapping: Generate extraction rules and discriminators Train all discriminators, and selected the 5 b