Automatic text decomposition using text segments and text themes

Abstract

levelevidenceabouttheuseofwordsinlocalcontextstrievaltaskandimprovingretrievaleectiveness.Passagebyimportanttextexcerpts,therebysimplifyingtheretrieval,passage-retrievaltechniquesarebecomingincreasinglypopular.LargertextscanthenbereplacedWiththewidespreaduseoffull-textinformationre- textpassages,astudyoftextpassagesisalsoimportant traversal,andtextsummarization. toformulatespecicationsforinformationretrieval,text themesisthenusedtocharacterizetextstructure,and exhibitstheresultsofsimilaritymeasurementsbetween knowledgeoftexttypeandtextstructureinturnaects pairsoftexts,ortextexcerpts.Typically,eachtext,or manytexthandlingoperations,includingretrieval,text textexcerptisrepresentedbyavectorofweightedterms readingandtraversal,andtextsummarization. ments,themes,informationretrieval,passageretrieval,oftheformDi=(di1;di2;:::;:::;dit)wheredikrepre- Thestructureofindividualtexts,orsetsofrelatedtexts, canbestudiedbyusingatextrelationshipmapthat Withtheadventoffull-textdocumentprocessing,the textsummarization. sentsanimportanceweightfortermTkattachedtodoctentrepresentationpurposesmaybewordsorphrases interestinmanipulatingtextpassagesratherthanonly wordusageinlocaltextenvironmentsisoftenhelpfulin full-textitemshascontinuedtogrow.Retrievinglarge improvingretrievaleectiveness,becausethemeaningof textsinanswertouserqueriestendstobeinecient thetermsintheindividualdocumentsandthedocument texts.Inaddition,passage-levelevidenceaccountingfor ambiguoustermsbecomesclearwhenthelocalcontext becausetheuseristhenforcedtocopewithlargemasses collectionasawhole.[7] umentDi.Thetermsattachedtodocumentsforconderivedfromthedocumenttextsbyanautomaticin- bytakingintoaccounttheoccurrencecharacteristicsof dexingprocedure,andthetermweightsarecomputed oftext,andineectivebecauserelevanttextpassages oftenprovidebetteranswersthancompletedocument

Keywords

Affiliated Institutions

• Cornell University US

Related Publications