1. introduction 2. retrieval strategies 2.1 vector space model 2.2 probabilistic retrieval strategies 2.3 language models 2.4 inference works 2.5 extended boolean retrieval 2.6 latent semantic indeng 2.7 neural works 2.8 geic algorithms 2.9 fuzzy set retrieval 2.10 summary 2.11 exercises 3. retrieval utilities 3.1 relevance feedback 3.2 clustering 3.3 passage-based retrieval 3.4 n-grams 3.5 regression analysis 3.6 thesauri 3.7 semantic works 3.8 parsing 3.9 summary 3.10 exercises 4. cross-language information retrieval 4.1 introduction 4.2 crossing the language barrier 4.3 cross-language retrieval strategies 4.4 cross language utilities 4.5 summary 4.6 exercises 5. efficiency 5.1 inverted index 5.2 query processing 5.3 signature files 5.4 duplicate document detection 5.5 summary 5.6 exercises 6. integrating structured data and text 6.1 review of the relational model 6.2 a historical progression 6.3 information retrieval as a relational application 6.4 semi-structured search using a relational schema 6.5 multi-dimensional data model 6.6 mediators 6.7 summary 6.8 exercises 7. parallel information retrieval 7.1 parallel text scanning 7.2 parallel indeng 7.3 clustering and classification 7.4 large parallel systems 7.5 summary 7.6 exercises 8. distributed information retrieval 8.1 a theoretical model of distributed retrieval 8.2 web search 8.3 result fusion 8.4 peer-to-peer information systems 8.5 other architectures 8.6 summary 8.7 exercises 9. summary and future directions references index
3.4.1 damore and mah initial information retrieval research focused on n-grams as presented in[damore and mah, 1985]. the motivation behind their work was the fact thatit is difficult to develop mathematical models for terms since the potential fora term that has not been seen before is infinite. with n-grams, only a fixednumber of n-grams can est for a given value of n. a mathematical modelwas developed to estimate the noise in indeng and to determine appropriatedocument similarity measures. damore and mahs method replaces terms with n-grams in the vector spacemodel. the only remaining issue is puting the weights for each n-gram.instead of simply using n-gram frequencies, a scaling method is used to nor-malize the length of the document. damore and mahs contention was that alarge document contains more n-grams than a small document, so it should bescaled based on its length. to pute the weights for a given n-gram, damore and mah estimatedthe number of occurrences of an n-gram in a document. the first simplifyingassumption was that n-grams occur with equal likelihood and follow a binomialdistribution. hence, it was no more likely for n-gram "abc" to occur than"dee" the zipfian distribution that is widely accepted for terms is not true forn-grams. damore and mah noted that n-grams are not equally likely to occur,but the removal of frequently occurring terms from the document collectionresulted in n-grams that follow a more binomial distribution than the terms. damore and mah puted the expected number of occurrences of an n-gram in a particular document. this is the product of the number of n-gramsin the document (the document length) and the probability that the n-gramoccurs. the n-grams probability of occurrence is puted as the ratio ofits number of occurrences to the total number of n-grams in the document.damore and mah continued their application of the bino ……