org.apdplat.word.analysis

Class SimHashPlusHammingDistanceTextSimilarity

java.lang.Object

org.apdplat.word.analysis.TextSimilarity

org.apdplat.word.analysis.SimHashPlusHammingDistanceTextSimilarity

All Implemented Interfaces:

Similarity, SimilarityRanker

public class SimHashPlusHammingDistanceTextSimilarity
extends TextSimilarity

文本相似度计算 判定方式：SimHash + 汉明距离（Hamming Distance） 先使用SimHash把不同长度的文本映射为等长文本，然后再计算等长文本的汉明距离 simhash和普通hash最大的不同在于: 普通hash对 仅有一个字节不同的文本 会映射成 两个完全不同的哈希结果 simhash对 相似的文本 会映射成 相似的哈希结果 汉明距离是以美国数学家Richard Wesley Hamming的名字命名的 两个等长字符串之间的汉明距离是两个字符串相应位置的不同字符的个数 换句话说，它就是将一个字符串变换成另外一个字符串所需要替换的字符个数 比如： 1011101 与 1001001 之间的汉明距离是 2 2143896 与 2233796 之间的汉明距离是 3 toned 与 roses 之间的汉明距离是 3

Author:

杨尚川

Field Summary

Fields inherited from class org.apdplat.word.analysis.TextSimilarity

filterStopWord

Fields inherited from interface org.apdplat.word.analysis.Similarity

thresholdRate

Constructor Summary

Constructors

Constructor and Description
SimHashPlusHammingDistanceTextSimilarity()
SimHashPlusHammingDistanceTextSimilarity(int hashBitCount)

Method Summary

Modifier and Type	Method and Description
int	getHashBitCount()
static void	main(String[] args)
protected double	scoreImpl(List<Word> words1, List<Word> words2) 计算相似度分值
void	setHashBitCount(int hashBitCount)

Methods inherited from class org.apdplat.word.analysis.TextSimilarity

setSegmentationAlgorithm, similarScore, similarScore, taggingWeightWithWordFrequency, toFastSearchMap

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apdplat.word.analysis.SimilarityRanker

rank, rank

Methods inherited from interface org.apdplat.word.analysis.Similarity

isSimilar, isSimilar, isSimilar, isSimilar, similarScore, similarScore

Constructor Detail

SimHashPlusHammingDistanceTextSimilarity

public SimHashPlusHammingDistanceTextSimilarity()

SimHashPlusHammingDistanceTextSimilarity

public SimHashPlusHammingDistanceTextSimilarity(int hashBitCount)

Method Detail

getHashBitCount

public int getHashBitCount()

setHashBitCount

public void setHashBitCount(int hashBitCount)

scoreImpl

protected double scoreImpl(List<Word> words1,
                           List<Word> words2)

计算相似度分值

Specified by:

scoreImpl in class TextSimilarity

Parameters:

words1 - 词列表1

words2 - 词列表2

Returns:

相似度分值

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception