[feature] variants of tf-idf weighting

Closes #8 (closed)

Implements three different variants of tfidf, while the main differences are in the calculation of the tf.

1. natural: tf x log(N/df)

2. logarithmic: log(1+tf) x log(N/df)

3. augmented: (0.5 + (0.5 * tf) / max(tf))) x log(N/df)

Even thought natural and logarithmic have similar performance, the results from the augmented variant looks very off. On a closer look we can see that most sim values are between 0.8 and 1.

added enhancement label

added 1 commit

• 04584839 - [feature] variants of tf-idf weighting

Compare with previous version

added 1 commit

• e7e4cfa6 - [feature] variants of tf-idf weighting

Compare with previous version

added 1 commit

• c5d99302 - [feature] variants of tf-idf weighting

Compare with previous version

unmarked as a Work In Progress

changed the description

mentioned in commit 096da6a7

merged

mentioned in issue #9 (closed)