pub fn ngram_buckets(word: &str, minn: u32, maxn: u32, bucket: u32) -> Vec<u32>Expand description
The buckets of a word’s character n-grams, as fastText 0.8.0 computes them
(https://github.com/messense/fasttext-rs, tag v0.8.0: src/dictionary.rs,
for_each_ngram and get_subwords_for_string; src/utils.rs, hash):
the word between < and >; from each character (not each byte: UTF-8
continuation bytes start nothing), every n-gram of minn to maxn
characters (a single character never at either end); each hashed
(fasttext_hash) modulo bucket; in that order, repeats kept. fastText’s
input row for one is nwords + bucket, and the vector of a word it never
saw is the mean of those rows (get_word_vector_into); of a word it knows,
the mean of its own row and those.