Module spam 

Sioul’s own spam filter, the part every device runs (docs/spam-filter.md): the tokenizer (tokenize), the header features (features), the reduced table that scores both (table), the label log and the log of what the filter moved (labels), and the filter as the Porch asks it (Filter). The training, on one computer, by hand, is the sioul-learn crate’s; it writes the table with table::Table::write, and shares these functions with the phone, so that both read a message the same way.

Where its verdict goes, and what it never touches (people you know, codes, projects, your own mail), is the Porch’s: porch::triage. What it does with each verdict is yours to choose, class by class (Actions).

Modules

features

The header features, version 3: what a message’s headers and shape say, as named numbers, read by the classifier as words of their own beside the message’s words (header_words), so that each can be explained by its name (docs/spam-filter.md, “The header features”).

labels

The label log: what you said of messages, spam or not, for the spam filter to learn from. One JSON line per act, added at the end of a file and never rewritten: “Junk” and “Not junk” (the Mail page, sioul mail), “Spam” and “Not spam” (the Porch’s review queue, the Reader), a sender blocked from one of their messages.

table

The reduced table: the spam filter as every device runs it, a megabyte or two, with no word of your mail in it: each word under a 64-bit hash of it (word_hash), never the word itself, and only words seen five times or more.

tokenize

The spam filter’s tokenizer, version 2: what a message says, as the words the language model learns on the desktop and the table scores on every device.

Structs

Actions

What the filter does with each class: the matrix {probably spam, maybe spam, probably not spam} × {move to spam, flag only, do nothing}.

Filter

The filter as the Porch asks it: your choices, and where its table is.

Verdict

The filter’s verdict on one message: how likely it is spam, from 0 to 1. What weighed stays in the table’s score (table::Score), for sioul spam why in your terminal; the Porch never says it.

Enums

Action

What the filter does with a message of a class, as you chose it (the matrix in Settings ▸ Mail ▸ Your own spam filter).

Class

Where a probability falls between the two thresholds: probably spam, maybe spam, probably not spam.

Constants

REMEMBERED_MOST 🔒

At most this many verdicts kept, then forgotten at once.

THRESHOLD_SPAM

From this probability on, a message is spam, unless said otherwise ([spam] threshold_spam).

THRESHOLD_UNSURE

From this one on, it is unsure ([spam] threshold_unsure).

Statics

REMEMBERED 🔒

Functions

remember 🔒

remembered 🔒

score

A message’s verdict by a table: its words, its header features (its file’s time standing for when its provider received it, as Sioul stores mail), these reading Sioul’s own checks first (its stamp, made as the message was stored), your provider’s second, as the training reads the corpus’s own checks (features::auth_results; trusted_ids: the account’s authserv-ids, Sioul’s own first). The words are read with the table’s own (Table::lexicon).

table_path

The table’s place on this device: $XDG_DATA_HOME/sioul/spam/table.bin.

Type Aliases

Remembered 🔒

The verdicts given with the table in use, by message file: forgotten when the table changes.