Skip to main content

Module features

Module features 

Source
Expand description

The header features, version 3: what a message’s headers and shape say, as named numbers, read by the classifier as words of their own beside the message’s words (header_words), so that each can be explained by its name (docs/spam-filter.md, “The header features”).

They are read from the raw header block (RawHeaders), from the authentication results (auth_results: Sioul’s own first, your provider’s second), from the message’s parts (card::Shape) and from the links its text holds (tokenize::Tokens::links); never from the order of the headers. Left out, because they would teach the filter where mail was collected rather than what it is: To and Cc, the account, dates as such, the provider’s own marks.

A value can be missing (f32::NAN): the checks when neither Sioul nor a provider you trust wrote any; the route where it cannot be read. A missing value stands at the training’s mean, which is over the messages that have it: it weighs nothing (table::Table::score). So mail from a provider that writes no results, or reached through one whose lines are written another way, is not told apart by that alone.

Version 2 (from 7 October 2026), measured on a real mailbox: version 1 taught the filter which account a message came to. Only one provider wrote results that were trusted, so “no results” meant “that other account”, whose share of spam the filter then learned; the count of Received lines and ARC were each provider’s own too. Version 2 makes missing checks missing (not “none”), drops the count of hops, ARC and the provider’s verdict (the Porch applies that verdict itself, as its own rule), and adds what the message carries wherever it is received: a DKIM signature, one by the sender’s own domain, a bounce address at it.

Version 3 (the same day): the same facts, read by fastText’s classifier as words (header_words: a fact that holds, its name; a number, its bin) rather than weighed by an SVM beside the mean of the words’ vectors.

Version 4 (the same evening): the checks are Sioul’s own first, your provider’s second (auth_results). Sioul checks each message whole as it stores it (its stamp on top of it, sioul_sync::verify), and the training corpus now keeps the same checks, made on the whole message when it fetched it (sioul_learn::corpus::Checked, read with stamp_results): the training and the sorting weigh the same verifier’s word. Before, the corpus had no such checks, and both read the provider’s alone, which a shared host may never write: there, every check was missing, and only a signature’s unverified claim was left.

How a result reads (features): a pass, the fact holds (DKIM: a signature verified against its key in the DNS; SPF: the server that handed the message to your provider is one its domain names; DMARC: the sender’s domain vouches for it); a fail or a softfail, the failure holds; anything else is neither, unknown, never a failure: no signature or no record (“none”), a key not found or revoked, a DNS error (“permerror”, “temperror”), a signature expired or a body changed since it was signed (“neutral”, as mail-auth says them). Checks made long after the message came (LATE: the corpus’s, of old mail) keep their passes, for what passes today passed then (the key that verifies the signature, the server its domain still names). Their failures hold only when the message carries no DKIM signature (late, signed): a signature that no longer verifies (expired, its key gone or rotated, some under the same selector) may have passed then, and DMARC with it, so beside one every failure reads unknown; a message without one has nothing of its own to expire, and a sender spoofed then is spoofed now (its domain’s records do not turn the failure into a pass).

The same function runs on the desktop when training and on every device when sorting: a change of a feature, of its order or of its scale raises FEATURES, and a table made with others is refused.

Enums§

Kind 🔒
What an attached or inline part is, by its name’s extension or its type.

Constants§

FEATURES
The header features’ version, stamped in every table.
FREE_MAIL 🔒
Mail providers whose domains millions share, for reply_to_freemail. Part of what a table learned, so versioned with FEATURES: never the words you change (words::BrandWords::shared holds the same list as shipped on 8 October 2026, and may grow; this one may not, unless FEATURES rises).
LATE
How long after a message came its checks read as they are; later, the failures of a signed message read unknown (see the module): 30 days, in seconds.
N
How many header features there are.
NAMES
Each feature’s name, in the order of features’ values.

Functions§

address_domain 🔒
The domain of a header’s first address: “Name a@b.example”, “a@b.example”.
all_header_words
Every word header_words can give, each with its feature’s place: what the classifier may have learned of the header facts.
auth_results
The authentication results the features read of a message as it is stored: Sioul’s own first (its stamp, under this installation’s id, which ends in .invalid), your provider’s second (the account’s other trusted ids), as the lanes, the forgeries and the senders read them. The training corpus keeps the message as the server holds it, without the stamp, and Sioul’s checks beside it (sioul_learn::corpus::Checked), which it reads with stamp_results first: the same verifier on both sides.
bare_ip 🔒
A HELO name that is an address: [192.0.2.1], 203.0.113.5, [IPv6:2001:db8::1].
dkim_domain 🔒
The signing domain of a DKIM-Signature (its d= tag), lowercase.
features
The header features of a message: its card (headers, parts, text), its authentication results (auth_results, or the corpus’s own checks), when its provider received it (IMAP’s INTERNALDATE, the Maildir file’s time), the domains of its links (tokenize::tokens). A value it cannot know is f32::NAN (see the module).
header_words
The header facts as words, as the classifier reads them after the message’s own (table::Kind::Rows), each with its feature’s place in NAMES: a fact that holds, H: and its name in capitals; a number in a few bins (the Date an hour or more ahead; an hour or a day or more behind; no link, 1 to 3, 4 to 10, 11 or more; links elsewhere, half of them or more); a fact that does not hold, or is missing, none. No word of a message can be one of them: the tokenizer lowercases every word and splits on punctuation, and its own placeholders read _URL_; so no sender writes a header fact into a text.
index
A feature’s place, by its name.
late
Checks made long after the message came (LATE), of a message that carries a DKIM signature or not (signed): signed, their passes only, every other result unknown; unsigned, as they are (see the module).
names_domain 🔒
Whether a display name names an address or a domain other than the sender’s own: “service@bank.example x@elsewhere.example”, “bank.example Security”.
signed
Whether a message carries a DKIM signature (one naming its domain, d=), whatever became of it since: what dkim_signed says, and what tells old mail’s failures apart (late).
stamp_results
What a stamp of Sioul’s says (“Authentication-Results: sioul-….invalid; dkim=pass …”), whatever installation’s id it bears: for a stamp kept apart from its message, where no sender could have written it (the training corpus’s Checked); none when it says nothing readable.