Expand description
The header features, version 3: what a message’s headers and shape say,
as named numbers, read by the classifier as words of their own beside the
message’s words (header_words), so that each can be explained by its
name (docs/spam-filter.md, “The header features”).
They are read from the raw header block (RawHeaders), from the
authentication results (auth_results: Sioul’s own first, your
provider’s second), from the message’s parts (card::Shape) and from
the links its text holds (tokenize::Tokens::links); never from the order
of the headers. Left out, because they would teach the filter where mail
was collected rather than what it is: To and Cc, the account, dates as
such, the provider’s own marks.
A value can be missing (f32::NAN): the checks when neither Sioul nor
a provider you trust wrote any; the route where it cannot be read.
A missing value stands at the training’s mean, which is over the messages
that have it: it weighs nothing (table::Table::score). So mail from a
provider that writes no results, or reached through one whose lines are
written another way, is not told apart by that alone.
Version 2 (from 7 October 2026), measured on a real mailbox: version 1
taught the filter which account a message came to. Only one provider
wrote results that were trusted, so “no results” meant “that other
account”, whose share of spam the filter then learned; the count of
Received lines and ARC were each provider’s own too. Version 2 makes
missing checks missing (not “none”), drops the count of hops, ARC and the
provider’s verdict (the Porch applies that verdict itself, as its own
rule), and adds what the message carries wherever it is received: a DKIM
signature, one by the sender’s own domain, a bounce address at it.
Version 3 (the same day): the same facts, read by fastText’s classifier
as words (header_words: a fact that holds, its name; a number, its bin)
rather than weighed by an SVM beside the mean of the words’ vectors.
Version 4 (the same evening): the checks are Sioul’s own first, your
provider’s second (auth_results). Sioul checks each message whole as it
stores it (its stamp on top of it, sioul_sync::verify), and the
training corpus now keeps the same checks, made on the whole message
when it fetched it (sioul_learn::corpus::Checked, read with
stamp_results): the training and the sorting weigh the same verifier’s
word. Before, the corpus had no such checks, and both read the
provider’s alone, which a shared host may never write: there, every
check was missing, and only a signature’s unverified claim was left.
How a result reads (features): a pass, the fact holds (DKIM: a
signature verified against its key in the DNS; SPF: the server that
handed the message to your provider is one its domain names; DMARC: the
sender’s domain vouches for it); a fail or a softfail, the failure
holds; anything else is neither, unknown, never a failure: no signature
or no record (“none”), a key not found or revoked, a DNS error
(“permerror”, “temperror”), a signature expired or a body changed since
it was signed (“neutral”, as mail-auth says them). Checks made long after
the message came (LATE: the corpus’s, of old mail) keep their passes,
for what passes today passed then (the key that verifies the signature,
the server its domain still names). Their failures hold only when the
message carries no DKIM signature (late, signed): a signature that no
longer verifies (expired, its key gone or rotated, some under the same
selector) may have passed then, and DMARC with it, so beside one every
failure reads unknown; a message without one has nothing of its own to
expire, and a sender spoofed then is spoofed now (its domain’s records do
not turn the failure into a pass).
The same function runs on the desktop when training and on every device
when sorting: a change of a feature, of its order or of its scale raises
FEATURES, and a table made with others is refused.
Enums§
- Kind 🔒
- What an attached or inline part is, by its name’s extension or its type.
Constants§
- FEATURES
- The header features’ version, stamped in every table.
- FREE_
MAIL 🔒 - Mail providers whose domains millions share, for
reply_to_freemail. Part of what a table learned, so versioned withFEATURES: never the words you change (words::BrandWords::sharedholds the same list as shipped on 8 October 2026, and may grow; this one may not, unlessFEATURESrises). - LATE
- How long after a message came its checks read as they are; later, the failures of a signed message read unknown (see the module): 30 days, in seconds.
- N
- How many header features there are.
- NAMES
- Each feature’s name, in the order of
features’ values.
Functions§
- address_
domain 🔒 - The domain of a header’s first address: “Name a@b.example”, “a@b.example”.
- all_
header_ words - Every word
header_wordscan give, each with its feature’s place: what the classifier may have learned of the header facts. - auth_
results - The authentication results the features read of a message as it is
stored: Sioul’s own first (its stamp, under this installation’s id, which
ends in
.invalid), your provider’s second (the account’s other trusted ids), as the lanes, the forgeries and the senders read them. The training corpus keeps the message as the server holds it, without the stamp, and Sioul’s checks beside it (sioul_learn::corpus::Checked), which it reads withstamp_resultsfirst: the same verifier on both sides. - bare_ip 🔒
- A HELO name that is an address:
[192.0.2.1],203.0.113.5,[IPv6:2001:db8::1]. - dkim_
domain 🔒 - The signing domain of a DKIM-Signature (its
d=tag), lowercase. - features
- The header features of a message: its card (headers, parts, text),
its authentication results (
auth_results, or the corpus’s own checks), when its provider received it (IMAP’s INTERNALDATE, the Maildir file’s time), the domains of its links (tokenize::tokens). A value it cannot know isf32::NAN(see the module). - header_
words - The header facts as words, as the classifier reads them after the
message’s own (
table::Kind::Rows), each with its feature’s place inNAMES: a fact that holds,H:and its name in capitals; a number in a few bins (the Date an hour or more ahead; an hour or a day or more behind; no link, 1 to 3, 4 to 10, 11 or more; links elsewhere, half of them or more); a fact that does not hold, or is missing, none. No word of a message can be one of them: the tokenizer lowercases every word and splits on punctuation, and its own placeholders read_URL_; so no sender writes a header fact into a text. - index
- A feature’s place, by its name.
- late
- Checks made long after the message came (
LATE), of a message that carries a DKIM signature or not (signed): signed, their passes only, every other result unknown; unsigned, as they are (see the module). - names_
domain 🔒 - Whether a display name names an address or a domain other than the sender’s own: “service@bank.example x@elsewhere.example”, “bank.example Security”.
- signed
- Whether a message carries a DKIM signature (one naming its domain,
d=), whatever became of it since: whatdkim_signedsays, and what tells old mail’s failures apart (late). - stamp_
results - What a stamp of Sioul’s says (“Authentication-Results: sioul-….invalid;
dkim=pass …”), whatever installation’s id it bears: for a stamp kept
apart from its message, where no sender could have written it (the
training corpus’s
Checked); none when it says nothing readable.