Sioul's own spam filter¶

A spam filter trained on your own mail, on one computer, when you ask and again by itself once a week when that computer is plugged in and idle; folded into a small table that every device reads, the phone too; never applied to people you know, codes, projects or your own mail. What it does with each verdict is yours to choose (the matrix): flag it, move it into the Junk folder, or nothing; what it flags or moves waits on the Porch, among its catches, never notified; the training learns from them as they are until you say otherwise, and your word, Spam or Not spam, which every device then knows, always wins. Code: crates/sioul-core/src/spam/ (what every device runs: the tokenizer, the header features, the table, the verdict, the label logs), crates/sioul-learn/ (the training, on a computer only: the corpus, outside material, the labels, fastText's classifier, the calibration, the evaluation, the export; the centroid and its SVM on request), crates/sioul-core/src/porch.rs (where the verdict goes: the lane of the filter's catches), crates/sioul-app/src/spam.rs (the moves and the flags after each fetch, the settings, "Train now") and qml/SpamFilter.qml, crates/sioul-cli/src/spam.rs (sioul spam; its reports in spam/report.rs, the jobs run apart in spam/jobs.rs), crates/sioul-cli/src/mcp/spam.rs (the same for AI agents: mcp.md, "The spam filter's tools"). The Porch's side: porch.md, "Spam"; the sharing's: database.md, "Parts".

Why¶

Before it, spam was your provider's word alone, read from its X-Spam-* headers (porch.md, "Trust"). That word ranked above everything but blocked and forged mail: a provider's flag set aside a safe sender's message and a real code. Nothing learned from your own "Junk" and "Not junk".
Only your own mail teaches well. Filters trained on other people's mail did about a hundred times worse than one-person filters in the TREC spam tracks; the labels you give, near the threshold, matter more than the choice of model (research/spam-filter.md: the evidence, graded, what was built from it and what was not).
No "why" in the window (the owner's decision, 7 October): explaining why a message was flagged is not needed; the reason says only that your own filter judged it ("your own filter: probably spam"). The linear model still knows what weighed, word by word and header by header, and sioul spam why says it, for you, in your terminal.
The owner's design, from Virtual Secretary's experience (virtual-secretary.md): a fastText language model trained on the desktop on your mail; a linear SVM on header features and the message's mean word vector (since features version 3, fastText's own classifier, the header facts as words, measured better: below); the phone gets the folded table; never spam for people you know, codes, project routes; two thresholds, spam, unsure, ham, yours to set; manual tags only spam or ham, folders and tags taken as they are; one model for every account; training on demand, from the mail settings.

What it learns from¶

The corpus¶

Sioul's sync keeps your mail, but whole messages since forever rarely fit a disk, and a provider purges its Junk folder (Gmail after a month). Training keeps its own copy of what it needs, per message, in $XDG_DATA_HOME/sioul/spam/corpus/<account>/ (sioul_learn::corpus):

where it is (account, folder and its role, UIDVALIDITY, UID), when the server received it (INTERNALDATE), its flags and keywords ($Junk, $NotJunk), its size;
its header block (at most 32 KB), its MIME structure from BODYSTRUCTURE (the parts' types and names, attachments included, never their content);
the text the Porch reads of it: the start of its text part, and its HTML when it has no text part or only a stand-in ("view it in your browser"), which part being which as mail-parser says for the Porch, so that training reads a message as the Porch does;
Sioul's own checks of it (corpus::Checked): SPF, DKIM, DMARC, ARC and the reverse DNS, made by the same verifier the sync runs on every message it stores (sioul_sync::verify), on the whole message: DKIM checked against the body, SPF from the server that handed the message to your provider (the boundary hop), the results kept as the stamp Sioul writes on top of a stored message ("Authentication-Results: sioul-….invalid; dkim=pass …"), with when they were made. The message is fetched whole for them, in memory only, and dropped; a message larger than 2 MB is checked by its header block alone (SPF, the reverse DNS, DMARC when SPF passes it; DKIM and ARC, which need the body, left unsaid).

Every folder of every address, since forever, but Trash, Drafts, Sent and Gmail's All Mail. Read with Sioul's own IMAP session (EXAMINE, BODY.PEEK): nothing changes on the server. Incremental: where each folder stopped is kept after every batch, so a download stopped goes on there; a folder the server renumbered is read again. A record is never deleted when the server deletes its message: that is how junk outlives the provider's purge. Gzipped JSON lines, a few kilobytes a message. It never fills the disk: it stops before the free space falls under 1 GB, and says so.

The checks of messages kept before them, or while the DNS did not answer, are made afterwards: by every download, after its new mail, and alone with sioul spam fetch --verify (nothing new downloaded). Each message is fetched once by its UID, whole, in memory only; the results are appended beside the records (checks.jsonl.gz), group after group, so a run stopped goes on there. A message its server no longer has (its folder gone or renumbered, its UID gone) stays unknown for good. The DNS: one resolver per run, so each answer (the names that do not exist too) is asked once; twenty checks started a second at most, four at a time, each question given 4 seconds, each message 15; when no DNS server answers at all (not even that a name made up for the question does not exist), the checks stop, those of the messages since the last answer are dropped, and what is left waits, unknown, for the next run.

The labels¶

One label per message, spam or ham, from what is already there (sioul_learn::labels):

spam: in a Junk-role folder (folders::Role::Junk: \Junk, or a name such as Junk, Spam, Courrier indésirable), whoever put it there (your provider, your own filter's "Move to spam", a mail filter's "spam", you, another mail app); carrying $Junk; or flagged by your own filter as probably spam (below);
ham: in any other folder kept, or carrying $NotJunk;
left out: flagged by your own filter as maybe spam, until you say: a doubt is no label;
what you did beats everything: the label log, the newest entry winning; a message you said is not spam is never learned as spam, whatever its folder, its keywords or your filter say of it;
one message, one label: copies (the same Message-ID in several folders or accounts) become one; when they disagree, the log beats the keywords, which beat the Junk folder, which beats your filter's flag, which beats the other folders; copies that disagree at the same level are left out.

Your filter's verdicts count as they are until you correct them (the owner's decision, 7 October 2026). The spam folder is taken for an accurate set of what spam is, however mail got there, since you look at it now and then for mistakes; the filter is trained again regularly, so the growing mail and your corrections outweigh its mistakes; and you are never asked to confirm what it did, only to spot and report a mistake, or spam detection would become one more job for you. So nothing waits: what it moved, or flagged as probably spam, is spam at the next training. Only a doubt waits for your word: "maybe spam" is uncertain by definition, and learning from it would teach the filter its own guesses.

The label log is one file per device, $XDG_STATE_HOME/sioul/spam/labels/<device>.jsonl (the device's name in the sharing, a UUID made once; spam::labels::own_log), one line per act ("Junk", "Not junk", "Spam" and "Not spam" among the filter's catches or in the Reader, blocking a sender from one of their messages): when, the account, the folder, UIDVALIDITY, UID, Message-ID, spam or ham, which act; never a word of the message (porch.md, "Trust"). Each device writes its own and reads every device's: the logs travel sealed with the part "Spam filter" (database.md, "Parts"), so that a message you said is not spam on one device is never flagged again on another (spam::labels::said_ham, by account and Message-ID, the newest word winning), and the training on your computer learns from what you said on your phone. The log of one device from before the logs were shared, labels.jsonl, is still read, and becomes that device's own file the first time it writes. The keywords $Junk and $NotJunk, set by Sioul or another mail client, are kept in each message's file name, so a correction made in Thunderbird reaches the trainer through the server too.

What your own filter did goes into the device's logs, apart from the label log, which is yours alone: what it moved into a Junk folder (the matrix's "Move to spam", below) into $XDG_STATE_HOME/sioul/spam/moved/<device>.jsonl, what it flagged where it is ("Flag only") into flagged/<device>.jsonl (spam::labels::Moved, Flagged: when, the account, the folder and place it was judged in, its Message-ID, how the filter judged it; shared like the label logs). A message it moved is spam as all Junk mail is (labels::Summary::filter_moved counts them). A message it flagged counts by its class, by account and Message-ID wherever its copies are now, since moving it elsewhere is no correction: probably spam, spam (labels::Evidence::Filter, Summary::by_filter); maybe spam, its copies in your folders have no say (Summary::unsure), while a copy in a Junk folder still speaks. Once you say Spam or Not spam, your word decides, as any. A flag is written as the mail arrives, after each fetch, by the device that judged it (sioul-app's spam::after_fetch, sioul_sync::mailbox::flagged), with the table then in use.

A Junk folder is mostly your provider's own filing: a model trained on it alone learns to copy the provider. Your own acts, logged and kept as keywords, are the labels that teach it something the provider did not know.

Outside material¶

Mail labelled elsewhere (an older filter's dump, another mail client's archive, a public corpus) can be imported once, to give the training more to learn from and a baseline to measure it against (sioul_learn::external; sioul spam import <file>, sioul spam import --remove <source>). The format: JSON lines, one message each, a plain file or gzipped:

{"label":"spam","date":"2023-06-20T16:23:00+02:00","subject":"You won","text":"Claim your prize…","source":"old-filter"}
{"label":"ham","date":1687271000,"subject":"Lunch","text":"Tomorrow?","from":"Jane <jane@example.org>"}

label: "spam" or "ham" (required);
date: when it arrived, RFC 3339 or Unix seconds (required): the split is by time;
subject, text: its subject and its text part (required; one of them, or html, not empty);
html: its HTML, read as text as the Porch reads it when there is no text, or only a stand-in;
from: its sender; headers: its raw header block, of which only the Message-ID is read now (a message also in your own mail is learned once, from yours); both kept for a later version;
source: a name for where it comes from; else the import's --source, else the file's name.

A line that does not fit is left out and counted (not JSON, no label, no readable date, a missing field, nothing to read, the same message again). The rest is kept apart from your own corpus, in $XDG_DATA_HOME/sioul/spam/external/<source>.jsonl.gz (yours alone: 0600 in a 0700 folder), each message cut to what the Porch reads (6 000 characters of text, 64 KB of HTML, 32 KB of headers), with <source>.toml beside it (counts and dates only: its oldest and newest dated messages, a date of 0 or before 1970 being an export's "no date"). Never shared, never shown to an AI agent. A source imported again replaces the one before; --remove takes it away; sioul spam status and the settings list what each source holds.

In training, outside material's words join the language model's corpus, and its messages the classifier's learning set at their dates, with the header features they lack set to the training mean: the mean and the deviation of each header feature are your own mail's alone, and outside material stands at 0 once standardized, so it pulls no header weight; only the words learn from it. Each source is split in time as your own mail is (below, "Evaluation").

Reading a message: the tokenizer¶

spam::tokenize, version 2 (TOKENIZER, stamped in the table: a table of another version is refused). Its words (a provider's marks, month and key names, French elisions, number words, stop words) are the spam lists of the language packs in use and your changes (words.md, "The spam filter's words"): a training reads with them and its table keeps them (Meta::lexicon), so every device reads each message with the words its table was trained with; a table made before 8 October 2026 keeps none and reads with the built-in French and English words, which made it. It ports Virtual Secretary's pipeline, rewritten for Rust's regex (no look-around), and is the same code on the computer that trains and the phone that judges:

Text: the subject and the Porch's excerpt (the text part, the HTML read as text after a stand-in), quotes and signatures kept. A mark your provider put at the start of the subject is taken off first (without_provider_tags: a word of spam, junk, pourriel or indésirable between stars, brackets, braces or parentheses, "Potentiel-SPAM", "[SPAM?]", "{Spam}", or "SPAM:"): it is the provider's verdict, not the sender's words, and a model that read it would learn to copy the provider. Version 2's reason, measured on a real mailbox: one provider's tag opened the subject of nearly every message it set aside, and of some ham in an older filter's dump.
Normalization: invisible characters out, NFKC, lookalike letters folded (Cyrillic, Greek, Armenian, only in words holding a Latin letter or made only of lookalikes), lowercase, Virtual Secretary's table (accents, ligatures, spaces, dashes, quotes, fractions), the remaining non-ASCII out, spaces collapsed.
Cleanup: runs of ., -, ? collapsed; a character repeated ten times and more, and long base64 blocks, out.
Placeholders: Virtual Secretary's 36 (an address, a link, a server's address, a date, a time, a price, a file of each kind, numbers with their units, a long number…) and two more, _PHONE_ and _IBAN_ (checked by its mod-97). A link also gives its domain (eTLD+1) to the header features.
Splitting: words on spaces and punctuation; French elisions split and dropped (l', d', qu', n', s', j', c', m', t', lorsqu', puisqu', jusqu', quelqu').
Each word: number words become _NUMBER_, five digits and more _HASH_, Virtual Secretary's stop words go, its French–English suffix stemmer cuts the rest ("lottery" reads "loteri").

Where it departs from Virtual Secretary, on purpose, is listed in tokenize.rs (every accent stripped, not only French ones; €, £ and ° kept as words; look-around rewritten; alternatives longest first). Tests: Virtual Secretary's test-patterns.py inputs as assertions, elisions, invisible characters, lookalikes.

The header features¶

spam::features, version 4 (FEATURES), 35 named numbers, each explained by its name, which the classifier reads as words of their own (features::header_words, below); read from the header block, the message's parts and links, and the authentication results, Sioul's own first, your provider's second (features::auth_results: on a stored message, Sioul's stamp, then the results of the provider's id the account trusts; in the corpus, the record's own checks, read as the stamp is: features::stamp_results), as the lanes, forgeries and senders read them, never from the headers' order:

authentication as Sioul checked it, else your provider: SPF, DKIM and DMARC each pass, fail or neither; aligned (the sender verified: DMARC passed for the From's domain, or a signature by that domain did). Missing when neither wrote results (Sioul's checks not made yet, and no provider id trusted);
authentication the message carries itself, the same wherever it is received: a DKIM signature; one by the sender's own domain (its d= the From's registrable domain); its bounce address (Return-Path) at that domain. A claim, which anyone can write: the checks above say whether it holds;
domains: Reply-To elsewhere than From, Reply-To at a free mailbox, the Message-ID made elsewhere, a display name showing another domain, links pointing elsewhere;
time: dated ahead of its arrival, or well before (against INTERNALDATE, clipped), no Date, no Message-ID;
the way it came: a server that gave no name, where it entered your provider; missing where that line cannot be read;
its shape: HTML only, images with almost no text, an archive, a program, an office document, a PDF or an image attached, the text's length in four steps, the number of links;
bulk: List-Unsubscribe, List-Id, Precedence: bulk or list. Bulk marks gray mail, not spam: the model weighs it.

As words (version 3): a fact that holds is the word H: and its name in capitals (H:DKIM_SIGNED_BY_SENDER), a number its bin (the Date an hour or more ahead; an hour, or a day or more, behind; no link, 1 to 3, 4 to 10, 11 or more; links elsewhere, half of them or more), read after the message's own words. No word of a text can be one (the tokenizer lowercases every word and splits on punctuation), so a sender cannot write a header fact into a message.

A missing value (f32::NAN) weighs nothing: the classifier reads no word for it; the centroid stands it at the training's mean, which is over the messages that have it (table::Table::score, and the training's standardization). Mail from a provider that writes no results is not told apart by that alone.

How each check reads (version 4): a pass is a fact (H:DKIM_PASS: a signature verified against its key in the DNS; H:SPF_PASS: the server that handed the message to your provider is one its domain names; H:DMARC_PASS, H:ALIGNED); a fail or a softfail is one too (H:DKIM_FAIL: a signature that does not verify against the key found); anything else is unknown, neither pass nor fail, no word: no signature or no record ("none"), a key not found or revoked, a DNS error ("permerror", "temperror"), a signature expired or a body changed since it was signed ("neutral", as mail-auth says them), a message larger than 2 MB whose signatures were left unchecked, a message its server no longer had. Old mail: checks made more than 30 days after the message came (features::LATE; the corpus's, when it fetches old mail for the first time) keep their passes: what passes today passed then (the key that verifies the signature, the server its domain still names). Their failures hold only when the message carries no DKIM signature (features::late, features::signed). A signature that no longer verifies (expired, its key gone or rotated, some under the same selector) may have passed then, and DMARC with it: beside one, every failure of old mail reads unknown, never failed. A message without one has nothing of its own to expire, and a sender spoofed then is spoofed now: its domain's records do not turn the failure into a pass. Measured on a real mailbox, most failures of old ham came beside a signature that no longer verified, and nearly every failure of old spam on an unsigned message. Reading every old failure as unknown instead would leave the model of the older mail with no failure to learn from: only the newest mail would keep its failures, and the test holds the newest. Sioul's stamp on a stored message, made as it arrived, reads as it is; so does the corpus's check of a message fetched within 30 days of its arrival.

Left out, because they would leak the label or how the mail was collected: To and Cc, the account itself, dates as such; your provider's own verdict (X-Spam*, X-Rspamd*: the Porch applies it as its own rule, above; it never enters as words either), the count of Received hops and ARC (each provider's own). Why version 2 (7 October 2026, measured on a real mailbox): only some of its addresses had a provider writing results the account trusts, so version 1's "no results" meant "the other addresses", and the hops and ARC told the providers apart too: the filter learned each address's share of spam instead of spam. sioul spam features prints each feature's mean by account and label: a feature that differs more between accounts than between ham and spam is such a marker. Why version 4 (7 October 2026): versions 2 and 3 read your provider's results alone, on both sides, since the corpus had no check of Sioul's; a provider that writes none (a shared host may not) left every check missing on its addresses, and only the signature a message claims (unverified, forgeable) told a bank's real mail apart there. Now the corpus keeps Sioul's own checks, and both sides read them first. Measured on a real mailbox, the strongest header facts learned became the verified passes, where they had been the signature a message claims.

The model, and how it folds¶

Trained on a computer (sioul_learn::train), with fastText (the fasttext crate 0.8.0, pure Rust, pinned). Two models (train::Model); the filter's is the first.

fastText's classifier (sioul_learn::supervised, --model supervised, the default since features version 3):

Each message one line: its words, then its header facts as words (above), labelled ham or spam; outside material, which has no headers, its words alone. fastText's supervised model, two labels, softmax: 100 dimensions, character n-grams of 3 to 6, 200 000 buckets, words seen 5 times and more, 25 epochs, learning rate 0.1, each word alone (no word pairs). Its labels are written in capitals, which no word of a message has, so that no text reads as a label. Kept in $XDG_DATA_HOME/sioul/spam/language.bin, on this computer only.
The calibration: Platt's A and B, fitted on scores of mail the model never learned from: in each account's and each outside source's mail of each label, the newest fifth of the messages learned from, scored by a model of the rest (split as the test is), Platt fitted on your own mail's part, the mail the filter will judge.

Why it, rather than the centroid (measured on a real mailbox, in time order, the same split): it ranked spam above ham better, on that mailbox's newest mail and on an older filter's held-out material; and two trainings on the same mail gave the same numbers, where the centroid's moved with its language model, which fastText learns on several threads without locks, and with the SVM's cost chosen on it. Word pairs (wordNgrams 2) measured no better: the table reads none, and the phone hashes none. The time-ordered calibration of the centroid's way (the newest fifth of everything learned from) made the classifier far too sure of spam there: that newest fifth was the mailbox's own recent mail alone, scored by a model of older outside material that has no header facts; calibrating by group, as the test is split, fixed it without changing the ranking.

The centroid (--model centroid, the filter's before features version 3, on request):

The language model: fastText skip-gram on the tokenized corpus, ham and spam of every account: 100 dimensions, character n-grams of 3 to 6, 200 000 buckets, words seen 5 times and more, 5 epochs.
A message's vector: the plain mean of its words' vectors (fastText's get_word_vector: a word's own row and its n-grams'; an unknown word, its n-grams' alone), never normalized, so that the model folds. The header features follow it; each dimension is standardized with the training messages' mean and deviation.
The SVM: linear (L2-regularized, squared hinge, dual coordinate descent as liblinear's: Hsieh et al. 2008), our own code; ham weighs 5 (calling ham spam costs five times more); its cost C chosen among 0.01, 0.1, 1 and 10 by the AUC on the newest fifth of the learning messages.
The calibration: Platt's A and B, fitted on scores each message got from an SVM trained on older mail only (forward folds).

The fold, the classifier's (table::Kind::Rows). With two labels, softmax's probability of spam is the sigmoid of f = (w_spam − w_ham) · h, where h is the mean of every input row the line reads: each word's own row when fastText knows it, the rows of its character n-grams' buckets, the end of the line's. So f is the mean of the rows' scalars s_r = (w_spam − w_ham) · row_r. The table keeps, for each word fastText knows, the sum of its rows' scalars; for each bucket, its scalar; for the end of the line, its own (the table's bias). A message's decision value is f = (bias + Σ its words' sums) / (1 + Σ their rows), a known word counting one row and its n-grams', an unknown word its n-grams' alone, each word as often as it comes, the header facts' words among them; its probability p = 1 / (1 + exp(A·f + B)).

The fold, the centroid's (table::Kind::Centroid). Everything is linear in the word vectors. With u = w / σ the SVM's embedding weights over the deviations, each vocabulary word scores s_w = u · v_w and each n-gram bucket s_b = u · row_b; the mean of a message's word scores is then exactly the SVM's term for the mean of its vectors. A message's decision value is f = bias + (mean of its words' scores, 0 without words) − text_mean + Σ wₕ·(xₕ − μₕ), a word missing from the table scoring the mean of its n-grams' buckets, as fastText builds the vector of a word it never saw.

The n-gram hashing is fastText's own (32-bit FNV-1a over sign-extended bytes of <word>, n-grams of whole characters), checked against fasttext's get_subwords on the computer.

The table (spam::table, $XDG_DATA_HOME/sioul/spam/table.bin, format 2): each vocabulary word's score under a 64-bit hash of the word (FNV-1a), never the word, and only words seen five times or more; each bucket's score; the centroid's header weights and means (none in the classifier's: that is how its kind is told); the bias, text_mean, Platt's A and B; the tokenizer's and features' versions, the n-grams' sizes; what the settings say of the training (when, on which device, from how many messages, its aggregate numbers) and the tokenizer's words it was trained with (lexicon). About a megabyte for 20 000 words. Checksummed, written whole or not at all. Before it is exported, the table must give the model's own score within 1e-4 on the test messages (the classifier's: fastText's own prediction), read back from its own bytes as every device reads it: else nothing is exported. A Sioul that reads format 1 refuses format 2, rather than reading the classifier's sums as scores.

What weighed: the table's score keeps each word's share of f and each header feature's, sorted (table::Score). The window never says it (the owner's decision: no "why"); sioul spam why <file> prints it, for you, in your terminal: the words as the filter reads them (stems, "loteri"; placeholders, _PRICE_), the header features by name.

Where it acts: never on people you know¶

The verdict is the Porch's (porch::triage). In order:

Blocked, forged, a borrowed name: set aside as before, whatever any filter says.
Protected, from every spam verdict, your provider's and Sioul's own: mail from anyone but a stranger (safe, neutral or restricted: in your address books, on a list, let in from the screener); a code or a password you asked for; mail a project's routes take (a thread pulled into a project leaves the review queue for the project); your own verified mail; a message you said is not spam, here, on another device or in another mail client (spam::labels::said_ham: $NotJunk, or every device's label log). Mail nothing authenticates (SPF and DKIM failed, no DMARC or ARC vouching) has a stranger for its sender whatever the address it shows, and no protection.
Your provider's verdict, for a stranger's mail only: set aside, as before.
Hostile mail to a shielded address: its own lane, as before, whatever the filter says.
Sioul's own, for a stranger's mail only, when a table is there: what the matrix says for its class (below).

Two thresholds, and what is done with each verdict¶

Two thresholds make three classes: probably spam, from threshold_spam (95 % unless you change it); maybe spam, from threshold_unsure (50 %) up to the spam threshold; probably not spam, below. What is done with each class is yours to choose, the matrix {probably spam, maybe spam, probably not spam} × {move to spam, flag only, do nothing} (spam::Actions; [spam] action_spam, action_unsure, action_ham: "move", "flag", "nothing"):

Move to spam: as it arrives, the message goes into its account's Junk folder on the server, as the Junk button moves it but without $Junk and without a line in the label log, which is yours alone. It happens after each fetch, on every device that fetches (the window's watchers, a phone's background step: sioul-app's spam::after_fetch), judged as the Porch judges, protections first; only mail as it arrives: what was in the inbox before stays there, flagged. Each move goes into the device's log of moves (above), and the message waits on the Porch, among the filter's catches, while it stays in the Junk folder and you said nothing of it. Not spam brings it back to the inbox ($NotJunk, a ham label); Spam keeps it there ($Junk, a spam label).
Flag only: it stays where it is on the server, marked ("probably spam", "maybe spam", "probably not spam"), among the filter's catches; the flag goes into the device's log of flags (above). Not spam leaves it where it is, $NotJunk on the server, a ham label, and it goes back to its lane; Spam moves it into the Junk folder, $Junk, a spam label.
Do nothing: nothing shows; it goes to its lane as if unjudged. All three "do nothing": no message is judged at all.

Until you choose: spam and doubts flagged, the rest nothing. An older Sioul's one choice, [spam] mode, is read until a row is first changed (then the three are written, the mode taken out): "off" → all "do nothing"; "say" → spam and doubts "flag only"; "act" → spam "move to spam", doubts "flag only". One filter for every account. In Mail ▸ ⚙, "Your own spam filter": the matrix (rows of round buttons; on a narrow screen each row's name above its buttons), the two thresholds as percentages (each slider stops a point short of the other: unsure always below spam), then the training's block. The matrix and the thresholds travel with your settings.

The lane of the filter's catches (porch::Lane::Review, the review queue in the code; "Caught by your own spam filter"): on the Porch, a quiet folded lane, no count beside its title, never in the Porch's count of what came nor on the phone's home card, opened when you want. Nothing in it asks to be confirmed: Sioul learns from it as it is, and Not spam is there for a mistake. Each message has its word, quiet ("probably spam"), never why nor how sure; Spam and Not spam on each, and Spam for all and Not spam for all, each with ten seconds to undo. Flagged mail stays in it for two weeks after you close the Porch on it ("Done" never closes it; the Porch's two weeks), moved mail while it is in the Junk folder and unreviewed (found by Message-ID in the account's Junk folders, among the files fetched after the move; a message without a Message-ID cannot be found again there). Said Spam or Not spam on any device, or marked junk or not junk in another mail client, it leaves the lane everywhere.

Notifications: never for what the filter flagged or moved, on any device: new mail's notifications (mailnote::never), the phone's background step, the home card (porch.md, "Notifications"). What it does nothing with is told as its lane says. Mail set aside is never told either. One-click unsubscribing is refused for mail called probably spam (answering it says your address is read), offered for "maybe spam".

Training¶

On one computer: by hand, in Mail ▸ ⚙ ("Train now") or with sioul spam train, and again by itself once a week (below, "Training by itself"); never on a phone. Two computers training tables of their own is not supported: the later table wins on every device; the other, kept beside it by the sharing as a conflict copy (table (conflict …).bin), is read by nothing, a safety only. Train on one. The steps (train::run):

the corpus brought up to date from every address;
each message's label, words and header features, read as the Porch reads mail;
for each account, its ham and its spam apart, the oldest 80 % to learn from, the newest 20 % to test on, by INTERNALDATE (so every account's ham and spam are on both sides: a provider that keeps its Junk folder a month would otherwise have all its spam in the test, and none to learn from); each outside source alike, its ham and its spam, by its own dates (the oldest 80 % learned from, at their dates; the newest 20 % held out);
the calibration (the classifier's: a model of four fifths of what is learned from scoring the fifth left, above);
the model learns from the oldest 80 % (the centroid's way: fastText learns your mail's words, each message gets its vector, the SVM's cost is chosen, then Platt);
its table, folded and checked;
the numbers, on your newest 20 % (and again without the ham an inbox holds since less than 30 days: its label may not be settled), and on each outside source's held-out part, apart;
the table compared with the one in place;
kept: the same model learned again from every message, the newest included, outside material whole (spam changes, and the filter that runs must have seen this month's), calibrated the same way, folded, checked, written. The numbers said are those of steps 3 to 7.

The table changes only when no worse: the new one replaces the one in place only if it takes no more ham for spam, at your spam threshold, than the one in place, on those of the newest 20 % the one in place never learned from (those that came after its own training: on older mail it would be judged on what it was taught). Each table reads those messages with its own words: the tokenizer's lists a table was trained with travel inside it (words.md, "The spam filter's words"), so when your words changed since the table in place was trained, its messages are read again from the corpus with its own words, and a message that can no longer be read is left out for both; a change of words never makes the one in place look worse than it is. It replaces it too when there is none, when the one in place is another version's, and when nothing came since it was trained ("nothing tells them apart"). The one replaced, when this Sioul can use it, stays beside it as table.prev.bin. What happened, and why, goes to $XDG_STATE_HOME/sioul/spam/trained.toml (numbers only), which the settings say: the reason, the comparison, what the table in use learned from.

In the window (crates/sioul-app/src/spam.rs): on a thread of its own, below the window's priority (Linux: setpriority, which fastText's threads take), so the computer stays usable while every core learns. Its progress is said as it goes, at most a few times a second: "Messages fetched: 1,200 of about 15,000 (home · INBOX)", each folder's count summed over the addresses, the ones not reached yet counted from what the last download left on their server; then each step, with its count when it has one ("Learning your mail's words" has none: fastText says nothing until done). Stop stops it at its next step; what was downloaded is kept, and the next time goes on there. The block also says the last training (what it did with the table and why, the two tables compared on the newest messages the one in place never saw; what the table in use learned from, again from every message; the numbers of the same model learned from the oldest 80 %, at both thresholds with their intervals; the thresholds of then when you changed yours since; each outside source's counts and its baseline), the corpus (messages kept, from how many addresses, its size, the last download, the room left on the disk, the addresses it could not read), and the outside material imported (each source's counts and dates). The window's Porch reads the new table at once.

From the command line (and for AI agents, the same through the MCP server: below, "Controls from outside"): sioul spam fetch [--account] [--verify] (the corpus only; --verify: only Sioul's checks of the messages kept without them, nothing new downloaded), train [--no-fetch] (a trial with --no-replace), eval [--errors N] (the table in place on the newest fifth, and each outside source's), status (the corpus, the outside material, the providers' verdicts as Sioul reads them, Sioul's own checks of the corpus's messages per account as training reads them: pass, fail, unknown, the sender verified, messages gone, old signed mail's failures read unknown; the table in use, the last training, the jobs), features [--domain D] (each header feature's mean by account and label: those that mark an account), dry-run, review, label, job, import <file> [--source] and import --remove <source> (outside material, above), why <file> (below). Each but import and why says it in lines, or as JSON with --json. Piped (to a file, a program, an agent), the progress goes to standard error one line per step, per folder downloaded and per account checked, at its end, never a carriage return; the download says first about how many messages are to come.

Training by itself¶

The filter trains again by itself, once a week at most, so that the growing mail and your corrections keep outweighing its mistakes, as long as it costs you nothing: never on a laptop's battery (the owner's words, 7 October 2026: "That depends on how much resources are needed to retrain. On battery, for example, you don't want the retraining to run freely on a laptop."). On a laptop with a real mailbox, a training takes a few minutes, half the cores and a few hundred megabytes. The rule (sioul_learn::auto::decide, a pure function of what is known; an unknown fact counts as not holding):

only the computer that made the table in use (the device its table records), so that two computers never compete; every other computer shows the switch greyed, with one sentence saying which computer trains; with no table yet, a training by hand comes first;
once a week at most after the last training, by hand or by itself (trained.toml); a quarter of an hour after a run it stopped, a day after one that failed or whose process vanished (auto.toml, auto::Run);
on mains power (UPower's OnBattery on Linux, GetSystemPowerStatus on Windows, IOKit's power sources on a Mac), not saving power (power-profiles-daemon's ActiveProfile, under org.freedesktop.UPower.PowerProfiles and net.hadess.PowerProfiles; Windows' battery saver; a Mac's Low Power Mode), the computer idle for 15 minutes or its session locked (the desktop's org.freedesktop.ScreenSaver, GNOME's idle monitor and screen saver, logind's LockedHint and IdleHint; GetLastInputInfo; CGEventSourceSecondsSinceLastEventType), and no focus session running (sioul_sync::power, timelog::running);
asked at the window's minute (sioul-app's spam::by_itself, off the window's thread) and at sioul watch's own (spam::by_itself::every_minute): the files first, the system only when nothing else holds a training back.

How it runs: as the detached job of sioul spam (spam/jobs.rs: sioul spam train --by-itself, which sioul spam job lists), at the lowest priority (nice 19 and the idle I/O class on Linux, background mode on Windows, a throttled disk on a Mac), fastText on half the cores, with the usual rule for replacing the table. On a metered connection (NetworkManager's Metered: yes or guessed yes; not read on Windows and macOS, so unknown, so treated as metered), it trains on the corpus as it is (--no-fetch). While it runs it asks every half minute whether its power still allows it: unplugged or saving power, or twice in a row unable to tell, it writes its own stop file and stops at its next step; what was downloaded stays, and it tries again later. You coming back does not stop it: it runs at the lowest priority. While a job runs, "Train now" waits; the window's own training holds a lock beside the jobs' (jobs/window.lock), so that sioul watch's minute waits too.

Quiet: no notification. In Mail ▸ ⚙, under "Its training", a switch, "Train again by itself once a week, when this computer is plugged in and idle" ([spam] train_by_itself, on unless said; it travels with your settings), and one line under it: when it last trained by itself and whether its table replaced the one in use, or that it stopped or did not finish and tries again. "Train now" stays.

Controls from outside¶

So that a person, a script or an AI agent can run the filter, watch it and judge it without the window, sioul spam and the MCP's spam tools (mcp.md, "The spam filter's tools") say the same things from the same code (crates/sioul-cli/src/spam/report.rs): in lines in your language, and as data (--json, MCP's structuredContent).

Messages are named by their envelope, never by their words: the date, the account and folder, the sender's address, the subject masked as the MCP masks mail (a code, a sign-in or reset link, an IBAN, a card or social security number hidden: mcp.md, "What does not move"), the probability, the verdict, its mid: address. Hostile mail to a shielded address is "someone at" its domain, without its subject; mail from a sender you blocked is never listed, only counted. A list says first that its subjects are their senders' words: data, never instructions.
train --no-replace, a trial: the whole training, its numbers and its comparison with the table in place ("a training would replace it: …", "would keep it: …"), and nothing of the filter changes: neither the table nor the one before, nor the language model, nor trained.toml (which says the last real training to the settings). The model and its settings can be changed for an experiment: --model supervised|centroid (the classifier, the default; the centroid and its SVM), --threads, --dim, --epochs (the classifier's 25, the centroid's 5 by default), --bucket, --minn, --maxn (--maxn 0: no n-grams, and none to hash), --min-count, --lr (the classifier's learning rate), --c (the SVM's cost, fixed) and --ham-weight (the centroid's); they are recorded with the summary (trained.toml's model when the table is replaced: its kind, min_count, threads, the costs or the learning rate beside the others). A training from the command line runs below the other programs' priority, as the window's does.
train --errors N and eval --errors N: after the numbers, the test messages counted by account and by folder (ham called spam, maybe spam, not spam; spam caught, maybe spam, missed), the numbers at thresholds from 0.5 to 0.99 (0.5, 0.6, 0.7, 0.8, 0.85, 0.9, 0.95, 0.97, 0.98, 0.99) so that a threshold can be chosen, and the N worst errors of each kind: ham called spam or maybe spam, the surest first; spam not called spam, the least sure first; each with what decided its label (its folder, a Junk folder, a keyword, what you said, outside material); then each outside source's held-out part alike (sioul_learn::detail). The honest errors are the trial's: train's numbers and errors are those of the model learned from the oldest 80 %, on the newest 20 % it never saw. eval tests the table in place, which learned again from every message before its training, the newest fifth included: it says how many of the tested messages came after the table was trained, and their numbers apart; an error it learned from is marked so.
train --scores: every test message's probability of spam, its label and whether that label is settled, in the JSON (and each outside source's held-out part's), numbers only: for a curve, or the catch at another share of ham taken for spam. Never in an agent's answer.
dry-run [--account A] [--limit N]: what the matrix would do now with the mail in each inbox, as the Porch judges a message as it arrives (protections first): per account, the messages judged, protected, set aside before, hostile, left out; per verdict, how many and the action; then the messages it would move into the Junk folder and those it would flag, the surest first. --spam, --unsure, --ham (move, flag, nothing) and --threshold-spam, --threshold-unsure try another matrix without changing yours: the step before choosing "Move to spam". Nothing is moved, nothing written (tested), and it says so.
review: the filter's catches as the Porch lists them, flagged or moved, the surest first.
label <file|mid:…> spam|ham: one line in this device's label log, as the window's Spam and Not spam write it (source junk or not-spam), which every device and the next training read. It only labels: the message stays where it is, nothing is said to the server. --move does the window's act itself, on the server: the keyword, and the move (Spam: into the Junk folder unless it is there; Not spam: out of it, back to the inbox; elsewhere it stays, marked). A message no longer stored on this computer (purged junk, mail older than the sync keeps), named by its Message-ID as eval lists it, is labelled by its place in the corpus, and is never moved.
Jobs, for agents: the MCP's spam_fetch and spam_train start sioul spam fetch|train --job <id> apart, in a process group of its own, and answer at once with its id; sioul spam job [<id>] [--stop] and the MCP's spam_job read where it stands (starting, running and on which step, done, failed, stopped, died) and its result, from $XDG_STATE_HOME/sioul/spam/jobs/<id>.json (0600, its folder 0700). Its process holds <id>.lock while it runs: a job whose file says it runs but whose lock nobody holds ended without a word (killed, the computer stopped), said "died" with the last lines it printed (<id>.log). --stop makes <id>.stop, which it sees within half a second, fastText included, and stops at its next step, what was downloaded kept. One job of each kind at a time; a job's files go a week after it ended (a training's result names its worst errors by sender and masked subject: kept no longer).

Evaluation¶

Aggregates only (sioul_learn::eval), by time, never by chance: a filter is used on tomorrow's mail, and spam drifts. Two sets, said apart: the newest 20 % of your own mail, each account's ham and spam apart (train::newest_fifths), which decides whether the table is replaced; and for each outside source, the newest 20 % of its ham and of its spam by its own dates, held out ("the baseline": the same numbers there, said apart; train::Outside), scored with no header facts, as it learned them.

the messages per label and per account, learned from and tested on;
ham taken for spam and spam caught, at each of your two thresholds, each with its 95 % exact interval (Clopper–Pearson);
the strict threshold: the lowest that takes at most one ham in two hundred for spam, and the spam it catches there (eval::STRICT);
the share called "maybe spam";
the area under the ROC curve;
the settled numbers: the same, without the ham an inbox holds since less than 30 days (detail::UNSETTLED_DAYS): spam nobody has looked at yet stays in an inbox as ham, and would be counted as ham taken for spam. Few ham may be left (the newest ham is mostly the inbox's): said with its count.

Per message, only what says which message it is: train --errors and eval --errors list the worst errors by their envelope (above, "Controls from outside"), never by their words. What weighed, word by word, is sioul spam why <file>'s: the probability, the score, the words and header facts that weighed. It is for you, in your terminal: the MCP server offers none of it (ai.md).

What the numbers cannot tell: labels from Junk folders are mostly the provider's filing; an inbox holds spam nobody has looked at, labelled ham (the settled numbers leave the youngest out); Sioul's checks of old mail are made with today's DNS (above): a spam campaign's domains, gone since, pass nothing, where at its arrival they may have passed, so old spam shows fewer passes than spam does as it arrives, and a pass weighs a little more toward ham than it should (the newest mail, checked within 30 days, is spared); and an unsigned sender that moved its servers since reads failed where it passed then; a test on your newest mail says how the filter did then, not how mail will change.

Privacy¶

The corpus and the language model stay on the computer that made them: never in a shared part (no store of the sharing covers them: tested), never in the repository nor its tests (tests train on invented mail at reserved domains). The language model holds the vocabulary of your mail in plain text, and no tool reads it; the corpus, the start of every message, of which an AI agent you connected sees only the envelope of a test's worst errors (sender's address, subject masked: as search_mail shows mail), through the spam filter's tools, which [mcp] spam = false keeps from agents (mcp.md).
The table travels, sealed: hashes and numbers, no word in plain text; still made from your mail (a guessed word can be tested against a hash), it is private as notes are: sealed in the sharing, never published.
The label logs and the logs of what the filter moved and flagged: which message, where and when, never a word of it; each device's own, shared sealed with the table.
Outside material: kept on the computer it was imported on, apart from the corpus, never shared; what it holds is said in numbers only, but for the envelope of its held-out part's worst errors (sender, subject masked), to you and, through the spam tools, to an agent you connected; sioul spam import --remove takes it away.
Nothing goes to any server: training reads your mail over IMAP, read-only, and writes local files. Sioul's checks fetch each message whole once, in memory only, never written; they ask the DNS what any receiving server asks (the sender's keys, SPF and DMARC records, the sending server's name), and the DNS of your mail server's domain, once per account, a name made up to learn whether the DNS answers at all.

The phone¶

It never trains: sioul-learn and fastText are dependencies of the window and the command line on computers only ([target.'cfg(not(target_os = "android"))'.dependencies]); cargo tree --target aarch64-linux-android -p sioul-app lists neither.
Its table comes through the sharing, the part "Spam filter" (on by default on every device): sealed apart, in every exchange, the background step's too (database.md, "Parts"). The one it replaces, when this Sioul can use it, is kept as table.prev.bin; a table this Sioul refuses (another version's tokenizer or features, damaged) leaves the one before in use (Table::cached), and the settings say so.
Judging: the window's Porch and the background process (:steps, new mail's notifications) read the same file through Config::mail_sources → spam::Filter::of → Table::cached: read once per process (a megabyte or two, a little more in memory), read again only when its size or time changes; each message's verdict kept per message file and per table (20 000 at most). No table, or the filter off: the Porch as before.
Its settings: the matrix and the thresholds travel with your settings; Mail ▸ ⚙ says where its table comes from ("In use: trained by desk on Tuesday 6 October at 10:00, from…") and what it measured then.
What it moves and flags: as a computer does, after each fetch of its background step (spam::after_fetch); its lane of the filter's catches, its Spam and Not spam, its logs of moves and flags and its label log go to the other devices sealed.

The label logs and the logs of moves and flags travel in the same part, as folders of lines (share::SPAM_LABELS, state/spam/labels/; share::SPAM_MOVED, state/spam/moved/; share::SPAM_FLAGGED, state/spam/flagged/; Shape::Lines): each line an entry, each file written by its own device alone, read in every exchange, a few kilobytes.

The table: one store, files/spam/table.bin (share::SPAM_TABLE), in its own part, sealed apart as notes are (Shape::Files): the record says the content's hash, size and time; the content goes once, as a sealed file in blobs/. Not whole in a record (Shape::Whole): its megabyte or two (nearly twice that once sealed in a line) would go again in every round its computer starts, and with Sioul keeping the folder itself the current round goes up whole at each change, every minute while a computer writes. Unlike notes and papers, it is read in every exchange, quick ones and the phone's background step included: one file of Sioul's own, quick to read, never in a shared storage ("All files access" is not needed). With Sioul keeping the folder itself, a device's own records go up before its sealed files, and the others' sealed files come down last, so that no large file on a slow line holds back the doses' answers (database.md, "Kept in step by Sioul itself").

Not built¶

Defences against poisoned training, which matter more since the filter learns from its own verdicts: a gate that tests each training batch on held-out mail before accepting it (RONI), and a cap on the weight of any one sender or campaign (research/spam-filter.md, section 3). Today a new table replaces the one in place only when it takes no more ham for spam on the newest mail (above, "The table changes only when no worse").
The connection's cost on Windows and macOS: not read; a run by itself there trains on the corpus as it is.
A sparse model to measure against: a linear model on hashed character n-grams is what the one-person filters of the TREC tracks were best with. fastText's classifier, now the filter's, is close to one (a linear model on words and their n-grams' buckets, learned for the labels); measured against the centroid on a real mailbox, it ranked mail better and gave the same numbers from one training to the next. A plain hashed n-gram model is still to measure.
Labels from what you do elsewhere: mail answered, flagged, tied to a task or a project, as ham.
Flags under a newer table: a flag is logged as the mail arrives, by the table then in use; a message a newer table flags later, while it still waits on the Porch, is not logged, and counts by its folder.
Outside material's headers: read now for the Message-ID only; their header features would need to know whose authentication results to trust there.
Moving what is already there when "Move to spam" is chosen, or a new table comes: only mail as it arrives moves; the rest waits, flagged, on the Porch. sioul spam dry-run --spam move says what it would move, first.
A next tokenizer: Virtual Secretary's photography units fire on everyday text ("10 to 20" read as a file size): consistent between training and judging, to drop in a next version.
The checks of the day the mail came, for old mail: the corpus checks a message when it fetches it, so an old message is checked with today's DNS (its passes kept, its failures unknown beside a signature: above). The stamp the sync wrote on the stored copy, made as it arrived, could stand for it where the stored copy is there; the corpus does not read the Maildir yet.

Tested¶

sioul-core: the tokenizer (Virtual Secretary's inputs, elisions, invisible characters, lookalikes), the features, the table (a score by hand, written and read back, refused when damaged or of another version, the last good one in use when the newest is refused), the verdict and the matrix (porch::tests::spam_never_touches_who_you_know, the_matrix_decides_what_the_filter_does: flag, move, nothing per class, protections, never notified, never counted, never closed by "Done"), the moved mail found in a Junk folder until you say (what_was_moved_waits_in_junk_for_your_word), the label logs (every device's read, the older one becoming this device's, the newest word winning by Message-ID, the moves and the flags kept apart, each in its own logs: labels::tests::what_the_filter_did_is_kept_apart), the settings (the matrix, the older mode's meaning kept), the view (no why, Spam and Not spam).
sioul-learn: the SVM against hand-solved optima, Platt, Clopper–Pearson, the corpus appended and read back after a crash, the labels' precedence (what the filter moved, and what it flagged as probably spam, learned as spam at once, wherever the message is now; a doubt left out unless a copy is in a Junk folder; your word above all, a keyword above a flag: labels::tests::the_filters_word_counts_until_you_say), a whole training on invented mail with the filter's logs as devices write them (train::tests::the_filters_word_is_learned_until_you_say), a whole training on invented mail (AUC, the fold within 1e-4, the replacement rule four ways), outside material (the format checked and counted, kept apart, replaced, removed; learned from with its header features at 0 once standardized, split by its own dates, its baseline measured, sioul spam eval's too), fastText's hashing against the crate's own; the corpus download against GreenMail (building.md).
sioul-sync: the table from the computer to the phone in quick exchanges only, byte for byte, the one before kept, nothing of it readable in the folder, never the language model nor the corpus (share::tests::the_spam_table_travels_alone_in_every_exchange); a label said on the phone reaching the computer's said_ham, the moves and the flags too, the newest word winning on both, nothing readable in the folder (a_label_on_one_device_reaches_the_others); a sealed file failing or slow, going up or coming down, never holding back a device's records (remote::tests::a_slow_or_failing_sealed_file_never_holds_back_the_records); "Spam" on what the filter moved keeping it in the Junk folder, the filter's move never a label (mailbox::tests::what_an_act_leaves_here_and_says).
sioul-app: what goes into the Junk folder after a fetch, and what is flagged where it is (spam::tests::what_the_filter_moves_or_flags_after_a_fetch).
Training by itself, sioul-learn: each condition alone holding it back, each unknown counting as no, a metered connection or one it cannot tell training without a download (auto::tests::each_condition_holds_it_back, unknown_counts_as_no, it_trains_when_everything_allows_it); a week after the last training, a quarter of an hour after a stop, a day after a failure or a vanished process (once_a_week_and_not_again_too_soon); the cheap facts first, the system asked only when the power alone holds it back, the record of a run kept and its end written by its own job only, a job's lock held (the_system_is_asked_last); the run stopped once unplugged, saving power or twice unable to tell, never otherwise (the_run_stops_when_the_power_changes). sioul-sync: NetworkManager's five answers, idle from the session, a supply known on both counts (power::tests). sioul-cli: how a run ended (spam::by_itself::tests::how_a_run_by_itself_ended); a run by itself as its job, a fake supply unplugged, its stop file seen, its end kept as stopped (mcp::tests::a_run_by_itself_stops_when_unplugged).
sioul-core, version 2: a provider's mark taken off a subject, the same words elsewhere kept (tokenize::tests::providers_marks_are_taken_off); the checks missing without results, the route missing where it cannot be read, a DKIM signature and a bounce address by the sender's own domain or not, the provider's verdict no feature (features::tests); a missing header value weighing nothing (table::tests::a_score_computed_by_hand).
sioul-learn, version 2: a missing value counted in neither the mean nor the deviation (train::tests::standardization_of_columns), each feature's mean by account and label, one sender's domain alone (diagnose::tests::means_by_account_and_label), the strict threshold (eval::tests::the_thresholds_count_what_they_should).
sioul-learn, the controls: the grid (eval::tests::the_thresholds_count_what_they_should), the counts by account and folder and the worst errors with their cards read again, outside material apart (detail::tests::the_worst_errors_with_their_cards), a trial writing nothing, its errors in order, eval's part never learned from (train::tests::a_trial_changes_nothing_and_names_its_errors).
sioul-core, version 3: the header facts as words, a missing one none, no text able to write one (features::tests::header_facts_as_words); the classifier's table scored by hand, a header fact's word and its share, written and read back, its kind told by its header weights (table::tests::the_classifier_scored_by_hand); format 1 refused (table::tests::a_damaged_or_foreign_table_is_refused).
sioul-learn, version 3: the classifier's table giving fastText's own prediction, words it knows and words it never saw, header facts and outside material without them, each known word's rows counted as the phone counts them, a header fact learned toward ham (supervised::tests::the_table_is_fasttexts_prediction); a whole training on invented mail with each model, the classifier's table of its kind (train::tests::learns_invented_spam_apart_from_ham); the replacement rule, the newest month, outside material and a trial, now the classifier's.
sioul-core, version 4: Sioul's checks first, the provider's when Sioul's are not there, another installation's stamp not read, the corpus's checks kept apart reading bit for bit as the stamp on top of the stored message (features::tests::sioul_first_then_the_provider); old mail's checks keeping their passes, their failures unknown beside a signature, never failed nor missing, as they are without one, and what counts as a signature (features::tests::old_checks_keep_their_passes_and_unsigned_failures); a table of features 3 refused (table::tests::a_damaged_or_foreign_table_is_refused).
sioul-learn, version 4: checks made after their records were kept read with them, a group cut short by a crash never read and cut off at the next, the counts as training reads them, an unsigned old message's failures kept (corpus::tests::checks_kept_apart_are_read_with_their_records); the groups fetched together, by size and count, the large ones by their headers (corpus::tests::checks_go_by_groups); what counts as no DNS answer (corpus::tests::checks_without_any_answer); the same message stamped by the sync and kept in the corpus with its checks giving the same words and features, its failure unknown when checked a year after it came beside its signature, kept without one, the provider's (missing here) without checks or when its server no longer had it (train::tests::training_and_sorting_read_the_same_checks); against GreenMail, each message checked as it comes, then a record kept before checked once by its UID, one whose UID or folder is gone said gone, nothing checked twice (corpus::tests::greenmail::greenmail_checks_each_message_once).
sioul-cli: the model chosen and its passes, word pairs only in a trial, the classifier's settings refused with the centroid (spam::report::tests::settings_in_their_ranges); the scores of every test message only when asked (mcp::tests::spam_reports_say_it_as_data).
sioul-cli, in a home of invented mail (mcp::tests): each report's data (spam_reports_say_it_as_data: status, eval with its errors, a trial, the review queue); a dry run with the settings' matrix and with "Move to spam" tried, the mail, the filter's files and its state the same after, file by file, size and time (a_dry_run_moves_and_writes_nothing); a label by Message-ID and by file, one line each, no file moved, a message only in the corpus labelled by its place, never moved (a_label_only_labels); the spam tools answering, refusing what is wrong before any job starts (spam_tools_answer); [mcp] spam = false (the_spam_tools_can_be_kept_from_agents); a job's file from start to stop, done, died, its permissions (a_jobs_file_from_start_to_end); and a job started apart by the real program, followed to its end (tests/spam_jobs.rs).