Skip to main content

Crate sioul_sync

Crate sioul_sync 

Source
Expand description

Sioul’s side that talks to the world: mail servers, calendars and contacts, the keyring, other services (Google, GitHub, Bitwarden), and the sharing between your devices.

Fetching changes nothing on the server: folders are opened with EXAMINE and messages fetched with BODY.PEEK, so read flags, folders and mail stay as your other apps left them (RFC 9051 §6.3.3 and §6.4.5). The server is written to only when you act (mailbox::act, send). Connections are encrypted from the first byte or upgraded by STARTTLS before the login; there is no plain option.

What to do with what comes is decided in sioul-core: this crate brings what servers hold into the plain files the core reads (a Maildir, vdir folders), and takes back to them what you did. The window (sioul-app) and the command line (sioul) call it, and so does the spam filter’s training (sioul-learn) for the mail it learns from.

The functions that reach a server block until it answers: those that speak IMAP make a small tokio runtime of their own and wait for it, those that speak HTTP use ureq, which blocks. The window calls them off its own thread.

§Where to start reading

Names such as docs/database.md are the design notes in the repository’s docs/ folder. The website shows them too, under the same name: https://aurelienpierre.github.io/sioul/dev/database.html.

§Modules, by theme

§Mail

  • discover: finding an account’s servers from its address, as Thunderbird does.
  • scout: what a server offers for one address: mail, calendars and contacts, a Nextcloud’s apps.
  • imap: an encrypted, logged-in IMAP session.
  • sasl: signing in to IMAP and SMTP with an access token instead of a password (XOAUTH2).
  • fetch: new mail fetched read-only into the account’s Maildir, once or as it arrives.
  • mailbox: an account’s folders, and what you do to its messages on the server.
  • search: searching the mail a server holds and this device does not; a message found there brought here.
  • send: sending through SMTP submission, then a copy into Sent.
  • verify: Sioul’s own checks of who sent a message: SPF, DKIM, DMARC, ARC, reverse DNS.
  • unsubscribe: leaving a mailing list in one click, the request itself (RFC 8058).
  • keys: looking up someone’s OpenPGP key, only when you ask.
  • securitykey: your security key’s certificate, looked up when you ask; GnuPG asked to let the key go.
  • antivirus: checking a file before it opens.
  • shield_ai: the AI reading of a shielded address, when you allow it.
  • notify: the desktop’s notifications: a verified code, a reminder, new mail, the time running.

§Contacts, calendars and other services

  • dav: contacts and calendars on a server (CardDAV, CalDAV), synced both ways with the folders on disk.
  • google: Google’s calendars and contacts, signed in with OAuth 2.0.
  • google_tasks: Google Tasks, over its REST API.
  • github: GitHub’s REST API, read only: the issues and pull requests that are yours.
  • bitwarden: a Bitwarden vault, read only: logins and their one-time codes, to fill a site’s form.
  • favicon: a site’s own icon, asked of the site itself.
  • geocode: where an address is, asked of OpenStreetMap’s geocoder.
  • weather: the weather, asked of Open-Meteo.
  • ocr: a scan’s text, read by the system’s own programs.

§Passwords

  • secret: passwords in the system keyring, never in a file.
  • android, built for Android only (so not shown here): Android’s KeyStore and the network’s DNS servers, reached through Java.

§Sharing between your devices

  • share: what only this device keeps, shared with your other devices through a folder your sync app carries.
  • blobs: notes and papers in the shared folder, sealed, one file at a time.
  • history: what a shared file held before another device’s change, kept on this device.
  • devices: each device says how it is: when it started, closed, last read and wrote.
  • lease: one computer at a time for a part of the work: reminding doses, numbering invoices.
  • remote: the shared folder read from its server too, when the sync app is late.

§The computer

  • disk: the room left on a disk.
  • dnd: the desktop’s do-not-disturb during Sioul’s pauses (Linux and the BSDs).
  • power: on mains power or not, saving power or not, idle or not, a metered connection or not: for the spam filter’s training by itself.

Re-exports§

pub use discover::Found;
pub use discover::FoundBy;
pub use discover::Smtp;
pub use discover::discover;
pub use discover::discover_smtp;
pub use fetch::Control;
pub use fetch::Report;
pub use fetch::inbox;
pub use fetch::sync;
pub use fetch::test;
pub use fetch::watch;

Modules§

antivirus
Checking a file before it opens: one question, three answers, one way per system (docs/client.md, “Antivirus”).
bitwarden
Bitwarden, read here with no other program: your vault’s logins and their one-time codes, to fill a site’s form when you ask. Read only: nothing is ever written to the vault.
blobs
Notes and papers in the shared folder, one file at a time (docs/database.md, “Notes and papers”). Each file is compressed (gzip), sealed in pieces (XChaCha20-Poly1305), and kept once under a name made from its content: blobs/<name>. A record in the log says which content each file holds (share): a change sends that file alone, and a content in two places is sent once. Written once, under a hidden name then renamed: any sync app carries a new file. The folder’s server sees how big each is, never its name nor what it holds.
dav
Contacts and calendars on a server: CardDAV (RFC 6352) and CalDAV (RFC 4791), both WebDAV (RFC 4918), synced both ways with the folders on disk (sioul_core::vdir).
devices
Each device says how it is, in the sharing folder (docs/database.md, “Devices”): when it last started, when it last closed cleanly, whether it is working now, when it last read the others and when it last wrote its own. The doses rely on it (docs/health.md, “Knowing”): a device that closed cleanly after its last export holds no answer the others have not read; one still working may.
discover
Finding an account’s IMAP server from its address, the way Thunderbird does (https://wiki.mozilla.org/Thunderbird:Autoconfiguration):
disk
The room left on a disk, so fetching older mail, or your other devices’ notes, never fills it.
dnd
The desktop’s do-not-disturb during Sioul’s pauses (sioul-app’s dnd.rs, docs/pauses.md): who serves the notifications, Plasma’s inhibition held for as long as a pause lasts, and dconf’s word that a GNOME setting changed. Linux and the BSDs; the session’s D-Bus.
favicon
A site’s own icon, for the list of sites (docs/sites.md, “The list”): asked of the site itself, never of an icon service that would learn which sites you keep. Its page is read for the icon it names (<link rel="icon">, the larger the better), else /favicon.ico; the icon is kept in the cache folder, by host, and asked again after a week.
fetch
Fetching new mail, read-only, into the account’s Maildir.
filters
Mail filters, on the server (docs/client.md, “Filters”; what they say and how a message is matched: sioul_core::rules).
geocode
Where an address is, asked of OpenStreetMap’s geocoder (Nominatim), under its usage policy: Sioul says who it is, asks one address at a time, at most one a second (the caller waits), and keeps the answers (sioul_core::places), so an address is never asked twice.
github
GitHub’s REST API, read only, with your fine-grained token from the keyring (docs/github.md): the open issues and pull requests that are yours, found by search across GitHub (assigned to you, your review asked, opened by you, mentioning you), and one issue by its number. Answers are asked with their ETag: an unchanged one costs nothing and comes from the cache.
google
Google’s calendars and contacts, over its CalDAV and CardDAV, signed in with OAuth 2.0 as native apps should be (RFC 8252): your browser opens on Google’s page, Google sends the answer back to this computer (a loopback address, a port chosen at random), PKCE (RFC 7636) proves it is the same program asking. The OAuth client is Sioul’s own when the build was given one (SIOUL_GOOGLE_CLIENT_ID and SIOUL_GOOGLE_CLIENT_SECRET at build time: a key registered once for Sioul, as Thunderbird has its own), else yours, made once in your Google Cloud project (docs/google.md). What stays: the client and the refresh token, in the system keyring; the access token, in memory.
google_tasks
Google Tasks, over its REST API (v1): Google’s CalDAV keeps no task, so a Google account’s task lists come this way. Each list is a folder of VTODO files like any other (calendars/<account>/tasks-<id>), read by the rest of Sioul as it reads a CalDAV list. Google keeps a title, notes, done or not, a day (no hour), and one level of steps; what it does not keep shows greyed in the task form (sioul_core::capabilities). Lines Google does not know stay in the file here.
history
What a shared file held before another device’s change was written into it (docs/database.md, “Earlier versions”), kept on this device only and never shared: <state>/share/history/<part>/<file>/<time>. Of each file, the last 20 versions are kept, and all those of the last 30 days; of a file no longer here, those of the last 30 days only; never more than a cap in all, the oldest going first. A note or a paper whose content a record named is kept by reference (<time>.sealed): its content is sealed in the sharing folder already, and stays there long enough. Any version can be put back (share::put_back): the file as it is then is kept first, and the next exchange sends what was put back as a change made here.
imap
An encrypted, logged-in IMAP session.
keys
Looking up someone’s OpenPGP key, only when you ask: it tells a server to whom you are writing. First their domain’s own Web Key Directory (draft-koch-openpgp-webkey-service), which only that domain can fill, then keys.openpgp.org, which publishes a key only for an address that confirmed it.
lease
One computer at a time for a part of Sioul’s work (docs/database.md, “Leases”): reminding medicines, numbering invoices. Two computers doing either at once is a harm: a dose reminded again after it was taken, two invoices with one number.
mailbox
An account’s folders, and what you do to its messages.
notify
The one exception to the admin windows: a quiet desktop notification for a verified code, with the code in it (docs/porch.md, “Right now”); a gentle reminder; new mail at the times it may come, once per batch (docs/porch.md, “Notifications”); and the time running, while a focus session runs (docs/tasks.md).
ocr
Reading a scan’s text (docs/porch.md, “Paper letters”): a PDF’s own text when it has one (Poppler’s pdftotext), else its pages as images (pdftoppm) read by Tesseract, as photos and scanned images are. Both are the system’s own programs, as the antivirus is: optional, said when missing, with the command that installs them. Nothing leaves this computer.
power
What this computer says of itself, for work it may do while you are away (the spam filter’s training by itself: docs/spam-filter.md, “Training by itself”): on mains power or on its battery, saving power or not, idle or in use, its connection metered or not. Each fact is None when this computer cannot tell it (no such service, no answer in time, a system not read here): whoever decides counts an unknown fact as not holding.
remote
The other devices’ files fetched from the server too (docs/database.md, “Fetched from the server too”). The sharing folder stays the one a sync app carries, whichever it is; when that folder lives on a WebDAV server Sioul has an account for (Nextcloud, Murena, ownCloud), Sioul also reads it there, in case the sync app is late bringing the others’ files, or stops: Murena’s eDrive was seen bringing a computer’s records an hour and a half late, and its entry among the devices never.
said
The sentences of SyncError::Said: what Sioul found itself, said in your language, never in English inside a translated sentence.
sasl
Signing in to IMAP and SMTP with an OAuth 2.0 access token instead of a password: SASL XOAUTH2, as Google defines it (https://developers.google.com/workspace/gmail/imap/xoauth2-protocol), and as Microsoft takes it too. The client’s first answer is base64("user=" user "\x01auth=Bearer " token "\x01\x01"); a refused token brings a last challenge, the reason as base64 JSON ({"status":"401","schemes":"bearer mac","scope":"https://mail.google.com/"}), answered with an empty line, then the refusal (NO, 535).
scout
What a server offers for one address (docs/client.md, “Accounts”): mail (the provider’s own settings, as discover finds them), calendars and contacts (DAV, at the domain’s well-known addresses or on the server already known), and, on a Nextcloud (Murena’s among them), its apps: files, notes, Talk, Deck… what Sioul takes, and what it does not yet. Asked without a password, except a Nextcloud’s list of apps, which needs the one its calendars use.
search
Searching the mail a server holds and this device does not (docs/client.md, “Searching”).
secret
Passwords live in the system keyring, never in a file Sioul writes: the Secret Service on Linux (KWallet, GNOME Keyring), the Keychain on macOS, the Credential Manager on Windows (Cargo.toml chooses the backend per system). Each is filed under the service sioul and <login> on <server>, so it is easy to find, check or remove there.
securitykey
Your security key where it meets the world: its certificate looked up when you ask, and GnuPG asked to let the key go when you press “Let GnuPG release it”. What is checked and kept is the core’s (sioul_core::securitykey).
send
Sending: SMTP submission (RFC 6409), over TLS from the first byte (465, RFC 8314) or upgraded by STARTTLS (587), with the account’s login and password, or an access token (AUTH XOAUTH2) for an account signed in with Google; then a copy into Sent with APPEND, except at Gmail, which files its own.
share
What only this computer keeps, shared with your other computers through a folder your sync already carries: Nextcloud, Dropbox, Syncthing (docs/database.md, “The log”).
shield_ai
The AI reading of a shielded address (docs/shield.md). Only when you allow it for that address: the subject and text of each new message in its inbox are sent once to Anthropic’s API, with the question of shield::ai_prompt, and the answer (tone, topic, one neutral line) is kept by Message-ID. The key lives in the system keyring, under the service “sioul”.
stalls 🔒
The one place Sioul builds on ureq’s transport, ureq::unversioned, which ureq keeps out of its semver promise: any ureq release may change it. Written for ureq 3.4 (3.4.2 in Cargo.lock, 9 October 2026). When an update breaks the build here, this file alone changes; what it does stays the same: each wait for the network bounded, not only each phase of a request.
textseal
Texts sealed at rest (sioul_core::texts::Sealer; docs/texts.md, “Sealed at rest”): each line of the texts’ stores, on every device, is sealed by the device that writes it, so that a computer’s copy of your texts is unreadable without the sharing key, which each device keeps in its keyring (the Secret Service, the Credential Manager, the Keychain, or Android’s KeyStore). The sharing then seals the lines again in the folder, as it seals every record.
trigger
Shared as soon as Sioul saves (docs/database.md, “Sent as soon as it is saved”): when a file a sharing part carries changes here (a dose answered, a setting, health.toml, a project, a budget), an exchange runs a few seconds later, then this device’s files go to the server. One mechanism for every change, whoever wrote it: the window looks at the shared files every two seconds (Stamps), and a change seen, like an exchange asked (a button, a dose answered, the minute’s tick), wants one (Soon). Saves that follow each other wait for a pause: two seconds after the last, five after the first at most, so that a page saving each field as you type sends once, and soon.
unsubscribe
Leaving a mailing list in one click, on the network: the HTTPS POST of RFC 8058 (docs/client.md, “Unsubscribing”). When it may be sent is the core’s to say (sioul_core::unsubscribe); here it is sent as the RFC asks: List-Unsubscribe=One-Click as a form, no cookie, no authorisation, nothing of yours but what the list wrote in its own address. A redirect to another site is never followed; the wait is short.
verify
Sioul’s own checks of who sent a message: SPF, DKIM, DMARC, ARC and the reverse DNS of the server that handed it to your provider, done when the message is fetched, through the system’s DNS (Stalwart’s mail-auth).
weather
The weather and its places, asked of Open-Meteo: a place’s coordinates (two decimals, about a kilometre) and nothing else. No key, no account.

Enums§

Learned
What learning the provider’s authserv-id gave.
SyncError
What went wrong, in the categories the interface says in your language.

Constants§

GMAIL_APP_PASSWORDS
Where Google makes app passwords: Gmail refuses the usual password over IMAP.

Functions§

command 🔒
Another program (a scanner, a reader of scans), run unseen: on Windows, without a console window flashing over Sioul’s (CREATE_NO_WINDOW).
is_gmail
Whether an address is Gmail’s.
learn_provider
Learns which authserv-id the provider writes, from the account’s mail, and records it in the configuration (see trust::learn_provider_id).
tidy_password
The password as the server wants it. Google shows app passwords as four groups of four letters (“abcd efgh ijkl mnop”); the spaces are not part of it.
translator 🔒
Sioul’s words for what this crate says itself (a hint, a page in your browser): the configuration’s language, else the session’s, as the window.