Module text 

Text helpers shared by the detectors.

Comparisons ignore case and accents (French, Spanish, Italian, German, Portuguese and Nordic ones), so “Réinitialiser” matches “reinitialiser” and “lunedì” “lunedi”. Folding maps one character to exactly one character, so a position found in the folded text is valid in the original text too: codes are matched on folded text and copied from the original.

Functions

contains_word

Whether text contains needle as a whole word, ignoring case and accents.

find_word

Position of needle as a whole word in folded haystack, from from on.

fold

The folded characters of a text.

fold_char

Lowercases one character and strips its accent, one character out for one in.

is_invisible 🔒

Characters that show nothing: zero-width spaces and joiners, the word joiner, the byte order mark, the soft hyphen, the combining grapheme joiner.

is_word_char

Whether a character can belong to a word, for word-boundary checks.

link_at 🔒

The length of the web address starting at at (“https://…”, “http://…”, “www.…”), its last punctuation left to the sentence; none when there is none.

readable

A text as the detectors read it: its web addresses left out (a tracking link is hundreds of characters long, and its digits are no code), with the brackets around them; invisible characters left out (the zero-width joiners that pad a newsletter’s preview line); each run of spaces made one space, or one line break when it holds one. A single space of any kind stays as it is: “482 913” with a narrow no-break space is still one code.

visible_len

How many characters a text shows, its web addresses and spaces left out.