spaCy - Util.compile_infix_regex

This utility function will compile a sequence of infix rules into a regex object.

Argument

The table below explains its argument −

Syntax

infixes = ("…", "-", "—", r"(?<=[0-9])[+-*^](?=[0-9-])")
infix_reg = util.compile_infix_regex(infixes)
nlp.tokenizer.infix_finditer = infix_reg.finditer

Example

import spacy
nlp = spacy.load('en_core_web_sm')
infixes = ('')
infix_reg = spacy.util.compile_infix_regex(infixes)
nlp.tokenizer.infix_finditer = infix_reg.finditer
doc = nlp("[A] works for [B] in [C].")
print([t.text for t in doc])
# ['[A]', 'works', 'for', '[B]', 'in', '[C]', '.']

Output

Output
['[', 'A', ']', 'w', 'o', 'r', 'k', 's', 'f', 'o', 'r', '[', 'B', ']', 'i', 'n', '[', 'C', ']', '.']