spaCy - Retokenizer.merge Method



This retokenizer method will mark a span for merging.

Arguments

The table below explains its arguments −

NAME TYPE DESCRIPTION
Span Span It represents the span to merge.
Attrs dict These are the attributes to set on the merged token.

Example

An example of Retokenizer.merge method is given below −

import spacy
nlp_model = spacy.load("en_core_web_sm")
doc = nlp_model("This is Tutorialspoint.com.")
with doc.retokenize() as retokenizer:
   attrs = {"LEMMA": "Tutorialspoint.com"}
   retokenizer.merge(doc[2:4], attrs=attrs)
doc

Output

You will receive the following output −

This is Tutorialspoint.com.
spacy_doc_class_contextmanager_and_property.htm
Advertisements