uralicNLP.string_processing

The uralicNLP.string_processing module has the following methods:

char_split

Splits words into characters better than Python's own " ".join("") method. This tries to maintain diacritics with the character they belong to instead of separating them. Take a look at the following example:

  from uralicNLP import string_processing
  s = 'h̭ɛ̮ŋkkɐᴅ'
  " ".join(s)
  >> h ̭ ɛ ̮ ŋ k k ɐ ᴅ
  string_processing.char_split(s)
  >> ['h̭', 'ɛ̮', 'ŋ', 'k', 'k', 'ɐ', 'ᴅ']

In short, it takes a string and returns a list split in characters.

filter_arabic

This return the parts of text that are written with Arabic characters. The parameters are

• text The text to process
• keep_vowels=True Whether diacritics should be removed
• combine_by="" Joins the Arabic text fragments by this string