Python nltk.tokenize Module

This page shows the popular functions and classes defined in the nltk.tokenize module. The items are ordered by their popularity in 40,000 open source Python projects. If you can not find a good example below, you can try the search function to search modules.

• 1. word_tokenize()    
  Used in 127 projects
• 2. sent_tokenize()    
  Used in 78 projects
• 3. RegexpTokenizer()    
  Used in 39 projects
• 4. TweetTokenizer()    
  Used in 28 projects
• 5. WordPunctTokenizer()    
  Used in 18 projects
• 6. wordpunct_tokenize()    
  Used in 15 projects
• 7. TreebankWordTokenizer()    
  Used in 12 projects