com.attivio.util

Class TokenUtils

java.lang.Object

com.attivio.util.TokenUtils

public final class TokenUtils
extends Object

Utility methods for working with TokenList objects.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TokenUtils.PositionBuffer Expert Use Only: Utility class for buffering and processing tokens indexed at the same position.
static class	TokenUtils.TokenBuffer Expert Use Only: represents a Token and associated positional data as part of a TokenUtils.PositionBuffer.

Constructor Summary

Constructors

Constructor and Description
TokenUtils()

Method Summary

Modifier and Type	Method and Description
static List<String>	extractScopes(String scopeName, IngestFieldValue value) Extract scopeName values from value.
static List<String>	extractScopes(String scopeName, TokenList tokens, String text) Extract substrings of text for scopeName from tokens.
static TokenList	filterTokens(IngestFieldValue value, Locale locale) Get all tokens for locale.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokenUtils

public TokenUtils()

Method Detail

filterTokens

public static TokenList filterTokens(IngestFieldValue value,
                                     Locale locale)

Get all tokens for locale.

NOTE: all returned tokens will be cloned.

Since:

5.5.0.1

extractScopes

public static List<String> extractScopes(String scopeName,
                                         IngestFieldValue value)

Extract scopeName values from value.

extractScopes

public static List<String> extractScopes(String scopeName,
                                         TokenList tokens,
                                         String text)

Extract substrings of text for scopeName from tokens.