Package com.attivio.util

Class TokenUtils

java.lang.Object

com.attivio.util.TokenUtils

public final class TokenUtils
extends java.lang.Object

Utility methods for working with TokenList objects.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	TokenUtils.PositionBuffer	Expert Use Only: Utility class for buffering and processing tokens indexed at the same position.
static class	TokenUtils.TokenBuffer	Expert Use Only: represents a Token and associated positional data as part of a TokenUtils.PositionBuffer.

Constructor Summary

Constructors

Constructor	Description
TokenUtils()

Method Summary

Modifier and Type	Method	Description
static java.util.List<java.lang.String>	extractScopes(java.lang.String scopeName, IngestFieldValue value)	Extract scopeName values from value.
static java.util.List<java.lang.String>	extractScopes(java.lang.String scopeName, TokenList tokens, java.lang.String text)	Extract substrings of text for scopeName from tokens.
static TokenList	filterTokens(IngestFieldValue value, java.util.Locale locale)	Get all tokens for locale.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokenUtils

public TokenUtils()

Method Detail

filterTokens

public static TokenList filterTokens(IngestFieldValue value,
                                     java.util.Locale locale)

Get all tokens for locale.

NOTE: all returned tokens will be cloned.

Since:

5.5.0 patch 95

extractScopes

public static java.util.List<java.lang.String> extractScopes(java.lang.String scopeName,
                                                             IngestFieldValue value)

Extract scopeName values from value.

extractScopes

public static java.util.List<java.lang.String> extractScopes(java.lang.String scopeName,
                                                             TokenList tokens,
                                                             java.lang.String text)

Extract substrings of text for scopeName from tokens.