Package com.attivio.platform.transformer.ingest.util

Class TransformerUtils

  • Direct Known Subclasses:
    SdkTestUtils
    public class TransformerUtils
extends java.lang.Object
    A series of utility functions to make common transformer code patterns easier to develop.

    Transformers usually follow one of the following two patterns:

    Inspecting or modifying values in place

    For example a logging transfomer might want to log all of the values for the 'title' field or a lower case transformer might want to convert all of the text in a given field to lowercase. The following methods can be used for this purpose:

    • TransformerUtils#modifyFieldValues(ProcessingResult, IngestDocument, Iterable, AdvancedFieldValueModifier)
    • TransformerUtils#modifyFieldValues(ProcessingResult, IngestDocument, Iterable, FieldValueModifier)

    Mapping values from one field to another

    For example a pattern based extractor might want to scan the text in one field an output pattern matches to another output field. The following methods can be used for this purpose:

    • TransformerUtils#mapFieldValues(ProcessingResult, IngestDocument, Map, AdvancedFieldValueMapper)
    • TransformerUtils#mapFieldValues(ProcessingResult, IngestDocument, Map, FieldValueMapper)
    • TransformerUtils#mapFieldValues(ProcessingResult, IngestDocument, Map, AdvancedFieldValueMapper, NewFieldCreator)
    • TransformerUtils#mapFieldValues(ProcessingResult, IngestDocument, Map, FieldValueMapper, NewFieldCreator)

    Developing transformers

    In most cases the following code is the simplest way to use these classes: 
     // The actual modifier/mapper interface implemented should be chosen based on your transformer's needs.
 public class MyTransformer extends AbstractSingleDocumentTransformer implements FieldValueMapper {

   // provide getters/setters for this
   private Map fieldMapping = null;

   &Override
   public ProcessingResult processDocument(IngestDocument doc) throws AttivioException {
     return TransformerUtils.mapFieldValues(okResult(), doc, fieldMapping, this);
   }

   &Override
   public Object createMappedValue(ProcessingResult result, String fieldName, IngestFieldValue fv) {
     // this logic should be replaced with your own
     return String.valueof(fv.getValue()).toLowerCase();
   }

 }

    • Constructor Detail

      • TransformerUtils

        public TransformerUtils()

    • Method Detail

      • modifyFieldValues

        public static void modifyFieldValues​(IngestDocument doc,
                                     java.lang.Iterable<java.lang.String> input,
                                     FieldValueModifier modifier)
        Iterates over all fields in input, passing the field values for each to the modifier.
        Parameters:
        doc - the document to process
        input - the input fields to work on
        modifier - the modifier call for each field value

      • modifyFieldValues

        public static void modifyFieldValues​(IngestDocument doc,
                                     java.lang.Iterable<java.lang.String> input,
                                     AdvancedFieldValueModifier handler)
        Iterates over all fields in input, passing the field values for each to the modifier. The AdvancedFieldValueModifier and this method provide access to more data during each call than the FieldValueModifier and the TransformerUtils#modifyFieldValues(ProcessingResult, IngestDocument, Iterable, FieldValueModifier) method does.
        Parameters:
        doc - the document to process
        input - the input fields to work on
        handler - the modifier call for each field value

      • processDocument

        public static <T> boolean processDocument​(IngestDocument doc,
                                          FieldValueCreatingTransformer<T> transformer)
                                   throws AttivioException
        A processDocument utility method for FieldValueCreatingTransformer, because that class doesn't have one.
        Parameters:
        transformer -
        Returns:
        false if an actual workflow is supposed to drop the document and true if it is supposed to continue with it
        Throws:
        AttivioException

      • mapFieldValues

        public static void mapFieldValues​(IngestDocument doc,
                                  java.util.Map<java.lang.String,​java.lang.String> fieldMapping,
                                  FieldValueMapper mapper)
        Map each document based on the fieldMapping using the supplied FieldValueMapper.
        Parameters:
        doc - the document to process
        fieldMapping - the field mapping to use. The keys represent the input field names, the values represent the output field names.
        mapper - the mapper to use when processing data.

      • mapFieldValues

        public static void mapFieldValues​(IngestDocument doc,
                                  java.util.Map<java.lang.String,​java.lang.String> fieldMapping,
                                  AdvancedFieldValueMapper mapper)
        Map each document based on the fieldMapping using the supplied AdvancedFieldValueMapper. This method and the AdvancedFieldValueMapper provide access to more information in each call than the FieldValueMapper and the TransformerUtils#mapFieldValues(ProcessingResult, IngestDocument, Map, FieldValueMapper) method does.
        Parameters:
        doc - the document to process
        fieldMapping - the field mapping to use. The keys represent the input field names, the values represent the output field names.
        mapper - the mapper to use when processing data.

      • mapFieldValues

        public static void mapFieldValues​(IngestDocument doc,
                                  java.util.Map<java.lang.String,​java.lang.String> fieldMapping,
                                  FieldValueMapper mapper,
                                  NewFieldCreator newFieldCreator)
        Map each document based on the fieldMapping using the supplied FieldValueMapper.
        Parameters:
        doc - the document to process
        fieldMapping - the field mapping to use. The keys represent the input field names, the values represent the output field names.
        newFieldCreator - any new fields specified in the fieldMapping will be created using this factory
        mapper - the mapper to use when processing data.

      • mapFieldValues

        public static void mapFieldValues​(IngestDocument doc,
                                  java.util.Map<java.lang.String,​java.lang.String> fieldMapping,
                                  AdvancedFieldValueMapper mapper,
                                  NewFieldCreator newFieldCreator)
        Map each document based on the fieldMapping using the supplied FieldValueMapper. This method and the AdvancedFieldValueMapper provide access to more information in each call than the FieldValueMapper and the TransformerUtils#mapFieldValues(ProcessingResult, IngestDocument, Map, FieldValueMapper) method does.
        Parameters:
        doc - the document to process
        fieldMapping - the field mapping to use. The keys represent the input field names, the values represent the output field names.
        newFieldCreator - any new fields specified in teh fieldMapping will be created using this factory
        mapper - the mapper to use when processing data.

      • getAllOrSpecificFieldList

        public static java.util.Collection<java.lang.String> getAllOrSpecificFieldList​(IngestDocument doc,
                                                                               java.util.Collection<java.lang.String> input)
        Returns a specific list of input field names or all fields in the document if the input list is null or empty. If input contains values, the returned collection is the same list as passed in.