Interface ConcurrentDocumentFetcherScanner

  • All Known Subinterfaces:
    ConcurrentScannerThrottlingAware

    public interface ConcurrentDocumentFetcherScanner
    This interface is implemented by the scanner if it wants to take advantage of the platform's concurrent scanner support. it allows the user to configure different aspects (e.g pool size and timeout values) of concurrent scanning. It also allow the connector framework to pass the scanner a ConcurrentProvider object the scanner will use to queue DocumentFetchRequest requests. It will also allow the connector framework to stop the scanner through the stop() call.

    If the scanner implements this interface it cannot feed document directly from the publisher, it will have to use the ConcurrencyProvider. Other publisher method such as delete will still be allowed.

    • Method Detail

      • setFetcherTimeout

        default void setFetcherTimeout​(long millis)
        Sets a timeout for the fetcher. The fetcher will fail if couldn't get the content before the timeout expired
        Parameters:
        millis -
      • getFetcherTimeout

        default long getFetcherTimeout()
        Gets a timeout for the fetcher. The fetcher will fail if couldn't get the content before the timeout expired
        Parameters:
        millis -
        Returns:
      • setThreadpoolSize

        default void setThreadpoolSize​(int queueSize)
        Sets the number of threads to use by fetch tasks.
        Parameters:
        queueSize -
      • getThreadpoolSize

        default int getThreadpoolSize()
        Gets the number of threads to use by fetch tasks.
        Returns:
      • setConcurrencyProvider

        void setConcurrencyProvider​(ConcurrencyProvider concurrencyProvider)
        The scanner will use the concurrency provider to queue content fetchers.
        Parameters:
        concurrencyProvider -
      • stop

        void stop()
        The stop call implementation should trigger the scanner to stop queuing new tasks and return from the start method