nl.inl.blacklab.index

Class DocIndexerAbstract

java.lang.Object

nl.inl.blacklab.index.DocIndexer

nl.inl.blacklab.index.DocIndexerAbstract

All Implemented Interfaces:

AutoCloseable

Direct Known Subclasses:

DocIndexerPlainTextBasic, DocIndexerXmlHandlers

public abstract class DocIndexerAbstract
extends DocIndexer

Abstract base class for a DocIndexer processing XML files.

Field Summary

Fields

Modifier and Type	Field and Description
protected int	nDocumentsSkipped
protected CountingReader	reader
protected boolean	skippingCurrentDocument
protected int	wordsDone Total words processed by this indexer.

Fields inherited from class nl.inl.blacklab.index.DocIndexer

currentLuceneDoc, documentName, docWriter, logger, MAX_DOCVALUES_LENGTH, metadataFieldValues, omitNorms, parameters

Constructor Summary

Constructors

Constructor and Description
DocIndexerAbstract() NOTE: newer DocIndexers should only have a default constructor, and provide methods to set the Indexer object and the document being indexed (which are called by the Indexer).
DocIndexerAbstract(DocWriter indexer, String fileName, Reader reader)

Method Summary

Modifier and Type	Method and Description
void	appendContent(char[] buffer, int start, int length)
void	appendContent(String str)
void	close()
protected int	getCharacterPosition() Returns the current position in the original XML content in chars.
static String	getDescription(Class<? extends DocIndexer> docIndexerClass) If the supplied class has a static getDescription() method, call it.
static String	getDisplayName(Class<? extends DocIndexer> docIndexerClass) If the supplied class has a static getDisplayName() method, call it.
static boolean	isVisible(Class<? extends DocIndexer> docIndexerClass) Should this docIndexer implementation be listed? A DocIndexer can be hidden by implementing a a static function named isVisible, returning false.
void	processContent(char[] buffer, int start, int length)
void	processContent(String contentToProcess)
void	reportCharsProcessed() Report the amount of new characters processed since the last call
void	reportTokensProcessed() Report the change in wordsDone since the last report
void	setDocument(Reader reader) Set the document to index.
void	startCaptureContent(String fieldName)
int	storeCapturedContent()
void	storePartCapturedContent()

Methods inherited from class nl.inl.blacklab.index.DocIndexer

addMetadataField, addMetadataFieldsFromParameters, addMetadataToDocument, addNumericFields, addToForwardIndex, getCurrentLuceneDoc, getDocWriter, getMetadataField, getMetadataFieldTypeFromIndexerProperties, getParameter, getParameter, getParameter, getParameter, getSensitivitySetting, hasParameter, index, luceneTypeFromIndexMetadataType, optTranslateFieldName, setDocument, setDocument, setDocument, setDocumentName, setDocWriter, setOmitNorms, setParameter, setParameters, tokenizeField, warn

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

skippingCurrentDocument

protected boolean skippingCurrentDocument

reader

protected CountingReader reader

wordsDone

protected int wordsDone

Total words processed by this indexer. Used for reporting progress, do not reset except when finished with file.

nDocumentsSkipped

protected int nDocumentsSkipped

Constructor Detail

DocIndexerAbstract

public DocIndexerAbstract()

NOTE: newer DocIndexers should only have a default constructor, and provide methods to set the Indexer object and the document being indexed (which are called by the Indexer). This allows us more flexibility in how we supply the document to this object (e.g. as a file, a byte array, an inputstream, a reader, ...), which helps if we want to use e.g. VTD-XML and could allow us to re-use DocIndexers in the future.

DocIndexerAbstract

public DocIndexerAbstract(DocWriter indexer,
                          String fileName,
                          Reader reader)

Method Detail

startCaptureContent

public void startCaptureContent(String fieldName)

storeCapturedContent

public int storeCapturedContent()

storePartCapturedContent

public void storePartCapturedContent()

appendContent

public void appendContent(String str)

appendContent

public void appendContent(char[] buffer,
                          int start,
                          int length)

processContent

public void processContent(char[] buffer,
                           int start,
                           int length)

processContent

public void processContent(String contentToProcess)

getCharacterPosition

protected int getCharacterPosition()

Returns the current position in the original XML content in chars.

Specified by:

getCharacterPosition in class DocIndexer

Returns:

the current char position

setDocument

public void setDocument(Reader reader)

Set the document to index.

Specified by:

setDocument in class DocIndexer

Parameters:

reader - document

close

public void close()
           throws BlackLabRuntimeException

Specified by:

close in interface AutoCloseable

Specified by:

close in class DocIndexer

Throws:

BlackLabRuntimeException

reportCharsProcessed

public final void reportCharsProcessed()

Description copied from class: DocIndexer

Report the amount of new characters processed since the last call

Specified by:

reportCharsProcessed in class DocIndexer

reportTokensProcessed

public final void reportTokensProcessed()

Report the change in wordsDone since the last report

Specified by:

reportTokensProcessed in class DocIndexer

getDisplayName

public static String getDisplayName(Class<? extends DocIndexer> docIndexerClass)

If the supplied class has a static getDisplayName() method, call it.

Parameters:

docIndexerClass - class to get the display name for

Returns:

display name, or empty string if method not found

getDescription

public static String getDescription(Class<? extends DocIndexer> docIndexerClass)

If the supplied class has a static getDescription() method, call it.

Parameters:

docIndexerClass - class to get the description for

Returns:

description, or empty string if method not found

isVisible

public static boolean isVisible(Class<? extends DocIndexer> docIndexerClass)

Should this docIndexer implementation be listed? A DocIndexer can be hidden by implementing a a static function named isVisible, returning false.

Parameters:

docIndexerClass -

Returns:

true if the format should be listed, false if it should be omitted. Defaults to true when the DocIndexer does not implement the method.