nl.inl.blacklab.indexers.config

Class DocIndexerPlainText

java.lang.Object

nl.inl.blacklab.index.DocIndexer

nl.inl.blacklab.indexers.config.DocIndexerBase

nl.inl.blacklab.indexers.config.DocIndexerConfig

nl.inl.blacklab.indexers.config.DocIndexerPlainText

All Implemented Interfaces:

AutoCloseable

public class DocIndexerPlainText
extends DocIndexerConfig

An indexer for tabular file formats, such as tab-separated or comma-separated values.

Field Summary

Fields inherited from class nl.inl.blacklab.indexers.config.DocIndexerConfig

config

Fields inherited from class nl.inl.blacklab.indexers.config.DocIndexerBase

wordsDone

Fields inherited from class nl.inl.blacklab.index.DocIndexer

currentLuceneDoc, documentName, docWriter, logger, MAX_DOCVALUES_LENGTH, metadataFieldValues, omitNorms, parameters

Constructor Summary

Constructors

Constructor and Description
DocIndexerPlainText()

Method Summary

Modifier and Type	Method and Description
void	close()
protected int	getCharacterPosition()
void	index() Index documents contained in a file.
void	indexSpecificDocument(String documentExpr) Index a specific document.
void	setConfigInputFormat(ConfigInputFormat config)
void	setDocument(byte[] contents, Charset defaultCharset) Set the document to index.
void	setDocument(File file, Charset defaultCharset) Set the document to index.
void	setDocument(InputStream is, Charset defaultCharset) Set the document to index.
void	setDocument(Reader reader) Set the document to index.
protected void	storeDocument() Store (or finish storing) the document in the content store.

Methods inherited from class nl.inl.blacklab.indexers.config.DocIndexerConfig

fromConfig, getSensitivitySetting, init, opChatFormatAgeToMonths, optTranslateFieldName, processLinkedDocument, processMetadataValue, processString, processStringMultipleValues, replaceDollarRefs

Methods inherited from class nl.inl.blacklab.indexers.config.DocIndexerBase

addAnnotatedField, addEndChar, addMetadataField, addStartChar, annotation, beginWord, dedupe, endDocument, endWord, getAnnotatedField, getAnnotatedFields, getAnnotation, getContentStoreName, getCurrentTokenPosition, getMainAnnotatedField, getMetadataFetcher, indexLinkedDocument, inlineTag, isStoreDocuments, propMain, propPunct, propTags, punctuation, reportCharsProcessed, reportTokensProcessed, resolveFileReference, setAddDefaultPunctuation, setCurrentAnnotatedFieldName, setPreventNextDefaultPunctuation, setStoreDocuments, shouldAddDefaultPunctuation, startDocument, storeWholeDocument, trace, traceln

Methods inherited from class nl.inl.blacklab.index.DocIndexer

addMetadataFieldsFromParameters, addMetadataToDocument, addNumericFields, addToForwardIndex, getCurrentLuceneDoc, getDocWriter, getMetadataField, getMetadataFieldTypeFromIndexerProperties, getParameter, getParameter, getParameter, getParameter, getSensitivitySetting, hasParameter, luceneTypeFromIndexMetadataType, setDocumentName, setDocWriter, setOmitNorms, setParameter, setParameters, tokenizeField, warn

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DocIndexerPlainText

public DocIndexerPlainText()

Method Detail

close

public void close()
           throws BlackLabRuntimeException

Specified by:

close in interface AutoCloseable

Specified by:

close in class DocIndexer

Throws:

BlackLabRuntimeException

setConfigInputFormat

public void setConfigInputFormat(ConfigInputFormat config)

Overrides:

setConfigInputFormat in class DocIndexerConfig

setDocument

public void setDocument(File file,
                        Charset defaultCharset)
                 throws FileNotFoundException

Description copied from class: DocIndexer

Set the document to index.

Overrides:

setDocument in class DocIndexer

Parameters:

file - file to index

defaultCharset - charset to use if no BOM found, or null for the default (utf-8)

Throws:

FileNotFoundException - if not found

setDocument

public void setDocument(byte[] contents,
                        Charset defaultCharset)

Description copied from class: DocIndexer

Set the document to index.

Overrides:

setDocument in class DocIndexer

Parameters:

contents - document contents

defaultCharset - charset to use if no BOM found, or null for the default (utf-8)

setDocument

public void setDocument(InputStream is,
                        Charset defaultCharset)

Description copied from class: DocIndexer

Set the document to index.

Overrides:

setDocument in class DocIndexer

Parameters:

is - document contents

defaultCharset - charset to use if no BOM found, or null for the default (utf-8)

setDocument

public void setDocument(Reader reader)

Description copied from class: DocIndexer

Set the document to index. NOTE: you should generally prefer calling the File or byte[] versions of this method, as those can be more efficient (e.g. when using DocIndexer that parses using VTD-XML).

Specified by:

setDocument in class DocIndexer

Parameters:

reader - document

index

public void index()
           throws MalformedInputFile,
                  PluginException,
                  IOException

Description copied from class: DocIndexer

Index documents contained in a file.

Overrides:

index in class DocIndexerConfig

Throws:

MalformedInputFile - if the input file wasn't valid

PluginException - if an error occurred in a plugin

IOException - if an I/O error occurred

indexSpecificDocument

public void indexSpecificDocument(String documentExpr)

Description copied from class: DocIndexerBase

Index a specific document. Only supported by DocIndexerBase.

Overrides:

indexSpecificDocument in class DocIndexerConfig

Parameters:

documentExpr - Expression (e.g. XPath) used to find the document to index in the file

storeDocument

protected void storeDocument()

Description copied from class: DocIndexerBase

Store (or finish storing) the document in the content store. Also set the content id field so we know how to retrieve it later.

Specified by:

storeDocument in class DocIndexerBase

getCharacterPosition

protected int getCharacterPosition()

Specified by:

getCharacterPosition in class DocIndexer