nl.inl.blacklab.contentstore

Class ContentStoreFixedBlockWriter

java.lang.Object

nl.inl.blacklab.contentstore.ContentStore

nl.inl.blacklab.contentstore.ContentStoreDirAbstract

nl.inl.blacklab.contentstore.ContentStoreFixedBlock

nl.inl.blacklab.contentstore.ContentStoreFixedBlockWriter

@NotThreadSafe
public class ContentStoreFixedBlockWriter
extends ContentStoreFixedBlock

Store string content by id in a compound file and a TOC file. Quickly retrieve (parts of) the string content. Stores files in a file containing fixed-length (4K) blocks of zipped UTF-8. A file allocation table keeps track of each file's blocks as well as the character offset associated with each block so we can quickly access the data. Free blocks will be re-used to save space. Thread-safety: not thread-safe in index mode, but thread-safe while searching

Nested Class Summary

Nested classes/interfaces inherited from class nl.inl.blacklab.contentstore.ContentStore

ContentStore.DocTask

Field Summary

Fields inherited from class nl.inl.blacklab.contentstore.ContentStoreFixedBlock

AVERAGE_COMPRESSION_FACTOR, BLOCK_SIZE_BYTES, BYTES_PER_INT, CONSERVATIVE_COMPRESSION_FACTOR, CONTENT_STORE_TYPE_NAME, CONTENTS_FILE_NAME, contentsFile, CURRENT_VERSION, freeBlocks, initialized, MAX_BLOCK_SIZE_BYTES, MAX_COMPRESSION_FACTOR, MINIMUM_ACCEPTABLE_BLOCK_SIZE, nextId, POOL_SIZE, toc, TOC_FILE_NAME, tocFile, tocFileBuffer, tocFileChannel, tocRaf, totalBlocks, TYPICAL_BLOCK_SIZE_CHARACTERS, zipbufPool

Fields inherited from class nl.inl.blacklab.contentstore.ContentStoreDirAbstract

dir

Constructor Summary

Constructors

Constructor and Description
ContentStoreFixedBlockWriter(File dir, boolean create)

Method Summary

Modifier and Type	Method and Description
void	clear() Delete all content in the document store
void	close() Close the content store.
void	delete(int id) Delete a document from the content store.
protected byte[]	encodeBlock()
protected void	mapToc(boolean writeable)
protected void	performInitialization()
String	retrieve(int id) Retrieve content with given id
String[]	retrieveParts(int contentId, int[] start, int[] end) Retrieve substrings from a document.
void	setWriteMapReserve(int writeMapReserve) Set the size of the write reserve (the amount of space allocated at the end of the file).
int	store(String content) Store the given content and assign an id to it.
void	storePart(String content) Store part of a piece of large content.

Methods inherited from class nl.inl.blacklab.contentstore.ContentStoreFixedBlock

closeMappedToc, docLength, idSet, initialize, isDeleted, readToc, setStoreType

Methods inherited from class nl.inl.blacklab.contentstore.ContentStoreDirAbstract

getStoreTypeVersion, setStoreType, toString

Methods inherited from class nl.inl.blacklab.contentstore.ContentStore

forEachDocument, open, retrievePart

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ContentStoreFixedBlockWriter

public ContentStoreFixedBlockWriter(File dir,
                                    boolean create)
                             throws ErrorOpeningIndex

Parameters:

dir - content store dir

create - if true, create a new content store

Throws:

ErrorOpeningIndex

Method Detail

setWriteMapReserve

public void setWriteMapReserve(int writeMapReserve)

Set the size of the write reserve (the amount of space allocated at the end of the file). Larger reserves means less re-mapping. The default is 1M bytes.

Parameters:

writeMapReserve - size of the reserve in bytes.

clear

public void clear()
           throws IOException

Delete all content in the document store

Specified by:

clear in class ContentStore

Throws:

IOException

mapToc

protected void mapToc(boolean writeable)
               throws IOException

Specified by:

mapToc in class ContentStoreFixedBlock

Throws:

IOException

close

public void close()

Close the content store. Writes the table of contents (if modified)

Overrides:

close in class ContentStoreFixedBlock

storePart

public void storePart(String content)

Store part of a piece of large content. This may be called several times to store chunks of content, but MUST be *finished* by calling the "normal" store() method. You may call store() with the empty string if you wish.

Specified by:

storePart in class ContentStore

Parameters:

content - the content to store

store

public int store(String content)

Store the given content and assign an id to it.

Specified by:

store in class ContentStore

Parameters:

content - the content to store

Returns:

the id assigned to the content

retrieve

public String retrieve(int id)

Retrieve content with given id

Specified by:

retrieve in class ContentStore

Parameters:

id - the id

Returns:

the string

retrieveParts

public String[] retrieveParts(int contentId,
                              int[] start,
                              int[] end)

Description copied from class: ContentStore

Retrieve substrings from a document.

Specified by:

retrieveParts in class ContentStore

Parameters:

contentId - content store document id

start - start of the substring

end - end of the substring

Returns:

the substrings

delete

public void delete(int id)

Description copied from class: ContentStore

Delete a document from the content store.

Specified by:

delete in class ContentStore

Parameters:

id - content store id of the document to delete

encodeBlock

protected byte[] encodeBlock()

performInitialization

protected void performInitialization()

Specified by:

performInitialization in class ContentStoreFixedBlock