nl.inl.blacklab.search.lucene

Class SpanQuerySequence

java.lang.Object

org.apache.lucene.search.Query

org.apache.lucene.search.spans.SpanQuery

nl.inl.blacklab.search.lucene.BLSpanQuery

nl.inl.blacklab.search.lucene.SpanQuerySequence

All Implemented Interfaces:

Cloneable

public class SpanQuerySequence
extends BLSpanQuery

Combines spans, keeping only combinations of hits that occur one after the other. The order is significant: a hit from the first span must be followed by a hit from the second. Note that this class is different from org.apache.lucene.search.spans.SpanNearQuery: it tries to make sure it generates *all* possible sequence matches. SpanNearQuery doesn't do this; once a hit is used in a SpanNearQuery match, it advances to the next hit. In the future, this class could be expanded to make the exact behaviour configurable: find all matches / find longest matches / find shortest matches / ... See SpanSequenceRaw for details on the matching process.

Field Summary

Fields

Modifier and Type	Field and Description
protected String	baseFieldName The field name for this query.
protected List<BLSpanQuery>	clauses
protected static org.apache.logging.log4j.Logger	logger
protected String	luceneFieldName

Fields inherited from class nl.inl.blacklab.search.lucene.BLSpanQuery

MAX_UNLIMITED

Constructor Summary

Constructors

Constructor and Description
SpanQuerySequence(BLSpanQuery[] clauses)
SpanQuerySequence(BLSpanQuery first, BLSpanQuery second)
SpanQuerySequence(List<BLSpanQuery> clauscol)

Method Summary

Modifier and Type	Method and Description
boolean	canInternalizeNeighbour(BLSpanQuery clause, boolean onTheRight) Can this query "internalize" the given neighbouring clause? Internalizing means adding the clause to its children, which is often more efficient because we create longer sequences that match fewer hits and may themselves be further optimized.
boolean	canMakeNfa()
String	clausesToString(String field)
BLSpanWeight	createWeight(org.apache.lucene.search.IndexSearcher searcher, boolean needsScores)
boolean	equals(Object o)
int	forwardMatchingCost() Return an (very rough) indication of how expensive finding a match for this query using an NFA would be.
String	getField() Returns the name of the search field.
Nfa	getNfa(ForwardIndexAccessor fiAccessor, int direction)
String	getRealField()
int	hashCode()
boolean	hitsAllSameLength() Do our hits have constant length?
boolean	hitsAreUnique() Is it guaranteed that no two hits have the same start and end position?
boolean	hitsEndPointSorted() When hit B follows hit A, is it guaranteed that B.end >= A.end? Also, if A.end == B.end, is B.start > A.start?
boolean	hitsHaveUniqueEnd() Is it guaranteed that no two hits have the same end position?
boolean	hitsHaveUniqueStart() Is it guaranteed that no two hits have the same start position?
int	hitsLengthMax() How long could our longest hit be?
int	hitsLengthMin() How long could our shortest hit be?
boolean	hitsStartPointSorted() When hit B follows hit A, is it guaranteed that B.start >= A.start? Also, if A.start == B.start, is B.end > A.end?
SpanQuerySequence	internalizeNeighbour(BLSpanQuery clause, boolean addToRight) Create a new sequence with a clause added to it.
boolean	matchesEmptySequence() Does this query match the empty sequence? For example, the query [word="cow"]* matches the empty sequence.
protected boolean	matchingTagsToPosFilter(List<BLSpanQuery> clauses) Try to match separate start and end tags in this sequence, and convert into a position filter (e.g.
BLSpanQuery	noEmpty() Return a version of this clause that cannot match the empty sequence.
BLSpanQuery	optimize(org.apache.lucene.index.IndexReader reader) Called before rewrite() to optimize certain parts of the query before they are rewritten (e.g.
long	reverseMatchingCost(org.apache.lucene.index.IndexReader reader) Return an (very rough) indication of how many hits this clause might return.
BLSpanQuery	rewrite(org.apache.lucene.index.IndexReader reader)
protected List<BLSpanQuery>	rewriteClauses(org.apache.lucene.index.IndexReader reader)
static SpanQuerySequence	sequenceInternalize(BLSpanQuery whereToInternalize, BLSpanQuery clauseToInternalize, boolean addToRight) Either add a clause to an existing SpanQuerySequence, or create a new SpanQuerySequence with the two specified clauses.
void	setQueryInfo(QueryInfo queryInfo)
String	toString(String field)

Methods inherited from class nl.inl.blacklab.search.lucene.BLSpanQuery

addMaxValues, ensureSorted, ensureSortedUnique, getNfaTwoWay, inf, inverted, isSingleTokenNot, log, producesSingleTokens, wrap

Methods inherited from class org.apache.lucene.search.spans.SpanQuery

getTermContexts, getTermContexts

Methods inherited from class org.apache.lucene.search.Query

clone, getBoost, setBoost, toString

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

logger

protected static final org.apache.logging.log4j.Logger logger

baseFieldName

protected String baseFieldName

The field name for this query. The "base" part is only applicable when dealing with annotated fields: the base field name of "contents" and "contents%pos" would both be "contents".

luceneFieldName

protected String luceneFieldName

clauses

protected List<BLSpanQuery> clauses

Constructor Detail

SpanQuerySequence

public SpanQuerySequence(BLSpanQuery first,
                         BLSpanQuery second)

SpanQuerySequence

public SpanQuerySequence(List<BLSpanQuery> clauscol)

SpanQuerySequence

public SpanQuerySequence(BLSpanQuery[] clauses)

Method Detail

matchingTagsToPosFilter

protected boolean matchingTagsToPosFilter(List<BLSpanQuery> clauses)

Try to match separate start and end tags in this sequence, and convert into a position filter (e.g. containing) query. For example: []* 'bla' []* ==> containing 'bla'

Parameters:

clauses - clauses in which to find matching tags

Returns:

true if any rewriting was done, false if not

optimize

public BLSpanQuery optimize(org.apache.lucene.index.IndexReader reader)
                     throws IOException

Description copied from class: BLSpanQuery

Called before rewrite() to optimize certain parts of the query before they are rewritten (e.g. match regex terms using NFA instead of OR). For now, only SpanQuerySequence overrides this to make sure certain clause combinations are performed before rewrite().

Overrides:

optimize in class BLSpanQuery

Parameters:

reader - index reader

Returns:

optimized query

Throws:

IOException - on error

rewrite

public BLSpanQuery rewrite(org.apache.lucene.index.IndexReader reader)
                    throws IOException

Specified by:

rewrite in class BLSpanQuery

Throws:

IOException

matchesEmptySequence

public boolean matchesEmptySequence()

Description copied from class: BLSpanQuery

Does this query match the empty sequence? For example, the query [word="cow"]* matches the empty sequence. We need to know this so we can rewrite to the appropriate queries. A query of the form "AB*" would be translated into "A|AB+", so each component of the query actually generates non-empty matches. We default to no because most queries don't match the empty sequence.

Overrides:

matchesEmptySequence in class BLSpanQuery

Returns:

true if this query matches the empty sequence, false otherwise

noEmpty

public BLSpanQuery noEmpty()

Description copied from class: BLSpanQuery

Return a version of this clause that cannot match the empty sequence.

Returns:

a version that doesn't match the empty sequence

createWeight

public BLSpanWeight createWeight(org.apache.lucene.search.IndexSearcher searcher,
                                 boolean needsScores)
                          throws IOException

Specified by:

createWeight in class BLSpanQuery

Throws:

IOException

toString

public String toString(String field)

Specified by:

toString in class BLSpanQuery

hitsAllSameLength

public boolean hitsAllSameLength()

Description copied from class: BLSpanQuery

Do our hits have constant length?

Specified by:

hitsAllSameLength in class BLSpanQuery

Returns:

true if they do, false if not

hitsLengthMin

public int hitsLengthMin()

Description copied from class: BLSpanQuery

How long could our shortest hit be?

Specified by:

hitsLengthMin in class BLSpanQuery

Returns:

length of the shortest hit possible

hitsLengthMax

public int hitsLengthMax()

Description copied from class: BLSpanQuery

How long could our longest hit be?

Specified by:

hitsLengthMax in class BLSpanQuery

Returns:

length of the longest hit possible, or Integer.MAX_VALUE if unlimited

hitsEndPointSorted

public boolean hitsEndPointSorted()

Description copied from class: BLSpanQuery

When hit B follows hit A, is it guaranteed that B.end >= A.end? Also, if A.end == B.end, is B.start > A.start?

Specified by:

hitsEndPointSorted in class BLSpanQuery

Returns:

true if this is guaranteed, false if not

hitsStartPointSorted

public boolean hitsStartPointSorted()

Description copied from class: BLSpanQuery

When hit B follows hit A, is it guaranteed that B.start >= A.start? Also, if A.start == B.start, is B.end > A.end?

Specified by:

hitsStartPointSorted in class BLSpanQuery

Returns:

true if this is guaranteed, false if not

hitsHaveUniqueStart

public boolean hitsHaveUniqueStart()

Description copied from class: BLSpanQuery

Is it guaranteed that no two hits have the same start position?

Specified by:

hitsHaveUniqueStart in class BLSpanQuery

Returns:

true if this is guaranteed, false if not

hitsHaveUniqueEnd

public boolean hitsHaveUniqueEnd()

Description copied from class: BLSpanQuery

Is it guaranteed that no two hits have the same end position?

Specified by:

hitsHaveUniqueEnd in class BLSpanQuery

Returns:

true if this is guaranteed, false if not

hitsAreUnique

public boolean hitsAreUnique()

Description copied from class: BLSpanQuery

Is it guaranteed that no two hits have the same start and end position?

Specified by:

hitsAreUnique in class BLSpanQuery

Returns:

true if this is guaranteed, false if not

getNfa

public Nfa getNfa(ForwardIndexAccessor fiAccessor,
                  int direction)

Overrides:

getNfa in class BLSpanQuery

canMakeNfa

public boolean canMakeNfa()

Overrides:

canMakeNfa in class BLSpanQuery

reverseMatchingCost

public long reverseMatchingCost(org.apache.lucene.index.IndexReader reader)

Description copied from class: BLSpanQuery

Return an (very rough) indication of how many hits this clause might return. Used to decide what parts of the query to match using the forward index. Based on term frequency, which are combined using simple rules of thumb. Another way to think of this is an indication of how much computation this clause will require when matching using the reverse index.

Specified by:

reverseMatchingCost in class BLSpanQuery

Parameters:

reader - the index reader

Returns:

rough estimation of the number of hits

forwardMatchingCost

public int forwardMatchingCost()

Description copied from class: BLSpanQuery

Return an (very rough) indication of how expensive finding a match for this query using an NFA would be. Used to decide what parts of the query to match using the forward index.

Specified by:

forwardMatchingCost in class BLSpanQuery

Returns:

rough estimation of the NFA complexity

canInternalizeNeighbour

public boolean canInternalizeNeighbour(BLSpanQuery clause,
                                       boolean onTheRight)

Description copied from class: BLSpanQuery

Can this query "internalize" the given neighbouring clause? Internalizing means adding the clause to its children, which is often more efficient because we create longer sequences that match fewer hits and may themselves be further optimized. An example is SpanQueryPosFilter, which can be combined with fixed-length neighbouring clauses (updating the SpanQueryPosFilters' left or right adjustment setting to match) to reduce the number of hits that have to be filter.

Overrides:

canInternalizeNeighbour in class BLSpanQuery

Parameters:

clause - clause we want to internalize

onTheRight - if true, clause is a right neighbour of this query; if false, a left neighbour

Returns:

true iff clause can be internalized

internalizeNeighbour

public SpanQuerySequence internalizeNeighbour(BLSpanQuery clause,
                                              boolean addToRight)

Create a new sequence with a clause added to it.

Overrides:

internalizeNeighbour in class BLSpanQuery

Parameters:

clause - clause to add

addToRight - if true, add to the right; if false, to the left

Returns:

new sequence with clause added

sequenceInternalize

public static SpanQuerySequence sequenceInternalize(BLSpanQuery whereToInternalize,
                                                    BLSpanQuery clauseToInternalize,
                                                    boolean addToRight)

Either add a clause to an existing SpanQuerySequence, or create a new SpanQuerySequence with the two specified clauses.

Parameters:

whereToInternalize - existing sequence, or existing non-sequence clause

clauseToInternalize - clause to add to sequence or add to existing clause

addToRight - if true, add new clause to the right of existing; if false, to the left

Returns:

the expanded or newly created sequence

equals

public boolean equals(Object o)

Specified by:

equals in class BLSpanQuery

hashCode

public int hashCode()

Specified by:

hashCode in class BLSpanQuery

getField

public String getField()

Returns the name of the search field. In the case of a annotated field, the clauses may actually query different properties of the same annotated field (e.g. "description" and "description__pos"). That's why only the prefix is returned.

Overrides:

getField in class BLSpanQuery

Returns:

name of the search field

getRealField

public String getRealField()

Specified by:

getRealField in class BLSpanQuery

rewriteClauses

protected List<BLSpanQuery> rewriteClauses(org.apache.lucene.index.IndexReader reader)
                                    throws IOException

Throws:

IOException

clausesToString

public String clausesToString(String field)

setQueryInfo

public void setQueryInfo(QueryInfo queryInfo)

Overrides:

setQueryInfo in class BLSpanQuery