Xapian: API Documentation: Xapian::LMDirichletWeight Class Reference

xapian-core 2.0.0

Public Member Functions | List of all members
Xapian::LMDirichletWeight Class Reference

Language Model weighting with Dirichlet or Dir+ smoothing. More...

`#include <weight.h>`

Inheritance diagram for Xapian::LMDirichletWeight:
[legend]

## Public Member Functions
| LMDirichletWeight (double mu=2000.0, double delta=0.05)
| Construct a LMDirichletWeight.
double | get_sumpart (Xapian::termcount wdf, Xapian::termcount doclen, Xapian::termcount uniqterm, Xapian::termcount wdfdocmax) const
| Calculate the weight contribution for this object's term to a document.
double | get_maxpart () const
| Return an upper bound on what get_sumpart() can return for any document.
double | get_sumextra (Xapian::termcount doclen, Xapian::termcount, Xapian::termcount) const
| Calculate the term-independent weight component for a document.
double | get_maxextra () const
| Return an upper bound on what get_sumextra() can return for any document.
std::string | name () const
| Return the name of this weighting scheme, e.g.
std::string | serialise () const
| Return this object's parameters serialised as a single string.
LMDirichletWeight * | unserialise (const std::string &serialised) const
| Unserialise parameters.
LMDirichletWeight * | create_from_parameters (const char *params) const
| Create from a human-readable parameter string.
Public Member Functions inherited from Xapian::Weight
| Weight ()
| Default constructor, needed by subclass constructors.
virtual | ~Weight ()
| Virtual destructor, because we have virtual methods.

## Additional Inherited Members
Static Public Member Functions inherited from Xapian::Weight
static const Weight * | create (const std::string &scheme, const Registry &reg=Registry())
| Return the appropriate weighting scheme object.
Protected Types inherited from Xapian::Weight
enum | stat_flags { COLLECTION_SIZE = 0 , RSET_SIZE = 0 , AVERAGE_LENGTH = 4 , TERMFREQ = 1 , RELTERMFREQ = 1 , QUERY_LENGTH = 0 , WQF = 0 , WDF = 2 , DOC_LENGTH = 8 , DOC_LENGTH_MIN = 16 , DOC_LENGTH_MAX = 32 , WDF_MAX = 64 , COLLECTION_FREQ = 1 , UNIQUE_TERMS = 128 , TOTAL_LENGTH = 256 , WDF_DOC_MAX = 512 , UNIQUE_TERMS_MIN = 1024 , UNIQUE_TERMS_MAX = 2048 , DB_DOC_LENGTH_MIN = 4096 , DB_DOC_LENGTH_MAX = 8192 , DB_UNIQUE_TERMS_MIN = 16384 , DB_UNIQUE_TERMS_MAX = 32768 , DB_WDF_MAX = 65536 , IS_BOOLWEIGHT_ = static_cast<int>(0x80000000) }
| Stats which the weighting scheme can use (see need_stat()). More...
Protected Member Functions inherited from Xapian::Weight
void | need_stat (stat_flags flag)
| Tell Xapian that your subclass will want a particular statistic.
| Weight (const Weight &)
| Don't allow copying.
Xapian::doccount | get_collection_size () const
| The number of documents in the collection.
Xapian::doccount | get_rset_size () const
| The number of documents marked as relevant.
Xapian::doclength | get_average_length () const
| The average length of a document in the collection.
Xapian::doccount | get_termfreq () const
| The number of documents which this term indexes.
Xapian::doccount | get_reltermfreq () const
| The number of relevant documents which this term indexes.
Xapian::termcount | get_collection_freq () const
| The collection frequency of the term.
Xapian::termcount | get_query_length () const
| The length of the query.
Xapian::termcount | get_wqf () const
| The within-query-frequency of this term.
Xapian::termcount | get_doclength_upper_bound () const
| An upper bound on the maximum length of any document in the shard.
Xapian::termcount | get_doclength_lower_bound () const
| A lower bound on the minimum length of any document in the shard.
Xapian::termcount | get_wdf_upper_bound () const
| An upper bound on the wdf of this term in the shard.
Xapian::totallength | get_total_length () const
| Total length of all documents in the collection.
Xapian::termcount | get_unique_terms_upper_bound () const
| A lower bound on the number of unique terms in any document in the shard.
Xapian::termcount | get_unique_terms_lower_bound () const
| An upper bound on the number of unique terms in any document in the shard.
Xapian::termcount | get_db_doclength_upper_bound () const
| An upper bound on the maximum length of any document in the database.
Xapian::termcount | get_db_doclength_lower_bound () const
| A lower bound on the minimum length of any document in the database.
Xapian::termcount | get_db_unique_terms_upper_bound () const
| A lower bound on the number of unique terms in any document in the database.
Xapian::termcount | get_db_unique_terms_lower_bound () const
| An upper bound on the number of unique terms in any document in the database.
Xapian::termcount | get_db_wdf_upper_bound () const
| An upper bound on the wdf of this term in the database.

## Detailed Description

Language Model weighting with Dirichlet or Dir+ smoothing.

Dirichlet smoothing is as described in:

Zhai, C., & Lafferty, J.D. (2004). A study of smoothing methods for language models applied to information retrieval. ACM Trans. Inf. Syst., 22, 179-214.

Dir+ is described in:

Lv, Y., & Zhai, C. (2011). Lower-bounding term frequency normalization. International Conference on Information and Knowledge Management.

Since
  2.0.0

## Constructor & Destructor Documentation

## ◆ LMDirichletWeight()

Xapian::LMDirichletWeight::LMDirichletWeight | ( | double | mu = 2000.0,
|  | double | delta = 0.05 )
| inlineexplicit

Construct a LMDirichletWeight.

Parameters

  mu | A parameter which is > 0. Default: 2000
  delta | A parameter which is >= 0, which is "a pseudo [wdf] value to control the scale of the [wdf] lower bound". If this parameter is > 0, then the smoothing is Dir+; if it's zero, it's Dirichlet. Default: 0.05

References Xapian::Weight::COLLECTION_FREQ, Xapian::Weight::DOC_LENGTH, Xapian::Weight::DOC_LENGTH_MAX, Xapian::Weight::DOC_LENGTH_MIN, Xapian::Weight::need_stat(), Xapian::Weight::QUERY_LENGTH, Xapian::Weight::TOTAL_LENGTH, Xapian::Weight::WDF, Xapian::Weight::WDF_MAX, and Xapian::Weight::WQF.

Referenced by create_from_parameters(), and unserialise().

## Member Function Documentation

## ◆ create_from_parameters()

LMDirichletWeight * Xapian::LMDirichletWeight::create_from_parameters | ( | const char * | params | ) | const
| virtual

Create from a human-readable parameter string.

Parameters

  params | string containing weighting scheme parameter values.

Since
  2.0.0

Reimplemented from Xapian::Weight.

References LMDirichletWeight().

## ◆ get_maxextra()

double Xapian::LMDirichletWeight::get_maxextra | ( |  | ) | const
| virtual

Return an upper bound on what get_sumextra() can return for any document.

The default implementation always returns 0 (in Xapian < 2.0.0 this was a pure virtual method).

This information is used by the matcher to perform various optimisations, so strive to make the bound as tight as possible.

Reimplemented from Xapian::Weight.

## ◆ get_maxpart()

double Xapian::LMDirichletWeight::get_maxpart | ( |  | ) | const
| virtual

Return an upper bound on what get_sumpart() can return for any document.

This information is used by the matcher to perform various optimisations, so strive to make the bound as tight as possible.

Implements Xapian::Weight.

## ◆ get_sumextra()

double Xapian::LMDirichletWeight::get_sumextra | ( | Xapian::termcount | doclen,
|  | Xapian::termcount | uniqterms,
|  | Xapian::termcount | wdfdocmax ) const
| virtual

Calculate the term-independent weight component for a document.

The default implementation always returns 0 (in Xapian < 2.0.0 this was a pure virtual method).

The parameter gives information about the document which may be used in the calculations:

Parameters

  doclen | The document's length (unnormalised). You need to call need_stat(DOC_LENGTH) if you use this value.
  uniqterms | Number of unique terms in the document. You need to call need_stat(UNIQUE_TERMS) if you use this value.
  wdfdocmax | Maximum wdf value in the document. You need to call need_stat(WDF_DOC_MAX) if you use this value.

Since
  The wdfdocmax parameter was added in Xapian 2.0.0.

Reimplemented from Xapian::Weight.

## ◆ get_sumpart()

double Xapian::LMDirichletWeight::get_sumpart | ( | Xapian::termcount | wdf,
|  | Xapian::termcount | doclen,
|  | Xapian::termcount | uniqterms,
|  | Xapian::termcount | wdfdocmax ) const
| virtual

Calculate the weight contribution for this object's term to a document.

The parameters give information about the document which may be used in the calculations:

Parameters

  wdf | The within document frequency of the term in the document. You need to call need_stat(WDF) if you use this value.
  doclen | The document's length (unnormalised). You need to call need_stat(DOC_LENGTH) if you use this value.
  uniqterms | Number of unique terms in the document. You need to call need_stat(UNIQUE_TERMS) if you use this value.
  wdfdocmax | Maximum wdf value in the document. You need to call need_stat(WDF_DOC_MAX) if you use this value.

You can rely of wdf <= doclen if you call both need_stat(WDF) and need_stat(DOC_LENGTH) - this is trivially true for terms, but Xapian also ensure it's true for OP_SYNONYM, where the wdf is approximated.

Since
  The wdfdocmax parameter was added in Xapian 2.0.0.

Implements Xapian::Weight.

## ◆ name()

std::string Xapian::LMDirichletWeight::name | ( |  | ) | const
| virtual

Return the name of this weighting scheme, e.g.

"bm25+".

This is the name that the weighting scheme gets registered under when passed to Xapian:Registry::register_weighting_scheme().

As a result:

- this is the name that needs to be used in Weight::create() to create a Weight object from a human-readable string description.
- it is also used by the remote backend where it is sent (along with the serialised parameters) to the remote server so that it knows which class to create.

For 1.4.x and earlier we recommended returning the full namespace-qualified name of your class here, but now we recommend returning a just the name in lower case, e.g. "foo" instead of "FooWeight", "bm25+" instead of "Xapian::BM25PlusWeight".

If you don't want to support creation via Weight::create() or the remote backend, you can use the default implementation which simply returns an empty string.

Reimplemented from Xapian::Weight.

## ◆ serialise()

std::string Xapian::LMDirichletWeight::serialise | ( |  | ) | const
| virtual

Return this object's parameters serialised as a single string.

If you don't want to support the remote backend, you can use the default implementation which simply throws Xapian::UnimplementedError.

Reimplemented from Xapian::Weight.

## ◆ unserialise()

LMDirichletWeight * Xapian::LMDirichletWeight::unserialise | ( | const std::string & | serialised | ) | const
| virtual

Unserialise parameters.

This method unserialises parameters serialised by the serialise() method and allocates and returns a new object initialised with them.

If you don't want to support the remote backend, you can use the default implementation which simply throws Xapian::UnimplementedError.

Note that the returned object will be deallocated by Xapian after use with "delete". If you want to handle the deletion in a special way (for example when wrapping the Xapian API for use from another language) then you can define a static `operator delete` method in your subclass as shown here: https://trac.xapian.org/ticket/554#comment:1

Parameters

  serialised | A string containing the serialised parameters.

Reimplemented from Xapian::Weight.

References LMDirichletWeight().

The documentation for this class was generated from the following file:

- xapian/weight.h