[[boosting-by-popularity]]
=== Boosting by Popularity
Imagine that we have a website that hosts blog posts and enables users to vote for the
blog posts that they like.((("relevance", "controlling", "boosting by popularity")))((("popularity", "boosting by")))((("boosting", "by popularity"))) We would like more-popular posts to appear higher in the
results list, but still have the full-text score as the main relevance driver.
We can do this easily by storing the number of votes with each blog post:
[role="pagebreak-before"]
[source,json]
-------------------------------
PUT /blogposts/post/1
{
"title": "About popularity",
"content": "In this post we will talk about...",
"votes": 6
}
-------------------------------
At search time, we can use the `function_score` query ((("function_score query", "field_value_factor function")))((("field_value_factor function")))with the
`field_value_factor` function to combine the number of votes with the full-text relevance score:
[source,json]
-------------------------------
GET /blogposts/post/_search
{
"query": {
"function_score": { <1>
"query": { <2>
"multi_match": {
"query": "popularity",
"fields": [ "title", "content" ]
}
},
"field_value_factor": { <3>
"field": "votes" <4>
}
}
}
}
-------------------------------
<1> The `function_score` query wraps the main query and the function we would
like to apply.
<2> The main query is executed first.
<3> The `field_value_factor` function is applied to every document matching
the main `query`.
<4> Every document _must_ have a number in the `votes` field for
the `function_score` to work.
In the preceding example, the final `_score` for each document has been altered as
follows:
new_score = old_score * number_of_votes
This will not give us great results. The full-text `_score` range
usually falls somewhere between 0 and 10. As can be seen in <<img-popularity-linear>>, a blog post with 10 votes will
completely swamp the effect of the full-text score, and a blog post with 0
votes will reset the score to zero.
[[img-popularity-linear]]
.Linear popularity based on an original `_score` of `2.0`
image::images/elas_1701.png[Linear popularity based on an original `_score` of `2.0`]
==== modifier
A better way to incorporate popularity is to smooth out the `votes` value
with some `modifier`. ((("modifier parameter")))((("field_value_factor function", "modifier parameter")))In other words, we want the first few votes to count a
lot, but for each subsequent vote to count less. The difference between 0
votes and 1 vote should be much bigger than the difference between 10 votes
and 11 votes.
A typical `modifier` for this use case is `log1p`, which changes the formula
to the following:
new_score = old_score * log(1 + number_of_votes)
The `log` function smooths out the effect of the `votes` field to provide a
curve like the one in <<img-popularity-log>>.
[[img-popularity-log]]
.Logarithmic popularity based on an original `_score` of `2.0`
image::images/elas_1702.png[Logarithmic popularity based on an original `_score` of `2.0`]
The request with the `modifier` parameter looks like the following:
[source,json]
-------------------------------
GET /blogposts/post/_search
{
"query": {
"function_score": {
"query": {
"multi_match": {
"query": "popularity",
"fields": [ "title", "content" ]
}
},
"field_value_factor": {
"field": "votes",
"modifier": "log1p" <1>
}
}
}
}
-------------------------------
<1> Set the `modifier` to `log1p`.
[role="pagebreak-before"]
The available modifiers are `none` (the default), `log`, `log1p`, `log2p`,
`ln`, `ln1p`, `ln2p`, `square`, `sqrt`, and `reciprocal`. You can read more
about them in the
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#_field_value_factor[`field_value_factor` documentation].
==== factor
The strength of the popularity effect can be increased or decreased by
multiplying the value((("factor (function_score)")))((("field_value_factor function", "factor parameter"))) in the `votes` field by some number, called the
`factor`:
[source,json]
-------------------------------
GET /blogposts/post/_search
{
"query": {
"function_score": {
"query": {
"multi_match": {
"query": "popularity",
"fields": [ "title", "content" ]
}
},
"field_value_factor": {
"field": "votes",
"modifier": "log1p",
"factor": 2 <1>
}
}
}
}
-------------------------------
<1> Doubles the popularity effect
Adding in a `factor` changes the formula to this:
new_score = old_score * log(1 + factor * number_of_votes)
A `factor` greater than `1` increases the effect, and a `factor` less than `1`
decreases the effect, as shown in <<img-popularity-factor>>.
[[img-popularity-factor]]
.Logarithmic popularity with different factors
image::images/elas_1703.png[Logarithmic popularity with different factors]
==== boost_mode
Perhaps multiplying the full-text score by the result of the
`field_value_factor` function ((("function_score query", "boost_mode parameter")))((("boost_mode parameter")))still has too large an effect. We can control
how the result of a function is combined with the `_score` from the query by
using the `boost_mode` parameter, which accepts the following values:
`multiply`::
Multiply the `_score` with the function result (default)
`sum`::
Add the function result to the `_score`
`min`::
The lower of the `_score` and the function result
`max`::
The higher of the `_score` and the function result
`replace`::
Replace the `_score` with the function result
If, instead of multiplying, we add the function result to the `_score`, we can
achieve a much smaller effect, especially if we use a low `factor`:
[source,json]
-------------------------------
GET /blogposts/post/_search
{
"query": {
"function_score": {
"query": {
"multi_match": {
"query": "popularity",
"fields": [ "title", "content" ]
}
},
"field_value_factor": {
"field": "votes",
"modifier": "log1p",
"factor": 0.1
},
"boost_mode": "sum" <1>
}
}
}
-------------------------------
<1> Add the function result to the `_score`.
The formula for the preceding request now looks like this (see <<img-popularity-sum>>):
new_score = old_score + log(1 + 0.1 * number_of_votes)
[[img-popularity-sum]]
.Combining popularity with `sum`
image::images/elas_1704.png["Combining popularity with `sum`"]
==== max_boost
Finally, we can cap the maximum effect((("function_score query", "max_boost parameter")))((("max_boost parameter"))) that the function can have by using the
`max_boost` parameter:
[source,json]
-------------------------------
GET /blogposts/post/_search
{
"query": {
"function_score": {
"query": {
"multi_match": {
"query": "popularity",
"fields": [ "title", "content" ]
}
},
"field_value_factor": {
"field": "votes",
"modifier": "log1p",
"factor": 0.1
},
"boost_mode": "sum",
"max_boost": 1.5 <1>
}
}
}
-------------------------------
<1> Whatever the result of the `field_value_factor` function, it will never be
greater than `1.5`.
NOTE: The `max_boost` applies a limit to the result of the function only, not
to the final `_score`.
- Introduction
- 入门
- 是什么
- 安装
- API
- 文档
- 索引
- 搜索
- 聚合
- 小结
- 分布式
- 结语
- 分布式集群
- 空集群
- 集群健康
- 添加索引
- 故障转移
- 横向扩展
- 更多扩展
- 应对故障
- 数据
- 文档
- 索引
- 获取
- 存在
- 更新
- 创建
- 删除
- 版本控制
- 局部更新
- Mget
- 批量
- 结语
- 分布式增删改查
- 路由
- 分片交互
- 新建、索引和删除
- 检索
- 局部更新
- 批量请求
- 批量格式
- 搜索
- 空搜索
- 多索引和多类型
- 分页
- 查询字符串
- 映射和分析
- 数据类型差异
- 确切值对决全文
- 倒排索引
- 分析
- 映射
- 复合类型
- 结构化查询
- 请求体查询
- 结构化查询
- 查询与过滤
- 重要的查询子句
- 过滤查询
- 验证查询
- 结语
- 排序
- 排序
- 字符串排序
- 相关性
- 字段数据
- 分布式搜索
- 查询阶段
- 取回阶段
- 搜索选项
- 扫描和滚屏
- 索引管理
- 创建删除
- 设置
- 配置分析器
- 自定义分析器
- 映射
- 根对象
- 元数据中的source字段
- 元数据中的all字段
- 元数据中的ID字段
- 动态映射
- 自定义动态映射
- 默认映射
- 重建索引
- 别名
- 深入分片
- 使文本可以被搜索
- 动态索引
- 近实时搜索
- 持久化变更
- 合并段
- 结构化搜索
- 查询准确值
- 组合过滤
- 查询多个准确值
- 包含,而不是相等
- 范围
- 处理 Null 值
- 缓存
- 过滤顺序
- 全文搜索
- 匹配查询
- 多词查询
- 组合查询
- 布尔匹配
- 增加子句
- 控制分析
- 关联失效
- 多字段搜索
- 多重查询字符串
- 单一查询字符串
- 最佳字段
- 最佳字段查询调优
- 多重匹配查询
- 最多字段查询
- 跨字段对象查询
- 以字段为中心查询
- 全字段查询
- 跨字段查询
- 精确查询
- 模糊匹配
- Phrase matching
- Slop
- Multi value fields
- Scoring
- Relevance
- Performance
- Shingles
- Partial_Matching
- Postcodes
- Prefix query
- Wildcard Regexp
- Match phrase prefix
- Index time
- Ngram intro
- Search as you type
- Compound words
- Relevance
- Scoring theory
- Practical scoring
- Query time boosting
- Query scoring
- Not quite not
- Ignoring TFIDF
- Function score query
- Popularity
- Boosting filtered subsets
- Random scoring
- Decay functions
- Pluggable similarities
- Conclusion
- Language intro
- Intro
- Using
- Configuring
- Language pitfalls
- One language per doc
- One language per field
- Mixed language fields
- Conclusion
- Identifying words
- Intro
- Standard analyzer
- Standard tokenizer
- ICU plugin
- ICU tokenizer
- Tidying text
- Token normalization
- Intro
- Lowercasing
- Removing diacritics
- Unicode world
- Case folding
- Character folding
- Sorting and collations
- Stemming
- Intro
- Algorithmic stemmers
- Dictionary stemmers
- Hunspell stemmer
- Choosing a stemmer
- Controlling stemming
- Stemming in situ
- Stopwords
- Intro
- Using stopwords
- Stopwords and performance
- Divide and conquer
- Phrase queries
- Common grams
- Relevance
- Synonyms
- Intro
- Using synonyms
- Synonym formats
- Expand contract
- Analysis chain
- Multi word synonyms
- Symbol synonyms
- Fuzzy matching
- Intro
- Fuzziness
- Fuzzy query
- Fuzzy match query
- Scoring fuzziness
- Phonetic matching
- Aggregations
- overview
- circuit breaker fd settings
- filtering
- facets
- docvalues
- eager
- breadth vs depth
- Conclusion
- concepts buckets
- basic example
- add metric
- nested bucket
- extra metrics
- bucket metric list
- histogram
- date histogram
- scope
- filtering
- sorting ordering
- approx intro
- cardinality
- percentiles
- sigterms intro
- sigterms
- fielddata
- analyzed vs not
- 地理坐标点
- 地理坐标点
- 通过地理坐标点过滤
- 地理坐标盒模型过滤器
- 地理距离过滤器
- 缓存地理位置过滤器
- 减少内存占用
- 按距离排序
- Geohashe
- Geohashe
- Geohashe映射
- Geohash单元过滤器
- 地理位置聚合
- 地理位置聚合
- 按距离聚合
- Geohash单元聚合器
- 范围(边界)聚合器
- 地理形状
- 地理形状
- 映射地理形状
- 索引地理形状
- 查询地理形状
- 在查询中使用已索引的形状
- 地理形状的过滤与缓存
- 关系
- 关系
- 应用级别的Join操作
- 扁平化你的数据
- Top hits
- Concurrency
- Concurrency solutions
- 嵌套
- 嵌套对象
- 嵌套映射
- 嵌套查询
- 嵌套排序
- 嵌套集合
- Parent Child
- Parent child
- Indexing parent child
- Has child
- Has parent
- Children agg
- Grandparents
- Practical considerations
- Scaling
- Shard
- Overallocation
- Kagillion shards
- Capacity planning
- Replica shards
- Multiple indices
- Index per timeframe
- Index templates
- Retiring data
- Index per user
- Shared index
- Faking it
- One big user
- Scale is not infinite
- Cluster Admin
- Marvel
- Health
- Node stats
- Other stats
- Deployment
- hardware
- other
- config
- dont touch
- heap
- file descriptors
- conclusion
- cluster settings
- Post Deployment
- dynamic settings
- logging
- indexing perf
- rolling restart
- backup
- restore
- conclusion