== Filtering Queries and Aggregations A natural extension to aggregation scoping is filtering. Because the aggregation operates in the context of the query scope, any filter applied to the query will also apply to the aggregation. [float="true"] === Filtered Query If we want to find all cars over $10,000 and also calculate the average price for those cars,((("filtering", "serch query results")))((("filtered query")))((("queries", "filtered"))) we can simply use a `filtered` query: [source,js] -------------------------------------------------- GET /cars/transactions/_search?search_type=count { "query" : { "filtered": { "filter": { "range": { "price": { "gte": 10000 } } } } }, "aggs" : { "single_avg_price": { "avg" : { "field" : "price" } } } } -------------------------------------------------- // SENSE: 300_Aggregations/45_filtering.json Fundamentally, using a `filtered` query is no different from using a `match` query, as we discussed in the previous chapter. The query (which happens to include a filter) returns a certain subset of documents, and the aggregation operates on those documents. [float="true"] === Filter Bucket But what if you would like to filter just the aggregation results?((("filtering", "aggregation results, not the query")))((("aggregations", "filtering just aggregations"))) Imagine we are building the search page for our car dealership. We want to display search results according to what the user searches for. But we also want to enrich the page by including the average price of cars (matching the search) that were sold in the last month. We can't use simple scoping here, since there are two different criteria. The search results must match +ford+, but the aggregation results must match +ford+ AND +sold > now - 1M+. To solve this problem, we can use a special bucket called `filter`.((("filter bucket")))((("buckets", "filter"))) You specify a filter, and when documents match the filter's criteria, they are added to the bucket. Here is the resulting query: [source,js] -------------------------------------------------- GET /cars/transactions/_search?search_type=count { "query":{ "match": { "make": "ford" } }, "aggs":{ "recent_sales": { "filter": { <1> "range": { "sold": { "from": "now-1M" } } }, "aggs": { "average_price":{ "avg": { "field": "price" <2> } } } } } } -------------------------------------------------- // SENSE: 300_Aggregations/45_filtering.json <1> Using the `filter` bucket to apply a filter in addition to the `query` scope. <2> This `avg` metric will therefore average only docs that are both +ford+ and sold in the last month. Since the `filter` bucket operates like any other bucket, you are free to nest other buckets and metrics inside. All nested components will "inherit" the filter. This allows you to filter selective portions of the aggregation as required. [float="true"] === Post Filter So far, we have a way to filter both the search results and aggregations (a `filtered` query), as well as filtering individual portions of the aggregation (`filter` bucket). You may be thinking to yourself, "hmm...is there a way to filter _just_ the search results but not the aggregation?"((("filtering", "search results, not the aggregation")))((("post filter"))) The answer is to use a `post_filter`. This is a top-level search-request element that accepts a filter. The filter is applied _after_ the query has executed (hence the +post+ moniker: it runs _post query_ execution). Because it operates after the query has executed, it does not affect the query scope--and thus does not affect the aggregations either. We can use this behavior to apply additional filters to our search criteria that don't affect things like categorical facets in your UI. Let's design another search page for our car dealer. This page will allow the user to search for a car and filter by color. Color choices are populated via an aggregation: [source,js] -------------------------------------------------- GET /cars/transactions/_search?search_type=count { "query": { "match": { "make": "ford" } }, "post_filter": { <1> "term" : { "color" : "green" } }, "aggs" : { "all_colors": { "terms" : { "field" : "color" } } } } -------------------------------------------------- // SENSE: 300_Aggregations/45_filtering.json <1> The `post_filter` element is a +top-level+ element and filters just the search hits. The `query` portion is finding all +ford+ cars. We are then building a list of colors with a `terms` aggregation. Because aggregations operate in the query scope, the list of colors will correspond with the colors that Ford cars are painted. Finally, the `post_filter` will filter the search results to show only green +ford+ cars. This happens _after_ the query is executed, so the aggregations are unaffected. This is often important for coherent UIs. Imagine that a user clicks a category in your UI (for example, green). The expectation is that the search results are filtered, but _not_ the UI options. If you applied a `filtered` query, the UI would instantly transform to show _only_ +green+ as an option--not what the user wants! [WARNING] .Performance consideration ==== Use a `post_filter` _only_ if you need to differentially filter search results and aggregations. ((("post filter", "performance and")))Sometimes people will use `post_filter` for regular searches. Don't do this! The nature of the `post_filter` means it runs _after_ the query, so any performance benefit of filtering (such as caches) is lost completely. The `post_filter` should be used only in combination with aggregations, and only when you need differential filtering. ==== [float="true"] === Recap Choosing the appropriate type of filtering--search hits, aggregations, or both--often boils down to how you want your user interface to behave. Choose the appropriate filter (or combinations) depending on how you want to display results to your user. - A `filtered` query affects both search results and aggregations. - A `filter` bucket affects just aggregations. - A `post_filter` affects just search results.