Searching Across Multiple Elasticsearch Indices

Querying multiple Elasticsearch indices at once with a wildcard pattern versus an explicit comma-separated list.

Takahiro Iwasa

2020-12-31

2 min read

Elasticsearch supports querying across several indices at once, either with a wildcard pattern or an explicit comma-separated list.

Launching Elasticsearch

The following docker-compose.yml file configures a single-node cluster.

1
version: '3'
2

3
services:
4
  elasticsearch:
5
    image: elasticsearch:7.10.1
6
    container_name: elasticsearch
7
    environment:
8
      - discovery.type=single-node
9
      - bootstrap.memory_lock=true
10
    ports:
11
      - 9200:9200

Start the cluster:

docker-compose up -d

Preparing Data

Use curl to add a document to each of the users-2020-11 and users-2020-12 indices:

curl -X POST -H 'Content-Type: application/json' -d '{"name": "hoge"}' localhost:9200/users-2020-11/_doc/
curl -X POST -H 'Content-Type: application/json' -d '{"name": "fuga"}' localhost:9200/users-2020-12/_doc/

Searching Data

Searching with Wildcards

A wildcard expression searches multiple indices in a single request:

curl localhost:9200/users-2020-*/_search | jq .hits.hits

The response will contain data from both users-2020-11 and users-2020-12:

1
[
2
  {
3
    "_index": "users-2020-11",
4
    "_type": "_doc",
5
    "_id": "PNQ3tXYBKT-fwQ71grcz",
6
    "_score": 1,
7
    "_source": {
8
      "name": "hoge"
9
    }
10
  },
11
  {
12
    "_index": "users-2020-12",
13
    "_type": "_doc",
14
    "_id": "PdQ3tXYBKT-fwQ71p7cy",
15
    "_score": 1,
16
    "_source": {
17
      "name": "fuga"
18
    }
19
  }
20
]

Searching with a Comma-Separated List

Another way to search is by specifying multiple indices as a comma-separated list:

🔥 Caution

Ensure that your URL length does not exceed the maximum limit when using this approach.

curl localhost:9200/users-2020-11,users-2020-12/_search | jq .hits.hits

The response will contain data from both users-2020-11 and users-2020-12:

1
[
2
  {
3
    "_index": "users-2020-11",
4
    "_type": "_doc",
5
    "_id": "PNQ3tXYBKT-fwQ71grcz",
6
    "_score": 1,
7
    "_source": {
8
      "name": "hoge"
9
    }
10
  },
11
  {
12
    "_index": "users-2020-12",
13
    "_type": "_doc",
14
    "_id": "PdQ3tXYBKT-fwQ71p7cy",
15
    "_score": 1,
16
    "_source": {
17
      "name": "fuga"
18
    }
19
  }
20
]

Conclusion

Both the wildcard pattern and the explicit comma-separated list return the same hits from users-2020-11 and users-2020-12. The main practical difference is how the target indices are specified and maintained.

A wildcard pattern such as users-2020-* remains the same length as monthly indices accumulate, making it convenient for time-based index names. A comma-separated list is more explicit when the target set is small and known in advance, but the URL length must be considered as the list grows.

Back to posts