Search over Multiple Indices in Elasticsearch

Takahiro Iwasa

Dec 31, 2020

2 min read

Elasticsearch

Elasticsearch supports search over multiple indices, for example, index-2020-01, index-2020-02…

www.elastic.co

Search API | Elasticsearch Guide [8.13] | Elastic

(Optional, string) Comma-separated list of data streams, indices, and index aliases to search. Wildcard (*) expressions are supported.

An Elasticsearch cluster can be launched on your local PC using the Docker image. Please use the example docker-compose.yml below.

version: '3'

services:
  elasticsearch:
    image: elasticsearch:7.10.1
    container_name: elasticsearch
    environment:
      - discovery.type=single-node
      - bootstrap.memory_lock=true
    ports:
      - 9200:9200

Launch the Elasticsearch cluster with the following command.

docker-compose up -d

Index the data in users-2020-11 and users-2020-12 with the following command.

curl -X POST -H 'Content-Type: application/json' -d '{"name": "hoge"}' localhost:9200/users-2020-11/_doc/
curl -X POST -H 'Content-Type: application/json' -d '{"name": "fuga"}' localhost:9200/users-2020-12/_doc/

Search with the following command. You can see the data returned from users-2020-11 and users-2020-12 indices.

$ curl localhost:9200/users-2020-*/_search | jq .hits.hits

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   380  100   380    0     0  95000      0 --:--:-- --:--:-- --:--:-- 95000
[
  {
    "_index": "users-2020-11",
    "_type": "_doc",
    "_id": "PNQ3tXYBKT-fwQ71grcz",
    "_score": 1,
    "_source": {
      "name": "hoge"
    }
  },
  {
    "_index": "users-2020-12",
    "_type": "_doc",
    "_id": "PdQ3tXYBKT-fwQ71p7cy",
    "_score": 1,
    "_source": {
      "name": "fuga"
    }
  }
]

Search with the following command. You can see the data returned from users-2020-11 and users-2020-12 indices.

$ curl localhost:9200/users-2020-11,users-2020-12/_search | jq .hits.hits

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   380  100   380    0     0  63333      0 --:--:-- --:--:-- --:--:-- 63333
[
  {
    "_index": "users-2020-11",
    "_type": "_doc",
    "_id": "PNQ3tXYBKT-fwQ71grcz",
    "_score": 1,
    "_source": {
      "name": "hoge"
    }
  },
  {
    "_index": "users-2020-12",
    "_type": "_doc",
    "_id": "PdQ3tXYBKT-fwQ71p7cy",
    "_score": 1,
    "_source": {
      "name": "fuga"
    }
  }
]

Splitting multiple indices may get your clusters easy to operate.

I hope you will find this post useful.