🔎
cbElasticsearch
  • Getting Started
    • Installation
    • Configuration
    • Secondary Cluster
  • Indices
    • Managing Indices
      • Index Lifecycles
      • Index Templates
    • Mapping Builder
    • Aliases
    • Reindexing
    • Data Streams
  • Searching
    • Search
    • Aggregations
    • Suggestions
  • Documents
  • Logging
  • Pipelines
  • Tasks
  • Contributing
Powered by GitBook
On this page
  • Retrieving information on Indices
  • Creating an Index
  • Creating an Explicit Index Mapping
  • Using Client.ApplyIndex
  • Configuring Index Settings
  • Updating an Existing Index
  • Retrieving Settings for an Index
  • Retrieving Mappings for an Index
  • Triggering an index refresh
  • Getting Index Statistics
  • Creating Runtime Fields
  • Opening or Closing an Index
  • Deleting an Index
  • Additional Reading

Was this helpful?

Edit on GitHub
Export as PDF
  1. Indices

Managing Indices

Learn how to create, update and delete indices with CBElasticsearch

PreviousIndicesNextIndex Lifecycles

Last updated 8 months ago

Was this helpful?

Elasticsearch documents are stored in an "index", with the document structure defined by a "mapping". An Elasticsearch index is a JSON document store, and the mapping is a JSON configuration which defines the data type Elasticsearch should use for each document field.

By default, Elasticsearch will dynamically generate these index mapping when a document is saved to the index. See for more details.

Retrieving information on Indices

To retrieve a list of all indices on the connected cluster, use the client getIndices method:

var indexMap = getInstance( "Client@cbelasticsearch" ).getIndices();

This will return a struct of all indices ( with the names as keys ), which will provide additional information on each index, such as:

  • Any assigned aliases

  • The number of documents in the index

  • The size of the storage space used for the index in bytes

Creating an Index

The IndexBuilder model assists with the creation and mapping of indices. Mappings define the allowable data types within your documents and allow for better and more accurate search aggregations. Let's say we have a book model that we intend to make searchable via a bookshop index. Let's go ahead and create the index using the IndexBuilder:

var indexBuilder = getInstance( "IndexBuilder@cbelasticsearch" ).new( "bookshop" ).save();

This will create an empty index which we can begin populating with documents.

Creating an Explicit Index Mapping

To avoid the inherent troubles with dynamic mappings, you can define an explicit mapping using the properties argument:

getInstance( "IndexBuilder@cbelasticsearch" )
    .new(
        name = "bookshop",
        properties = {
            "title" : { "type" : "text" },
            "summary" : { "type" : "text" },
            "description" : { "type" : "text" },
            // denotes a nested struct with additional keys
            "author" : { "type" : "object" },
            // date with specific format type
            "publishDate" : {
                "type" : "date",
                // our format will be = yyyy-mm-dd
                "format" = "strict_date"
            },
            "edition" : { "type" : "integer" },
            "ISBN" : { "type" : "integer" }
        }
    )
    .save();

While it is not required that you explicitly define an index mapping, it is highly recommended since Elasticsearch's assumptions about the incoming document data may not always be correct. This leads to issues where the Elasticsearch-generated mapping is wrong and prevents further data from being indexed if it does not match the expected data type.

Using Client.ApplyIndex

In the previous examples, we've created the index and mapping from the IndexBuilder itself. If we wish, we could instead pass the IndexBuilder object to the Client@cbelasticsearch instance's applyIndex( required IndexBuilder indexBuilder ) method:

var myNewIndex = indexBuilder.new( "bookshop" )
                    .populate( getInstance( "BookshopIndexConfig@myApp" ).getConfig() );
getInstance( "Client@cbelasticsearch" ).applyIndex( myNewIndex );

Configuring Index Settings

indexBuilder.new(
    "bookshop",
    {
        "settings" : {
            "number_of_shards" : 10,
            "number_of_replicas" : 2,
            "auto_expand_replicas" : true,
            "shard.check_on_startup" : "checksum"
        },
        "mappings" : {
            "properties" : {
                "title" : { "type" : "text" },
                "summary" : { "type" : "text" },
                "description" : { "type" : "text" },
                // denotes a nested struct with additional keys
                "author" : { "type" : "object" },
                // date with specific format type
                "publishDate" : {
                    "type" : "date",
                    // our format will be = yyyy-mm-dd
                    "format" : "strict_date"
                },
                "edition" : { "type" : "integer" },
                "ISBN" : { "type" : "integer" }
            }
        }
    }

);

Updating an Existing Index

The IndexBuilder model also provides a patch() convenience method for updating the mapping or settings on an index:

indexBuilder.patch(
    index = "bookshop",
    settings = {
        "number_of_shards"      : 10,
        "number_of_replicas"    : 2,
        "auto_expand_replicas"  : true,
        "shard.check_on_startup": "checksum"
    }
);

Here's a quick example of using indexBuilder.patch() to add two new fields to an existing reviews index:

indexBuilder.patch(
    index = "reviews",
    properties = {
        "authorName"   : { "type" : "text" },
        "helpfulRating": { "type" : "integer" }
    }
);

Retrieving Settings for an Index

To retreive a list of all settings for an index you may use the getSettings method on the client.

var indexSettings = getInstance( "Client@cbelasticsearch" ).getSettings( "bookshop" )

Retrieving Mappings for an Index

To retreive a list of the configured mappings for an index you may use the getMappings method on the client.

var mappings = getInstance( "Client@cbelasticsearch" ).getMappings( "reviews" );

Triggering an index refresh

On occasion, you may need to ensure the index is updated in real time (immediately and synchronously). This can be done via the refreshIndex() client method:

var mappings = getInstance( "Client@cbelasticsearch" ).refreshIndex( "reviews" );

You can refresh multiple indices at once:

var mappings = getInstance( "Client@cbelasticsearch" ).refreshIndex( [ "reviews", "books" ] );
// OR
var mappings = getInstance( "Client@cbelasticsearch" ).refreshIndex( "reviews,books" );
var mappings = getInstance( "Client@cbelasticsearch" ).refreshIndex(
    [ "reviews", "book*" ],
    { "ignore_unavailable" : true }
);

Getting Index Statistics

To retrieve statistics on an index, use the getIndexStats() method:

var mappings = getInstance( "Client@cbelasticsearch" ).getIndexStats( "reviews" );

You can retrieve particular statistics metrics:

var mappings = getInstance( "Client@cbelasticsearch" )
                    .getIndexStats( "reviews", [ "indexing", "search" ] );

Or all metrics:

var mappings = getInstance( "Client@cbelasticsearch" )
                    .getIndexStats( "reviews", [ "_all" ] );

You can even retrieve all metrics on all indices by skipping the indexName parameter entirely:

var mappings = getInstance( "Client@cbelasticsearch" ).getIndexStats();

Finally, you can pass a struct of parameters to fine-tune the statistics result:

var mappings = getInstance( "Client@cbelasticsearch" )
                    .getIndexStats(
                        "reviews",
                        [ "_all" ],
                        { "level" : "shards", "fields" : "title,createdTime" }
                    );

Creating Runtime Fields

var script = getInstance( "Util@cbelasticsearch" )
.formatToPainless("
  if( doc['summary'].value.contains('love') ){ emit('😍');}
  if( doc['summary'].value.contains('great') ){ emit('🚀');}
  if( doc['summary'].value.contains('hate') ){ emit('😡');}
  if( doc['summary'].value.contains('broke') ){ emit('💔');}
");
getInstance( "Client@cbelasticsearch" )
        .patch( "reviews", {
            "mappings" : {
                "runtime" : {
                    "summarized_emotions" : {
                        "type" : "text",
                        "script" : {
                            "source" : script
                        }
                    }
                }
            }
        } );

Opening or Closing an Index

Certain index-level settings can not be applied while the index is open. To solve this, CBElasticsearch offers the .closeIndex() and .openIndex() methods:

var mappings = getInstance( "Client@CBElasticsearch" ).closeIndex( "reviews" );

// apply settings...

var mappings = getInstance( "Client@CBElasticsearch" ).openIndex( "reviews" );

Each of these methods accepts a struct of name/value (simple values only) arguments to pass in the query string:

var mappings = getInstance( "Client@CBElasticsearch" ).closeIndex( "reviews", { "ignore_unavailable" : true } );

Deleting an Index

All good things must come to an end, eh? You can use Client.deleteIndex() to delete an existing index:

getInstance( "Client@cbelasticsearch" ).deleteIndex( "reviews" )

Or you can use IndexBuilder.delete():

IndexBuilder.new( "reviews" ).delete();

Additional Reading

So far we've passed a simple struct of field mappings in to the index properties. If we wanted to add additional settings or configure replicas and shards, we could pass a more comprehensive struct, including a to the new() method to do so:

as well as pass to the refresh endpoint. This can be useful when using wildcards in the index/alias names:

Elasticsearch allows , which are fields calculated at search time and returned in the "fields" array.

This summarized_emotions field to display an array of emotions matching the review summary.

See for more information.

Deprecation notice: Custom index types since Elasticsearch v7.0, and should no longer be used. Only a single type will be accepted in future releases.

Dynamic Mappings in Elasticsearch
range of settings
supported query parameters
mapping runtime fields
can then be retrieved during a search
the Elasticsearch "Close Index" documentation
are deprecated
Elasticsearch Mapping Guide
Index Settings Reference