Configuration

cbElasticsearch
Search…
Introduction
Configuration
Configuration
Once you have installed the module, you may add a custom configuration, specific to your environment, by adding an cbElasticsearch configuration object to your moduleSettings inside your Coldbox.cfc configuration file.
By default the following are in place, without additional configuration:
1
moduleSettings = {
2
    "cbElasticsearch" = {
3
        //The native client Wirebox DSL for the transport client
4
        client="[email protected]",
5
        // The default hosts - an array of host connections
6
        //  - REST-based clients (e.g. JEST):  round robin connections will be used
7
        //  - Socket-based clients (e.g. Transport):  cluster-aware routing used
8
        versionTarget = getSystemSetting( "ELASTICSEARCH_VERSION", '' ),
9
        hosts = [
10
            //The default connection is made to http://127.0.0.1:9200
11
            {
12
                serverProtocol: getSystemSetting( "ELASTICSEARCH_PROTOCOL", "http" ),
13
                serverName: getSystemSetting( "ELASTICSEARCH_HOST", "127.0.0.1" ),
14
                serverPort: getSystemSetting( "ELASTICSEARCH_PORT", 9200 )
15
            }
16
        ],
17
        // The default credentials for access, if any - may also be overridden when searching index collections
18
        defaultCredentials = {
19
            "username" : getSystemSetting( "ELASTICSEARCH_USERNAME", "" ),
20
            "password" : getSystemSetting( "ELASTICSEARCH_PASSWORD", "" )
21
        },
22
        // The default index
23
        defaultIndex           = getSystemSetting( "ELASTICSEARCH_INDEX", "cbElasticsearch" ),
24
        // The default number of shards to use when creating an index
25
        defaultIndexShards     = getSystemSetting( "ELASTICSEARCH_SHARDS", 5 ),
26
        // The default number of index replicas to create
27
        defaultIndexReplicas   = getSystemSetting( "ELASTICSEARCH_REPLICAS", 0 ),
28
        // Whether to use separate threads for client transactions
29
        multiThreaded          = true,
30
        // The maximum amount of time to wait until releasing a connection (in seconds)
31
        maxConnectionIdleTime = 30,
32
        // The maximum number of connections allowed per route ( e.g. search URI endpoint )
33
        maxConnectionsPerRoute = 10,
34
        // The maxium number of connections, in total for all Elasticsearch requests
35
        maxConnections         = getSystemSetting( "ELASTICSEARCH_MAX_CONNECTIONS", 100 ),
36
        // Read timeout - the read timeout in milliseconds
37
        readTimeout            = getSystemSetting( "ELASTICSEARCH_READ_TIMEOUT", 3000 ),
38
        // Connection timeout - timeout attempts to connect to elasticsearch after this timeout
39
        connectionTimeout      = getSystemSetting( "ELASTICSEARCH_CONNECT_TIMEOUT", 3000 )
40
    }
41
};
Copied!
At the current time only the REST-based [Hyper] native client is available. Support is in development for a socket based-client. For most applications, however the REST-based native client will be a good fit.
Configuration via Environment Variables
Since the default settings will read from environment variables if they exist, we can easily configure cbElasticsearch from a .env file:
1
# .env
2
​
3
# Configure elasticsearch connection
4
ELASTICSEARCH_HOST=localhost
5
ELASTICSEARCH_PORT=9222
6
ELASTICSEARCH_PASSWORD=B0xify_3v3ryth1ng
7
​
8
# Configure data storage and retrieval
9
ELASTICSEARCH_INDEX=books
10
ELASTICSEARCH_SHARDS=5
11
ELASTICSEARCH_REPLICAS=0
12
ELASTICSEARCH_MAX_CONNECTIONS=100
13
ELASTICSEARCH_READ_TIMEOUT=3000
14
ELASTICSEARCH_CONNECT_TIMEOUT=3000
Copied!
You will need to read these settings into the coldfusion server upon server start via commandbox-dotenv or some other method.
For security reasons, make sure to add .env to your .gitignore file to avoid committing environment secrets to github/your git server.
Connection to secondary Elasticsearch Clusters
As of the current version, the module conventions only allow for a default connection to one cluster. Multi-cluster native configuration is planned for a future major release, as it will be a breaking change. You may, however, create a separate instance of the client to connect to a different cluster. Since this needs to be accomplished after the module is loaded, the easiest way to do this is to create an application-specific module which is dedicated to connecting to that cluster. A major caveat, at this time, however is that native CRUD methods in the Document, SearchBuilder, IndexBuilder, and ElasticsearchAppender components will not work, as they are hard-wired to connect to the main client. As such, execution will need to be performed through the separate client instance. If you wish to use the secondary cluster for logging, a new Appender will also need to be created.
Below is an example of creating a secondary client connection to an alternate cluster.
1.First create a new application module
1
box coldbox create module name=SecondaryCluster directory=modules_app dependencies=cbElasticsearch
Copied!
Note: the above command will also create views, models and handlers directories. These can be removed as they will not be used.
1.Once your module is created, open up the ModuleConfig.cfc and add cbElasticsearch to this.dependencies
2.Now change the settings object in the configure() method to use your new configuration. Note that we have omitted the client key. We do this in order to prevent usage of member functions in the internal objects, by ensuring an error is thrown if we attempt to invoke them. All transactions need to pass through the client.
1
settings = {
2
    versionTarget = '7.0.0',
3
    hosts = [
4
        //In this example, our secondary is on the same server, different port
5
        {
6
            serverProtocol: "http",
7
            serverName: "elasticsearch-cluster2",
8
            serverPort: 9200
9
        }
10
    ],
11
    // keep these credentials, but leave blank
12
    defaultCredentials = {
13
        "username" : "",
14
        "password" : ""
15
    },
16
    defaultIndex           = "otherData",
17
    // The default number of shards to use when creating an index
18
    defaultIndexShards     = getSystemSetting( "ELASTICSEARCH_SHARDS", 5 ),
19
    // The default number of index replicas to create
20
    defaultIndexReplicas   = getSystemSetting( "ELASTICSEARCH_REPLICAS", 0 ),
21
    // Whether to use separate threads for client transactions
22
    multiThreaded          = true,
23
    // The maximum amount of time to wait until releasing a connection (in seconds)
24
    maxConnectionIdleTime = 30,
25
    // The maximum number of connections allowed per route ( e.g. search URI endpoint )
26
    maxConnectionsPerRoute = 10,
27
    // The maxium number of connectsion, in total for all Elasticsearch requests
28
    maxConnections         = getSystemSetting( "ELASTICSEARCH_MAX_CONNECTIONS", 100 ),
29
    // Read timeout - the read timeout in milliseconds
30
    readTimeout            = getSystemSetting( "ELASTICSEARCH_READ_TIMEOUT", 3000 ),
31
    // Connection timeout - timeout attempts to connect to elasticsearch after this timeout
32
    connectionTimeout      = getSystemSetting( "ELASTICSEARCH_CONNECT_TIMEOUT", 3000 )
33
};
Copied!
1.Now that we have our settings in place, add our new bindings to the internals onLoad method
1
// map a new singleton instance of the config client
2
binder.map( "[email protected]" )
3
                .to( 'cbelasticsearch.models.Config' )
4
                .threadSafe()
5
                .asSingleton();
6
​
7
var secondaryConfig = wirebox.getInstance( "[email protected]" );
8
​
9
// override the module-injected config struct to our new configuration
10
// note that we need a full config structure passed in as an override to the coldbox settings
11
secondaryConfig.setConfigStruct( settings );
12
​
13
// note that we are using the native JEST client rather than [email protected]
14
binder.map( "[email protected]" )
15
                        .to( "cbElasticsearch.models.JestClient" )
16
                        .initWith( configuration=secondaryConfig )
17
                        .threadSafe()
18
                        .asSingleton();
Copied!
1.After you have created your bindings, make sure you add a closing routine in your onUnload method for the client when the module is unloaded ( e.g. during a framework reinit ):
1
// Close all active pool connections - necessary for native driver implementation
2
if( wirebox.containsInstance( "[email protected]" ) ){
3
    wirebox.getInstance( "[email protected]" ).close();
4
}
Copied!
Now you may perform a search, considering the caveat that the search must now be executed through the client:
1
var searchBuilder = getInstance( "[email protected]" ).new( "myOtherIndex" );
2
searchBuilder.term( "foo", "bar" );
3
​
4
var searchResult = getInstance( "[email protected]" ).executeSearch( searchBuilder );
Copied!
Document saves, retreivals, and deletions would need to be routed through the client, as well, rather than using the save() function:
1
var newDocument = getInstance( "[email protected]" ).new( { "id" : createUUID(), "foo" : "bar" } );
2
getInstance( "[email protected]" ).save( newDocument );
3
​
4
​
5
var existingDocument = getInstance( "[email protected]" ).get( newDocument.getId() );
6
getInstance( "[email protected]" ).delete( existingDocument );
Copied!
As you can see, connecting to a secondary Elasticsearch cluster, while not as fluent, is workable. Version 2.0 of this module has multi-cluster support planned via the native configuration.
Tests
To run the test suite you need a running instance of ElasticSearch. We have provided a docker-compose.yml file in the root of the repo to make this easy as possible. Run docker-compose up --build ( omit the --build after the first startup ) in the root of the project and open http://localhost:8080/tests/runner.cfm to run the tests.
If you would prefer to set this up yourself, make sure you start this app with the correct environment variables set:
1
ELASTICSEARCH_PROTOCOL=http
2
ELASTICSEARCH_HOST=127.0.0.1
3
ELASTICSEARCH_PORT=9200
Copied!
Introduction
Indexes
Last modified 11mo ago
Copy link
Edit on GitHub
Contents
Configuration via Environment Variables
Connection to secondary Elasticsearch Clusters
Tests