Secondary Cluster
Learn how to connect to a secondary Elasticsearch cluster using CBElasticsearch
As of the current version, the module conventions only allow for a default connection to one cluster. Multi-cluster native configuration is planned for a future major release, as it will be a breaking change. You may, however, create a separate instance of the client to connect to a different cluster. Since this needs to be accomplished after the module is loaded, the easiest way to do this is to create an application-specific module which is dedicated to connecting to that cluster. A major caveat, at this time, however is that native CRUD methods in the Document, SearchBuilder, IndexBuilder, and ElasticsearchAppender components will not work, as they are hard-wired to connect to the main client. As such, execution will need to be performed through the separate client instance. If you wish to use the secondary cluster for logging, a new Appender will also need to be created.
Below is an example of creating a secondary client connection to an alternate cluster.
  1. 1.
    First create a new application module
box coldbox create module name=SecondaryCluster directory=modules_app dependencies=cbElasticsearch
Note: the above command will also create views, models and handlers directories. These can be removed as they will not be used.
  1. 1.
    Once your module is created, open up the ModuleConfig.cfc and add cbElasticsearch to this.dependencies
  2. 2.
    Now change the settings object in the configure() method to use your new configuration. Note that we have omitted the client key. We do this in order to prevent usage of member functions in the internal objects, by ensuring an error is thrown if we attempt to invoke them. All transactions need to pass through the client.
settings = {
versionTarget = '7.0.0',
hosts = [
//In this example, our secondary is on the same server, different port
{
serverProtocol: "http",
serverName: "elasticsearch-cluster2",
serverPort: 9200
}
],
// keep these credentials, but leave blank
defaultCredentials = {
"username" : "",
"password" : ""
},
defaultIndex = "otherData",
// The default number of shards to use when creating an index
defaultIndexShards = getSystemSetting( "ELASTICSEARCH_SHARDS", 5 ),
// The default number of index replicas to create
defaultIndexReplicas = getSystemSetting( "ELASTICSEARCH_REPLICAS", 0 ),
// Whether to use separate threads for client transactions
multiThreaded = true,
// The maximum amount of time to wait until releasing a connection (in seconds)
maxConnectionIdleTime = 30,
// The maximum number of connections allowed per route ( e.g. search URI endpoint )
maxConnectionsPerRoute = 10,
// The maxium number of connectsion, in total for all Elasticsearch requests
maxConnections = getSystemSetting( "ELASTICSEARCH_MAX_CONNECTIONS", 100 ),
// Read timeout - the read timeout in milliseconds
readTimeout = getSystemSetting( "ELASTICSEARCH_READ_TIMEOUT", 3000 ),
// Connection timeout - timeout attempts to connect to elasticsearch after this timeout
connectionTimeout = getSystemSetting( "ELASTICSEARCH_CONNECT_TIMEOUT", 3000 )
};
  1. 1.
    Now that we have our settings in place, add our new bindings to the internals onLoad method
// map a new singleton instance of the config client
binder.map( "[email protected]" )
.to( 'cbelasticsearch.models.Config' )
.threadSafe()
.asSingleton();
var secondaryConfig = wirebox.getInstance( "[email protected]" );
// override the module-injected config struct to our new configuration
// note that we need a full config structure passed in as an override to the coldbox settings
secondaryConfig.setConfigStruct( settings );
// note that we are using the native JEST client rather than [email protected]
binder.map( "[email protected]" )
.to( "cbElasticsearch.models.JestClient" )
.initWith( configuration=secondaryConfig )
.threadSafe()
.asSingleton();
  1. 1.
    After you have created your bindings, make sure you add a closing routine in your onUnload method for the client when the module is unloaded ( e.g. during a framework reinit ):
// Close all active pool connections - necessary for native driver implementation
if( wirebox.containsInstance( "[email protected]" ) ){
wirebox.getInstance( "[email protected]" ).close();
}
Now you may perform a search, considering the caveat that the search must now be executed through the client:
var searchBuilder = getInstance( "[email protected]" ).new( "myOtherIndex" );
searchBuilder.term( "foo", "bar" );
var searchResult = getInstance( "[email protected]" ).executeSearch( searchBuilder );
Document saves, retrievals, and deletions would need to be routed through the client, as well, rather than using the save() function:
var newDocument = getInstance( "[email protected]" ).new( { "id" : createUUID(), "foo" : "bar" } );
getInstance( "[email protected]" ).save( newDocument );
var existingDocument = getInstance( "[email protected]" ).get( newDocument.getId() );
getInstance( "[email protected]" ).delete( existingDocument );
As you can see, connecting to a secondary Elasticsearch cluster, while not as fluent, is workable. Version 2.0 of this module has multi-cluster support planned via the native configuration.
Copy link
Edit on GitHub