Once you have installed the module, you may add a custom configuration, specific to your environment, by adding an cbElasticsearch
configuration object to your moduleSettings
inside your Coldbox.cfc
configuration file.
By default the following are in place, without additional configuration:
moduleSettings = {"cbElasticsearch" = {//The native client Wirebox DSL for the transport clientclient="[email protected]",// The default hosts - an array of host connections// - REST-based clients (e.g. JEST): round robin connections will be used// - Socket-based clients (e.g. Transport): cluster-aware routing usedversionTarget = getSystemSetting( "ELASTICSEARCH_VERSION", '' ),hosts = [//The default connection is made to http://127.0.0.1:9200{serverProtocol: getSystemSetting( "ELASTICSEARCH_PROTOCOL", "http" ),serverName: getSystemSetting( "ELASTICSEARCH_HOST", "127.0.0.1" ),serverPort: getSystemSetting( "ELASTICSEARCH_PORT", 9200 )}],// The default credentials for access, if any - may also be overridden when searching index collectionsdefaultCredentials = {"username" : getSystemSetting( "ELASTICSEARCH_USERNAME", "" ),"password" : getSystemSetting( "ELASTICSEARCH_PASSWORD", "" )},// The default indexdefaultIndex = getSystemSetting( "ELASTICSEARCH_INDEX", "cbElasticsearch" ),// The default number of shards to use when creating an indexdefaultIndexShards = getSystemSetting( "ELASTICSEARCH_SHARDS", 5 ),// The default number of index replicas to createdefaultIndexReplicas = getSystemSetting( "ELASTICSEARCH_REPLICAS", 0 ),// Whether to use separate threads for client transactionsmultiThreaded = true,// The maximum amount of time to wait until releasing a connection (in seconds)maxConnectionIdleTime = 30,// The maximum number of connections allowed per route ( e.g. search URI endpoint )maxConnectionsPerRoute = 10,// The maxium number of connections, in total for all Elasticsearch requestsmaxConnections = getSystemSetting( "ELASTICSEARCH_MAX_CONNECTIONS", 100 ),// Read timeout - the read timeout in millisecondsreadTimeout = getSystemSetting( "ELASTICSEARCH_READ_TIMEOUT", 3000 ),// Connection timeout - timeout attempts to connect to elasticsearch after this timeoutconnectionTimeout = getSystemSetting( "ELASTICSEARCH_CONNECT_TIMEOUT", 3000 )}};
At the current time only the REST-based [Hyper] native client is available. Support is in development for a socket based-client. For most applications, however the REST-based native client will be a good fit.
Since the default settings will read from environment variables if they exist, we can easily configure cbElasticsearch from a .env
file:
# .env​# Configure elasticsearch connectionELASTICSEARCH_HOST=localhostELASTICSEARCH_PORT=9222ELASTICSEARCH_PASSWORD=B0xify_3v3ryth1ng​# Configure data storage and retrievalELASTICSEARCH_INDEX=booksELASTICSEARCH_SHARDS=5ELASTICSEARCH_REPLICAS=0ELASTICSEARCH_MAX_CONNECTIONS=100ELASTICSEARCH_READ_TIMEOUT=3000ELASTICSEARCH_CONNECT_TIMEOUT=3000
You will need to read these settings into the coldfusion server upon server start via commandbox-dotenv
or some other method.
For security reasons, make sure to add .env
to your .gitignore
file to avoid committing environment secrets to github/your git server.
As of the current version, the module conventions only allow for a default connection to one cluster. Multi-cluster native configuration is planned for a future major release, as it will be a breaking change. You may, however, create a separate instance of the client to connect to a different cluster. Since this needs to be accomplished after the module is loaded, the easiest way to do this is to create an application-specific module which is dedicated to connecting to that cluster. A major caveat, at this time, however is that native CRUD methods in the Document
, SearchBuilder
, IndexBuilder
, and ElasticsearchAppender
components will not work, as they are hard-wired to connect to the main client. As such, execution will need to be performed through the separate client instance. If you wish to use the secondary cluster for logging, a new Appender will also need to be created.
Below is an example of creating a secondary client connection to an alternate cluster.
First create a new application module
box coldbox create module name=SecondaryCluster directory=modules_app dependencies=cbElasticsearch
Note: the above command will also create views
, models
and handlers
directories. These can be removed as they will not be used.
Once your module is created, open up the ModuleConfig.cfc
and add cbElasticsearch
to this.dependencies
Now change the settings
object in the configure()
method to use your new configuration. Note that we have omitted the client
key. We do this in order to prevent usage of member functions in the internal objects, by ensuring an error is thrown if we attempt to invoke them. All transactions need to pass through the client.
settings = {versionTarget = '7.0.0',hosts = [//In this example, our secondary is on the same server, different port{serverProtocol: "http",serverName: "elasticsearch-cluster2",serverPort: 9200}],// keep these credentials, but leave blankdefaultCredentials = {"username" : "","password" : ""},defaultIndex = "otherData",// The default number of shards to use when creating an indexdefaultIndexShards = getSystemSetting( "ELASTICSEARCH_SHARDS", 5 ),// The default number of index replicas to createdefaultIndexReplicas = getSystemSetting( "ELASTICSEARCH_REPLICAS", 0 ),// Whether to use separate threads for client transactionsmultiThreaded = true,// The maximum amount of time to wait until releasing a connection (in seconds)maxConnectionIdleTime = 30,// The maximum number of connections allowed per route ( e.g. search URI endpoint )maxConnectionsPerRoute = 10,// The maxium number of connectsion, in total for all Elasticsearch requestsmaxConnections = getSystemSetting( "ELASTICSEARCH_MAX_CONNECTIONS", 100 ),// Read timeout - the read timeout in millisecondsreadTimeout = getSystemSetting( "ELASTICSEARCH_READ_TIMEOUT", 3000 ),// Connection timeout - timeout attempts to connect to elasticsearch after this timeoutconnectionTimeout = getSystemSetting( "ELASTICSEARCH_CONNECT_TIMEOUT", 3000 )};
Now that we have our settings in place, add our new bindings to the internals onLoad
method
// map a new singleton instance of the config clientbinder.map( "[email protected]" ).to( 'cbelasticsearch.models.Config' ).threadSafe().asSingleton();​var secondaryConfig = wirebox.getInstance( "[email protected]" );​// override the module-injected config struct to our new configuration// note that we need a full config structure passed in as an override to the coldbox settingssecondaryConfig.setConfigStruct( settings );​// note that we are using the native JEST client rather than [email protected]binder.map( "[email protected]" ).to( "cbElasticsearch.models.JestClient" ).initWith( configuration=secondaryConfig ).threadSafe().asSingleton();
After you have created your bindings, make sure you add a closing routine in your onUnload
method for the client when the module is unloaded ( e.g. during a framework reinit ):
// Close all active pool connections - necessary for native driver implementationif( wirebox.containsInstance( "[email protected]" ) ){wirebox.getInstance( "[email protected]" ).close();}
Now you may perform a search, considering the caveat that the search must now be executed through the client:
var searchBuilder = getInstance( "[email protected]" ).new( "myOtherIndex" );searchBuilder.term( "foo", "bar" );​var searchResult = getInstance( "[email protected]" ).executeSearch( searchBuilder );
Document saves, retreivals, and deletions would need to be routed through the client, as well, rather than using the save()
function:
var newDocument = getInstance( "[email protected]" ).new( { "id" : createUUID(), "foo" : "bar" } );getInstance( "[email protected]" ).save( newDocument );​​var existingDocument = getInstance( "[email protected]" ).get( newDocument.getId() );getInstance( "[email protected]" ).delete( existingDocument );
As you can see, connecting to a secondary Elasticsearch cluster, while not as fluent, is workable. Version 2.0 of this module has multi-cluster support planned via the native configuration.
To run the test suite you need a running instance of ElasticSearch. We have provided a docker-compose.yml
file in the root of the repo to make this easy as possible. Run docker-compose up --build
( omit the --build
after the first startup ) in the root of the project and open http://localhost:8080/tests/runner.cfm
to run the tests.
If you would prefer to set this up yourself, make sure you start this app with the correct environment variables set:
ELASTICSEARCH_PROTOCOL=httpELASTICSEARCH_HOST=127.0.0.1ELASTICSEARCH_PORT=9200