Endeca | Removing Endeca application from Workbench

Issue : In the case when you try to remove Endeca application using runcommand (runcommand --remove-app). If workbench is down at that point of time. Then this application won't be removed from workbench.

In this case runcommand throws below exception.

=============================================================================
Caused by org.apache.http.conn.HttpHostConnectException
org.apache.http.impl.conn.HttpClientConnectionOperator connect - Connect to JagdevSingh-PC:8006 [JagdevSingh-PC/127.0.0.1] failed: Connection refused: connect
Caused by java.net.ConnectException
java.net.DualStackPlainSocketImpl connect0 - Connection refused: connect
=============================================================================

In this case if you start workbench your application site will display there.

Endeca Application

Solution : Shutdown workbench, clean workbench workspace folder and restart it.

Endeca Application Removed

ATG 11.1 | Create Endeca application based on CRS

Sometime you need to create Endeca application based on CRS.

Follow below steps as command line log to create new application [Here Store application created].

CRS deploy.xml location : <CRS Installation Folder>\Store\Storefront\deploy\deploy.xml
=========================================================================
C:\Endeca\ToolsAndFrameworks\11.1.0\deployment_template\bin>deploy.bat --app C:\ATG\ATG11.1\CommerceReferenceStore\Store\Storefront\deploy\deploy.xml

 ------------------------------------------------------------------------------
   Found a supported version of Platform Services installed in directory
   C:\Endeca\PlatformServices\11.1.0. If the location is incorrect, type 'Q' to
   quit and adjust your ENDECA_ROOT environment variable. Press enter to
   continue with these settings.

   Continue?
 ------------------------------------------------------------------------------
     Deployment Template installation script.

     This script creates the directory structure for your
     deployment and installs configuration files and scripts
     into the directory structure.

07/06/2016 17:02:47 [deploy.pl] INFO:  Starting deployment template installation.

 ------------------------------------------------------------------------------
   The following app modules were specified on the command line argument:

   C:\ATG\ATG11.1\CommerceReferenceStore\Store\Storefront\deploy\deploy.xml


   Modules may be deployed standalone or on top of a base deployment type. Do
   you want to install a base deployment with the specified module(s)?

   Install base deployment? [Y/N]:
Y
07/06/2016 17:02:51 [AppDescriptorReader] INFO:  Parsing application descriptor
   file
   C:\ATG\ATG11.1\CommerceReferenceStore\Store\Storefront\deploy\deploy.xml.
07/06/2016 17:02:51 [AppDescriptorReader] INFO:  Parsing application descriptor
   file
   C:\Endeca\ToolsAndFrameworks\11.1.0\deployment_template\app-templates\base_descriptor.xml.

 ------------------------------------------------------------------------------
   Enter a short name for your application.

   Note: The name must conform to this regular expression: ^[a-zA-Z0-9]+$
    [default: CRS]

   Choosing a different application name may require additional configuration on
   your web application.

   Application name:
Store

 ------------------------------------------------------------------------------
   Specify the path into which the application will be deployed. The specified
   directory must exist and cannot contain spaces.

   For example, to deploy into c:\apps\Store, specify the path as c:\apps.

   Deployment directory:
C:\Endeca\apps

 ------------------------------------------------------------------------------
   Specify the port on which the Endeca Application Controller is running. This
   is configured in the server.xml file in the workspace of the Endeca software
   install and should be the same for all applications deployed in this
   environment. Ports must be in the range 1024-65535 [default: 8888].

   EAC port:

07/06/2016 17:03:26 [deploy.pl] INFO:  Deploying application into
   C:\Endeca\apps\Store

 ------------------------------------------------------------------------------
   Enter the port where workbench is running. [Default: 8006]

 ------------------------------------------------------------------------------
   Enter the port to use for Dgraph1. [Default: 15000]


 ------------------------------------------------------------------------------
   Enter the port to use for AuthoringDgraph. [Default: 15002]


 ------------------------------------------------------------------------------
   Enter the port to use for LogServer. [Default: 15010]

 ------------------------------------------------------------------------------
   Please enter the CAS install directory using Unix-style forward slashes,
   including the version number (e.g. C:/Endeca/CAS/11.1.0).
C:/Endeca/CAS/11.1.0

 ------------------------------------------------------------------------------
   Please enter the version for CAS jar files (or hit 'enter' to use the
   default).  [Default: 11.1.0]

 ------------------------------------------------------------------------------
   Please enter the hostname where CAS is running (or hit 'enter' to use the
   default). [Default: localhost]

 ------------------------------------------------------------------------------
   Please enter the port where CAS is running (or hit 'enter' to use the
   default). [Default: 8500]

 ------------------------------------------------------------------------------
   Please enter the language code to use (or hit 'enter' to use the default).
   [Default: en]

 ------------------------------------------------------------------------------
   Please enter the fully-qualified Workbench hostname, including domain.
localhost

 ------------------------------------------------------------------------------
   Please enter the production application server hostname. [Default: localhost]


 ------------------------------------------------------------------------------
   Please enter the production application server port number, Typical values
   are 7003 for WebLogic, 8080 for JBoss and Tomcat, and 9080 for WebSphere.
7003

 ------------------------------------------------------------------------------
   Please enter the preview hostname. [Default: localhost]

 ------------------------------------------------------------------------------
   Please enter the preview port number. Typical values are 7003 for WebLogic,
   8080 for JBoss and Tomcat, and 9080 for WebSphere.
7003

 ------------------------------------------------------------------------------
   Please enter the context root of the preview application. [Default: crs]
Store

 ------------------------------------------------------------------------------
   Enter the hostname of the user segment server. Oracle recommends using the
   Publishing server for this purpose. If your environment does not have a
   Publishing server, enter the Production server host name and refer to the
   Guided Search Integration Guide for additional configuration requirements.
   [Default: localhost]


 ------------------------------------------------------------------------------
   Enter the HTTP port of the user segment server. Oracle recommends using the
   Publishing server for this purpose. If your environment does not have a
   Publishing server, enter the Production server HTTP port and refer to the
   Guided Search Integration Guide for additional configuration requirements.
   [Default: 8080]

 ------------------------------------------------------------------------------
   Specify the path for the Oracle Wallet jps-config.xml file. [Default:
   C:\Endeca\ToolsAndFrameworks\11.1.0\server\workspace\credential_store\jps-config.xml]

 ------------------------------------------------------------------------------
   Which directory will application configuration archives be exported to.
   [Default: C:\Endeca\Apps\CRS\data\workbench\application_export_archive]
C:\Endeca\apps\Store\data\workbench\application_export_archive

 ------------------------------------------------------------------------------
   Specify the port on which Oracle Endeca Workbench is running: [Default: 8006]


 ------------------------------------------------------------------------------
   Specify the port to use for the Live Dgraph: [Default: 15000]


 ------------------------------------------------------------------------------
   Specify the port to use for the Authoring Dgraph: [Default: 15002]


 ------------------------------------------------------------------------------
   Specify the port to use for the Log Server Component: [Default: 15010]


 ------------------------------------------------------------------------------
   Specify the path for the Oracle Wallet jps-config.xml file: [Default:
   C:\Endeca\ToolsAndFrameworks\11.1.0\server\workspace\credential_store\jps-config.xml]

 ------------------------------------------------------------------------------
   Specify the path for the application configuration archives to be exported
   to: [Default:
   C:\Endeca\ToolsAndFrameworks\11.1.0\server\workspace\state\repository]

07/06/2016 17:08:13 [AppDescriptorReader] INFO:  Parsing application descriptor
   file
   C:\Endeca\ToolsAndFrameworks\11.1.0\deployment_template\app-templates\base_descriptor.xml.
07/06/2016 17:08:13 [deploy.pl] INFO:  Processing install with id 'Dgraph'
07/06/2016 17:08:15 [AppDescriptorReader] INFO:  Parsing application descriptor
   file
   C:\ATG\ATG11.1\CommerceReferenceStore\Store\Storefront\deploy\deploy.xml.
07/06/2016 17:08:15 [deploy.pl] INFO:  Processing install with id 'CRSApp'
07/06/2016 17:08:23 [deploy.pl] INFO:  Application successfully deployed.

C:\Endeca\ToolsAndFrameworks\11.1.0\deployment_template\bin>

=========================================================================
This will create application and supported directories. Next you need to initialize the application in order to provision it. Once application is provisioned you can proceed with indexing and other activities. Optionally you can export CRS record store content , dimension value id manager data and import in new application. In this case you also need to export index-config for ATG user and import in new application.

Endeca Application Initialization error

After creating new endeca application successfully. Endeca application initialization failed with below error.
=========================================================================

SEVERE: Caught an exception while invoking method 'run' on object 'InitialSetup'. Releasing locks.

java.lang.reflect.InvocationTargetException

  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

  at java.lang.reflect.Method.invoke(Method.java:606)

  at com.endeca.soleng.eac.toolkit.Controller.invokeRequestedMethod(Controller.java:931)

  at com.endeca.soleng.eac.toolkit.Controller.execute(Controller.java:269)

  at com.endeca.soleng.eac.toolkit.Controller.main(Controller.java:137)

Caused by: com.endeca.soleng.eac.toolkit.exception.AppControlException: Unknown error executing a BeanShell script.

  at com.endeca.soleng.eac.toolkit.script.Script.runBeanShellScript(Script.java:184)

  at com.endeca.soleng.eac.toolkit.script.Script.run(Script.java:127)

  ... 7 more

Caused by: Sourced file: inline evaluation of: ``             IFCR.provisionSite();     CAS.importDimensionValueIdMappings("Store-d . . . '' : Error in method invocation: Method importDimensionValueIdMappings( java.lang.String, java.lang.String ) not found in class'com.endeca.soleng.eac.toolkit.component.CustomComponent' : at Line: 4 : in file: inline evaluation of: ``             IFCR.provisionSite();     CAS.importDimensionValueIdMappings("Store-d . . . '' : CAS .importDimensionValueIdMappings ( "Store-dimension-value-id-manager" , InitialSetup .getWorkingDir ( ) + "/test_data/initial_dval_id_mappings.csv" )

  at bsh.BSHMethodInvocation.eval(Unknown Source)

  at bsh.BSHPrimaryExpression.eval(Unknown Source)

  at bsh.BSHPrimaryExpression.eval(Unknown Source)

=========================================================================

Below is the error in application log file (MyStore.0.0.log) 

WARNING: Error parsing 'class' attribute:com.endeca.eac.toolkit.component.cas.ContentAcquisitionServerComponent

This attribute is in CAS custom component definition (DataIngest.xml).

Solution :  Add casStubs.jar from CAS_ROOT\lib\cas-dt\ to classpath in runcommand script.

Below is example from runcomman.bat file.

set CLASSPATH=%CLASSPATH%;C:\Endeca\CAS\11.1.0\lib\cas-dt\casStubs.jar

Creating Endeca Application For Integration with ATG

In the case you want to integrate Endeca with ATG. You must have Endeca application which can be integrate with ATG. Here I am going to explain how to create Endeca application which can be integrated with product catalog systems like ATG.

For product catalog applications Endeca utilizes the Product Catalog Deployment Template from Discover reference application. This is available after installing ToolsAndFramwork.

This is located at :  \ToolsAndFrameworks\<version>\reference\discover-data-catalog-integration

There are two ways to deploy Product Catalog application.

1. Interactive Deployment : Here you need to respond command prompt during deployment process.
2. Automated Deployment : Here you can specify command prompt values in file. Then no need to interact with command prompt.

Here I am going to use interactive mode.

1. Navigate to C:\Endeca\ToolsAndFrameworks\3.1.2\deployment_template\bin.
2. Invoke deploy script as follows.

deploy.bat --app C:\Endeca\ToolsAndFrameworks\3.1.2\reference\discover-data-catalog-integration\deploy.xml

Accept most defaults. Choose “Y” to install a base deployment. Specify Store (You can choose name)as the application name. Specify the path to CAS path using forward slashes (UNIX style). Specify the appropriate ISO language code, for example, en for English. Here is command line out put for this script.

=============================================================================  

C:\Endeca\ToolsAndFrameworks\3.1.2\deployment_template\bin>deploy.bat --app C:\Endeca\ToolsAndFrameworks\3.1.2\reference\discover-data-catalog-integration\deploy.xml

 ------------------------------------------------------------------------------
   Found version 6.1 of the Endeca IAP installed in directory
   C:\Endeca\PlatformServices\6.1.3. If either the version or location are
   incorrect, type 'Q' to quit and adjust your ENDECA_ROOT environment variable.
   Press enter to continue with these settings.

   Continue?


 ------------------------------------------------------------------------------
     Deployment Template installation script.

     This script creates the directory structure for your
     deployment and installs configuration files and scripts
     into the directory structure.

05/24/2015 13:36:18 [deploy.pl] INFO:  Starting deployment template
   installation.

 ------------------------------------------------------------------------------
   The following app modules were specified on the command line argument:


   C:\Endeca\ToolsAndFrameworks\3.1.2\reference\discover-data-catalog-integration\deploy.xml


   Modules may be deployed standalone or on top of a base deployment type. Do
   you want to install a base deployment with the specified module(s)?

   Install base deployment? [Y/N]:
Y
05/24/2015 13:36:22 [AppDescriptorReader] INFO:  Parsing application descriptor
   file
   C:\Endeca\ToolsAndFrameworks\3.1.2\reference\discover-data-catalog-integration\deploy.xml.
05/24/2015 13:36:22 [AppDescriptorReader] INFO:  Parsing application descriptor
   file
   C:\Endeca\ToolsAndFrameworks\3.1.2\deployment_template\app-templates\base_descriptor.xml.

 ------------------------------------------------------------------------------
   Enter a short name for your application.

   Note: The name must conform to this regular expression: ^[a-zA-Z0-9]+$
    [default: Discover]

   Choosing a different application name may require additional configuration on
   your web application.

   Application name:
Store

 ------------------------------------------------------------------------------
   Specify the path into which the application will be deployed. The specified
   directory must exist and cannot contain spaces.

   For example, to deploy into c:\apps\Store, specify the path as c:\apps.

   Deployment directory:
C:\Endeca\apps

 ------------------------------------------------------------------------------
   Specify the port on which the Endeca Application Controller is running. This
   is configured in the server.xml file in the workspace of the Endeca software
   install and should be the same for all applications deployed in this
   environment. Ports must be in the range 1024-65535 [default: 8888].

   EAC port:
8888
05/24/2015 13:36:55 [deploy.pl] INFO:  Deploying application into
   C:\Endeca\apps\Store

 ------------------------------------------------------------------------------
   Please enter the CAS install directory using Unix-style forward slashes,
   including the version number (e.g. C:/Endeca/CAS/3.1.1).
C:/Endeca/CAS/3.1.2

 ------------------------------------------------------------------------------
   Please enter the version for CAS jar files (or hit 'enter' to use the
   default).  [Default: 3.1.1]


 ------------------------------------------------------------------------------
   Please enter the hostname where CAS is running (or hit 'enter' to use the
   default). [Default: localhost]


 ------------------------------------------------------------------------------
   Please enter the port where CAS is running (or hit 'enter' to use the
   default). [Default: 8500]


 ------------------------------------------------------------------------------
   Please enter the language code to use (or hit 'enter' to use the default).
   [Default: en]


 ------------------------------------------------------------------------------
   What port is the Workbench running? [Default: 8006]


 ------------------------------------------------------------------------------
   What port should be used for the Live Dgraph? [Default: 15000]


 ------------------------------------------------------------------------------
   What port should be used for the Authoring Dgraph? [Default: 15002]


 ------------------------------------------------------------------------------
   What port should be used for LogServer? [Default: 15010]

05/24/2015 13:37:33 [AppDescriptorReader] INFO:  Parsing application descriptor
   file
   C:\Endeca\ToolsAndFrameworks\3.1.2\deployment_template\app-templates\base_descriptor.xml.
05/24/2015 13:37:33 [deploy.pl] INFO:  Processing install with id 'Dgraph'
05/24/2015 13:37:34 [AppDescriptorReader] INFO:  Parsing application descriptor
   file
   C:\Endeca\ToolsAndFrameworks\3.1.2\reference\discover-data-catalog-integration\deploy.xml.
05/24/2015 13:37:34 [deploy.pl] INFO:  Processing install with id
   'DefaultTemplateManagerApp'
05/24/2015 13:37:35 [deploy.pl] INFO:  Application successfully deployed.

=============================================================================  

    3. Navigate to control directory of the newly created application.Then run initialize_services script.
     This will create required record stores and provision this application. Below is the command line output of this script.
=============================================================================
C:\Endeca\apps\Store\control>initialize_services.bat
Setting EAC provisioning and performing initial setup...
[05.24.15 13:47:12] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
[05.24.15 13:47:12] INFO: Setting definition for application 'Store'.
[05.24.15 13:47:13] INFO: Setting definition for host 'AuthoringMDEXHost'.
[05.24.15 13:47:13] INFO: Setting definition for host 'LiveMDEXHostA'.
[05.24.15 13:47:13] INFO: Setting definition for host 'ReportGenerationHost'.
[05.24.15 13:47:13] INFO: Setting definition for host 'WorkbenchHost'.
[05.24.15 13:47:13] INFO: Setting definition for host 'ITLHost'.
[05.24.15 13:47:13] INFO: Setting definition for component 'AuthoringDgraph'.
[05.24.15 13:47:13] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_-data-dgidx-output'.
[05.24.15 13:47:14] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_-data-partials-forge-output'.
[05.24.15 13:47:16] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_-data-partials-cumulative-partials'.
[05.24.15 13:47:17] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_-data-workbench-dgraph-config'.
[05.24.15 13:47:18] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_-data-dgraphs-local-dgraph-input'.
[05.24.15 13:47:19] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_-data-dgraphs-local-cumulative-partials'.
[05.24.15 13:47:20] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_-data-dgraphs-local-dgraph-config'.
[05.24.15 13:47:22] INFO: Setting definition for component 'DgraphA1'.
[05.24.15 13:47:22] INFO: Setting definition for script 'PromoteAuthoringToLive'.
[05.24.15 13:47:22] INFO: Setting definition for custom component 'WorkbenchManager'.
[05.24.15 13:47:22] INFO: Updating provisioning for host 'ITLHost'.
[05.24.15 13:47:22] INFO: Updating definition for host 'ITLHost'.
[05.24.15 13:47:22] INFO: [ITLHost] Starting shell utility 'mkpath_-'.
[05.24.15 13:47:23] INFO: [ITLHost] Starting shell utility 'mkpath_-data-workbench-temp'.
[05.24.15 13:47:24] INFO: [ITLHost] Starting shell utility 'mkpath_-data-processing'.
[05.24.15 13:47:26] INFO: Setting definition for custom component 'IFCR'.
[05.24.15 13:47:26] INFO: Updating provisioning for host 'ITLHost'.
[05.24.15 13:47:26] INFO: Updating definition for host 'ITLHost'.
[05.24.15 13:47:26] INFO: [ITLHost] Starting shell utility 'mkpath_-'.
[05.24.15 13:47:27] INFO: [ITLHost] Starting shell utility 'mkpath_-'.
[05.24.15 13:47:28] INFO: Setting definition for component 'LogServer'.
[05.24.15 13:47:28] INFO: [ReportGenerationHost] Starting shell utility 'mkpath_-reports-input'.
[05.24.15 13:47:30] INFO: Setting definition for script 'DaySoFarReports'.
[05.24.15 13:47:30] INFO: Setting definition for script 'DailyReports'.
[05.24.15 13:47:30] INFO: Setting definition for script 'WeeklyReports'.
[05.24.15 13:47:30] INFO: Setting definition for script 'DaySoFarHtmlReports'.
[05.24.15 13:47:30] INFO: Setting definition for script 'DailyHtmlReports'.
[05.24.15 13:47:30] INFO: Setting definition for script 'WeeklyHtmlReports'.
[05.24.15 13:47:31] INFO: Setting definition for component 'WeeklyReportGenerator'.
[05.24.15 13:47:31] INFO: Setting definition for component 'DailyReportGenerator'.
[05.24.15 13:47:31] INFO: Setting definition for component 'DaySoFarReportGenerator'.
[05.24.15 13:47:31] INFO: Setting definition for component 'WeeklyHtmlReportGenerator'.
[05.24.15 13:47:31] INFO: Setting definition for component 'DailyHtmlReportGenerator'.
[05.24.15 13:47:32] INFO: Setting definition for component 'DaySoFarHtmlReportGenerator'.
[05.24.15 13:47:32] INFO: Setting definition for script 'BaselineUpdate'.
[05.24.15 13:47:32] INFO: Setting definition for script 'PartialUpdate'.
[05.24.15 13:47:32] INFO: Setting definition for component 'ConfigurationGeneratorForge'.
[05.24.15 13:47:32] INFO: Setting definition for component 'Forge'.
[05.24.15 13:47:33] INFO: [ITLHost] Starting shell utility 'mkpath_-data-incoming'.
[05.24.15 13:47:34] INFO: Setting definition for component 'PartialForge'.
[05.24.15 13:47:34] INFO: [ITLHost] Starting shell utility 'mkpath_-data-partials-incoming'.
[05.24.15 13:47:35] INFO: Setting definition for component 'Dgidx'.
[05.24.15 13:47:36] INFO: Definition updated.
[05.24.15 13:47:36] INFO: Provisioning site from prototype...
[05.24.15 13:47:36] INFO: Finished provisioning site from prototype.
Finished updating EAC.
Initializing record stores...
These record stores exist:
NAME                    TYPE                    STATUS
CRS_en_schema                   RecordStore                     RUNNING
CRS_en_dimvals                  RecordStore                     RUNNING
CRS_en_prules                   RecordStore                     RUNNING
CRS_en_data                     RecordStore                     RUNNING
==================================================
Dropping old record stores (errors will occur on non-existent rs):
Error during execution (check log for more info): Component instance not found: Store_en_schema
Error during execution (check log for more info): Component instance not found: Store_en_dimvals
Error during execution (check log for more info): Component instance not found: Store_en_prules
Error during execution (check log for more info): Component instance not found: Store_en_data
==================================================
Creating fresh record stores:
Successfully created component: Store_en_schema
Successfully created component: Store_en_dimvals
Successfully created component: Store_en_prules
Successfully created component: Store_en_data
==================================================
Deploying rs configs:
Successfully set recordstore configuration.
Successfully set recordstore configuration.
Successfully set recordstore configuration.
Successfully set recordstore configuration.
==================================================
Finished initializing record stores
Importing sample content...
[05.24.15 13:48:33] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
[05.24.15 13:48:33] INFO: Definition has not changed.
[05.24.15 13:48:33] INFO: Packaging contents for upload...
[05.24.15 13:48:34] INFO: Finished packaging contents.
[05.24.15 13:48:34] INFO: Uploading contents to: http://JagdevSingh-PC:8006/ifcr/sites/Store/
[05.24.15 13:48:35] INFO: Finished uploading contents.
[05.24.15 13:48:37] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
[05.24.15 13:48:38] INFO: Definition has not changed.
[05.24.15 13:48:38] INFO: Packaging contents for upload...
[05.24.15 13:48:38] INFO: Finished packaging contents.
[05.24.15 13:48:38] INFO: Uploading contents to: http://JagdevSingh-PC:8006/ifcr/sites/Store/content
[05.24.15 13:48:39] INFO: Finished uploading contents.
[05.24.15 13:48:41] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
[05.24.15 13:48:42] INFO: Definition has not changed.
[05.24.15 13:48:42] INFO: Packaging contents for upload...
[05.24.15 13:48:42] INFO: Finished packaging contents.
[05.24.15 13:48:42] INFO: Uploading contents to: http://JagdevSingh-PC:8006/ifcr/sites/Store/pages
[05.24.15 13:48:43] INFO: Finished uploading contents.
Finished importing sample content
Importing media...
[05.24.15 13:48:45] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
[05.24.15 13:48:46] INFO: Definition has not changed.
[05.24.15 13:48:46] INFO: Packaging contents for upload...
[05.24.15 13:48:46] INFO: Finished packaging contents.
[05.24.15 13:48:46] INFO: Uploading contents to: http://JagdevSingh-PC:8006/ifcr/sites/Store/media
[05.24.15 13:48:49] INFO: Finished uploading contents.
Finished importing media
Importing editors configuration...
[05.24.15 13:48:51] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
[05.24.15 13:48:52] INFO: Definition has not changed.
[05.24.15 13:48:52] INFO: Packaging contents for upload...
[05.24.15 13:48:52] INFO: Finished packaging contents.
[05.24.15 13:48:52] INFO: Uploading contents to: http://JagdevSingh-PC:8006/ifcr/sites/Store/configuration/tools/xmgr
[05.24.15 13:48:52] INFO: Finished uploading contents.
Finished importing editors configuration
Importing templates...
Removing existing cartridge templates for Store
Setting new cartridge templates for Store
Finished setting templates
Finished importing templates

C:\Endeca\apps\Store\control>
=============================================================================
At this phase your Endeca application is ready. Next you need to configure ATG application to use this application.

Create And Provision Endeca Application with the Deployment Template

Here are the steps to create an Endeca application from scratch and provision it using deployment template.

Application Creation 
 

      1. Open command line and change directory to ..\...\ToolsAndFrameworks\3.1.2  

          \deployment_template\bin and invoke deploy script. 

           For ex. C:\Endeca\ToolsAndFrameworks\3.1.2\deployment_template\bin\deply.bat

 

      2. Provide configuration parameter/values as script prompts you. First of all it will confirm

          IAP version. Then it will prompt following information.

• Application Name.
• Application Deployment directory.
• EAC port.
• Workbench port.
• Live dgraph port.
• Authoring Dgraph port.
• Log Server port.

Screen  Captures for application creation.

For port values you can use any available port on your machine, or use default one.At the end you will get message that appplication deployed successfully.

Now application creation is done. It is time to provision it. Once application provisioned it will be available for configuration in Endeca Workbench.

To provision an application.

Go to \control directory of your application you created earlier (above steps).

Invoke initialize_services.bat or initialize_services.sh script. This script won't prompt for any values.

Here is the screen capture of application provisioning.

 

 

Resolving Performance Puzzle

It is challenging for beginners to start performance tuning of an application. I was in same situation when got my first performance tuning assignment. Before starting with performance tuning we need to understand the aspects of it. 

In simple words Performance tuning is the improvement of system performance. Most systems will respond to increased load with some degree of decreasing performance. The core of performance tuning is the performance testing.

Here is the list of steps to execute systematic tuning.

1. Assess the issue, get the numbers (to baseline the performance) to measure the performance (Mostly available/defined in Non Functional Requirement [NFR] document).
2.  Measure the performance of system.
3. Identify the bottlenecks (Part of the system that is critical for performance).
4. Modify the system update/remove the bottleneck.
5. Measure the system performance after above modification.
6. In the case modification improves the performance adopt it,otherwise revert the above modification.
7. Repeat from steps 2 to 6 in cycles to improve the performance requirement.

These are high level steps. The most confusing/challenging question comes into mind is. 

Where (which part of application) to start performance testing ?.

Most of the web applications are modularized/divided into 3 major parts.

1. Database.
2. Back-end code (Including third party calls). 
3. Front-end.

Its always better to start from database and end at front-end.

ATG Application Performance Tuning Toolkit

Here is list of tools I am using for application performance analysis and tuning.

 1.   JMeter : The Apache JMeter™ desktop application is open source software, a 100% pure Java application designed to load test functional behavior and measure performance. Apache JMeter may be used to test performance both on static and dynamic resources (Files, Web dynamic languages - PHP, Java, ASP.NET, etc. -, Java Objects, Data Bases and Queries, FTP Servers and more). It can be used to simulate a heavy load on a server, group of servers, network or object to test its strength or to analyze overall performance under different load types. You can use it to make a graphical analysis of performance or to test your server/script/object behavior under heavy concurrent load. 

2.   JvisualVM (Java VisualVM) :  Java VisualVM can be used by Java application developers to troubleshoot applications and to monitor and improve the applications' performance. Java VisualVM can allow developers to generate and analyse heap dumps, track down memory leaks, browse the platform's MBeans and perform operations on those MBeans, perform and monitor garbage collection, and perform lightweight memory and CPU profiling.

3.   TDA - Thread Dump Analyzer : The TDA Thread Dump Analyzer for Java is a small Swing GUI for analyzing Thread Dumps and Heap Information generated by the Sun Java VM.  
 

4.   MAT - Eclipse Memory Analyzer : The Eclipse Memory Analyzer is a fast and feature-rich Java heap analyzer that helps you find memory leaks and reduce memory consumption. Use the Memory Analyzer to analyze productive heap dumps with hundreds of millions of objects, quickly calculate the retained sizes of objects, see who is preventing the Garbage Collector from collecting objects, run a report to automatically extract leak suspects.

5.  Automatic Workload Repository (AWR) in Oracle Database : The Automatic Workload Repository (AWR) collects, processes, and maintains performance statistics for problem detection and self-tuning purposes. This data is both in memory and stored in the database.

The statistics collected and processed by AWR include :

• Object statistics that determine both access and usage statistics of database segments.

• Time model statistics based on time usage for activities, displayed in the V$SYS_TIME_MODEL and V$SESS_TIME_MODEL views.

• Some of the system and session statistics collected in the V$SYSSTAT and V$SESSTAT views SQL statements that are producing the highest load on the system, based on criteria such as elapsed time and CPU time.

• ASH statistics, representing the history of recent sessions activity.

I will update more on performance tuning.....

Introduction to Endeca Application Controller (EAC)

The EAC provides the infrastructure to support Endeca application.Endeca Application Controller (EAC) is a control system used to control, manage,and monitor components in Endeca implementation.

There are two types of EAC instances.

1. EAC Central Server
2. EAC Agent

1. EAC Central Server

One instance of the EAC serves as the EAC Central Server.The EAC Central Server also contains a repository that stores provisioning information — such as, data about the hosts, components, applications and scripts.This instance includes and WSDL enabled public interface.Other applications communicates with EAC using this interface.EAC central server works as an controller of EAC agents.EAC central server communicates with EAC agents using internal web service interface.

2. EAC Agents 

EAC instances other then EAC Central server are EAC agents.These EAC agents instruct host machine for execution of actual task such Forging or Indexing.

There are 3 ways to communicate with EAC.

1. Endeca Workbench.
2. The command line utility, eaccmd.
3. Endeca WSDL-enabled interface(using language that supports web services).Endeca workbench and Endeca deployment template utilize this method.

You can use below scripts to start and stop EAC from commandline.
1. EAC start script.
   $ENDECA_ROOT/tools/server/bin/startup.sh
2. EAC stop script.
   $ENDECA_ROOT/tools/server/bin/shutdown.sh
Note : On windows EAC is created,configured and registered as "Endeca HTTP service".

EAC configuration file
EAC reads configuration from eac.properties file.This file is located in the $ENDECA_CONF/conf directory.

EAC log file.

EAC log file is located in the $ENDECA_CONF/logs directory.You can update EAC logging related configuration in the logging.properties file.This file is located under $ENDECA_CONF/conf.