Saturday, 11 July 2015

ATG REST MVC Overview

Nowadays it is very common for enterprise applications to share data and business logic with other applications. This can be easily achieved using web services. This becomes very tricky when providing omni channel support in E-commerce (accessing site using android or any other native application).

ATG allows developers to create their own webservices along with pre-packages services.

These pre-packages services are available in below modules.
  1. DAS.WebServices
  2. DPS.WebServices
  3. DCS.WebServices 
ATG supports two types of REST webservices (webservices APIs).
  1. Legacy REST API
  2. REST MVC API
 

Here I am going to explain REST MVC.

Steps to create New REST MVC Call
  • Create Actor.
  • Define Actor chain(s) for that actor.
  • Register Actor with ActorChainRestRegistry.
  • Create Bean filter (optional).
Steps to create REST Actor
  1. Create Component of atg.service.actor.ActorChainService class.
  2. Define actor chains for this component (xml configuration).
  3. Point definitionFile property of the component to xml file created in step 2.
Below is the example of Hello World Actor

=======================================================================
#/com/test/web/actor/HelloWorldActor.properties
$class=atg.service.actor.ActorChainService
definitionFile=/com/test/web/actor/helloWorldActor.xml
=======================================================================

Actor Chain definition file (helloWorldActor.xml)
 
=======================================================================
<?xml version="1.0" encoding="UTF-8"?>
<actor-template default-chain-id="sayHello" xsi:noNamespaceSchemaLocation="http://www.atg.com/xsds/actorChain_1.0.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<actor-chain id="sayHello" transaction="TX_SUPPORTS">
       <component id="sayHello" name="/com/test/web/HelloWorld"
           method="sayHello">
      </component>
 </actor-chain>
</actor-template>
=======================================================================

Registering this actor with ActorChainRestRegistry

To register this actor you need to add the actor path with chain id to registeredUrls property of /atg/rest/registry/ActorChainRestRegistry component.

One important thing to remember here is ,that you are registering actor chain not actor. In the case there are more then one chains defined for that actor you need to register each one here. In  other words you can say each chain ID should be registered separately.

By default, no actors are registered.

In below code snippet we are registering sayHello chain. 
=======================================================================
#/atg/rest/registry/ActorChainRestRegistry.properties
registeredUrls=\
         /com/test/web/actor/HelloWorldActor/sayHello

=======================================================================

 ATG REST MVC Supports below Actor Types.
  • Component Actor 
  • Droplet Actor
  • Form Actor
  • JSP Actor
  • Nested Actor
  • Variable Actor 

Filtering in MVC REST

Filtering is used in REST MVC to control the property in the response object. In other words filter is way to configure which properties will be available in the response object. This is to avoid unnecessary data in the response.
 
REST MVC support two types of filters.
  1. Java bean filtering.
  2. Repository item filtering.
Steps to configure filter
  • Layer /atg/dynamo/service/filter/bean/beanFilteringConfiguration.xml 
  • Configure filter in this file
  • Refer this filter in actor chain
  • ATG recommends 3 types of filters
                   Short
                   Summary
                   Detailed


Once filter is defined you can use filter in actor chain using filter-id attribute.

Filter definition example.

======================================================================= 
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE bean-filtering SYSTEM "dynamosystemresource:/atg/dtds/beanfilter/beanFiltering_1.0.dtd">
<bean-filtering>
<repository name="/atg/userprofiling/ProfileAdapterRepository">
   <item-descriptor name="user">
      <filter id="customDetail" default-include="false">
        <property hidden="false" name="firstName"/>     
        <property hidden="false" name="lastName"/>
      </filter>
   </item-descriptor>
</repository>

</bean-filtering>
======================================================================= 

Using filter in actor chain.  

=======================================================================
 <?xml version="1.0" encoding="UTF-8"?>
<actor-template default-chain-id="summary" xsi:noNamespaceSchemaLocation="http://www.atg.com/xsds/actorChain_1.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <actor-chain id="customDetail" transaction="TX_SUPPORTS">
    <component id="profile" name="/atg/userprofiling/Profile" invoke-method-requires-session-confirmation="true" component-var="profile" set-property-requires-session-confirmation="true">
      <output id="profile" filter-id="customDetail" name="profile" value="${profile.dataSource}"/>
    </component>
  </actor-chain>
</actor-template>

=======================================================================  
Note : It is best to define a filter for every object, so that you can control its output. Note that if an object has no filters defined, it will output all properties.

Configure Security in REST MVC 

Once REST call is implemented then It is time to secure it. Security is crucial to avoid unauthorized access. 

Follow below steps to secure rest webservice
  • Create the RuleSetService.
  • Create Access Controller.
  • Add mapping from actor chain to Access Controller in /atg/dynamo/servlet/dafpipeline/AccessControlServlet. 
 
CustomRuleService (Only logged in user can access).

======================================================================= 
#/atg/rest/CustomRuleSetService.properties
$class=atg.targeting.RuleSetService
updatesEnabled=true
rulesFileCheckSeconds=0

# Use must have securityStatus 4 or higher (EXPLICIT-SIGNIN, SECURE-SIGNIN, CERTIFICATE)
ruleSet=<ruleset>\n  <accepts>\n    <rule op\=and tag\="Show">\n      <rule op\=and tag\="Content">\n      </rule>\n      <rule op\=and tag\="Environment">\n        <rule op\=gt>\n          <valueof target\="securityStatus">\n          <valueof constant\="3">\n        </rule>\n      </rule>\n    </rule>\n  </accepts>\n</ruleset>
======================================================================= 

CustomAccessController
 
=======================================================================
#/atg/userprofiling/CustomAccessController.properties
$class=atg.userprofiling.RuleAccessController
enabled=true
# Rules used to determine whether access should be allowed
ruleSetService=/atg/rest/CustomRuleSetService
# URL to redirect to if access is denied
deniedAccessURL=/rest/model/atg/userprofiling/SecurityStatusActor/authenticationRequired
======================================================================= 
 
 AccessControlServlet
 
======================================================================= 
#/atg/dynamo/servlet/dafpipeline/AccessControlServlet.properties
accessControllers=\
   
/com/test/web/actor/HelloWorldActor/sayHello=\
         /atg/userprofiling/CustomAccessController
======================================================================= 
      
ATG REST MVC Key Points
  • Get and Post are supported.
  • Access restriction  by AccessControllerService.
  • Also support implicit objects (session, request).
  • URL syntax http://host:port/rest/model/actor_component/tail.

ATG REST MVC Key Components
  • /atg/rest/Configuration/ 
  • /atg/rest/registry/ActorChainRestRegistry/
  • /atg/dynamo/service/filter/bean/XmlFilterService
  • /atg/dynamo/service/actor/ActorChainValidationService

Saturday, 4 July 2015

Enabling the older IR version in MDEX

Every request in newly created Endeca application was throwing ENEException.

Here is the error from Dgraph log.

=======================================================================

ERROR 06/13/15 14:51:16.438 UTC (1434207076438) DGRAPH {dgraph} Requested output IR version '620' is supported by this release of the software but not enabled. Use the --back_compat flag to enable the older IR version.
WARN 06/13/15 14:51:16.438 UTC (1434207076438) DGRAPH {dgraph} Error processing HTTP exchange 2: Error:[MDEX] Failed to parse URL: '/graph?node=0&offset=0&nbins=10&irversion=620'

=======================================================================

The issue was due to version mismatch in API and running MDEX.

Solution : Enable backward compatibility in MDEX.

This can be done in two ways.

1.  MDEX configuration window in EAC admin console of the workbench.
  • Stop MDEX.
  • Add --back_compat 620 to the argument. Here is configuration snapshot.



  • Start MDEX.
 2.  Update startup argument in MDEX configuration file [DgraphDefaults.xml].

 <arg>--back_compat</arg>
      <arg>620</arg>

Sunday, 21 June 2015

ATG-Endeca indexing reduce logging messages to avoid terminal lock up (Hang/Not responding)

While indexing newly configured application, JBoss terminal hanged. It was not responding at all. The issue was due to high volume of log messages during indexing. Here is solution.........


Explaining the fix for Weblogic , JBoss and IBM WebSphere.

1.   Oracle WebLogic Server

Create a WebLogic filter in $WL_HOME/../user_projects/domains/base-domain-name/config/config.xml.
<log-filter>
  <name>CXFFilter</name>
  <filter-expression>
  ((SUBSYSTEM = org.apache.cxf.interceptor.LoggingOutInterceptor') OR
  (SUBSYSTEM = 'org.apache.cxf.interceptor.LoggingInInterceptor')) AND
  (SEVERITY = 'WARNING')
  </filter-expression>
</log-filter>
In the same file, add configuration to apply the filter. The following example applies the filter to the server log file and to standard output for a server instance named Prod.
<server>
   <name>Prod</name>
   <log>
     <log-file-filter>CXFFilter</log-file-filter>
     <stdout-filter>CXFFilter</stdout-filter>
     <memory-buffer-severity>Debug</memory-buffer-severity>
   </log>
   <listen-port>7103</listen-port>
   <web-server>
     <web-server-log>
       <number-of-files-limited>false</number-of-files-limited>
     </web-server-log>
   </web-server>
   <listen-address></listen-address>
 </server>

 2.   JBoss Enterprise Application Platform

Add the following to jboss-as\server\server-name\conf\jboss-log4j.xml:
<category name="org.apache.cxf.interceptor.LoggingInInterceptor">
  <priority value="WARN"/>
</category>
<category name="org.apache.cxf.interceptor.LoggingOutInterceptor">
  <priority value="WARN"/>
</category>

3.   IBM WebSphere Application Server

Edit the server.xml of the WebSphere application server instance ($WAS_HOME/profiles/AppSrv/config/cells/HostCell/nodes/HostNode/servers/Server/server.xml).
In the traceservice:TraceService tag, add these strings, separated by colons, to the startupTraceSpecification property:
org.apache.cxf.interceptor.LoggingInInterceptor=warning
org.apache.cxf.interceptor.LoggingOutInterceptor=warning
For example
<services xmi:type="traceservice:TraceService"
  xmi:id="TraceService_131/2495363666"
  enable="true" startupTraceSpecification=
  "*=info:org.apache.cxf.interceptor.LoggingInInterceptor=warning:
  org.apache.cxf.interceptor.LoggingOutInterceptor=warning"
  traceOutputType="SPECIFIED_FILE" traceFormat="BASIC">
    <traceLog xmi:id="TraceLog_1312495363666"
      fileName="${SERVER_LOG_ROOT}/trace.log" rolloverSize="20"
      maxNumberOfBackupFiles="5"/>
</services>

Note  : I tried this for JBoss 5.1.0, this is working well. This is very helpful to reduce logging during development (low configuration machine).

Tuesday, 16 June 2015

Creating Endeca Application For Integration with ATG

In the case you want to integrate Endeca with ATG. You must have Endeca application which can be integrate with ATG. Here I am going to explain how to create Endeca application which can be integrated with product catalog systems like ATG.

For product catalog applications Endeca utilizes the Product Catalog Deployment Template from Discover reference application. This is available after installing ToolsAndFramwork.

This is located at :  \ToolsAndFrameworks\<version>\reference\discover-data-catalog-integration

There are two ways to deploy Product Catalog application.
  1. Interactive Deployment : Here you need to respond command prompt during deployment process.
  2. Automated Deployment : Here you can specify command prompt values in file. Then no need to interact with command prompt.
Here I am going to use interactive mode.
  1. Navigate to C:\Endeca\ToolsAndFrameworks\3.1.2\deployment_template\bin.
  2. Invoke deploy script as follows.
deploy.bat --app C:\Endeca\ToolsAndFrameworks\3.1.2\reference\discover-data-catalog-integration\deploy.xml

Accept most defaults. Choose “Y” to install a base deployment. Specify Store (You can choose name)as the application name. Specify the path to CAS path using forward slashes (UNIX style). Specify the appropriate ISO language code, for example, en for English. Here is command line out put for this script.
=============================================================================  
C:\Endeca\ToolsAndFrameworks\3.1.2\deployment_template\bin>deploy.bat --app C:\Endeca\ToolsAndFrameworks\3.1.2\reference\discover-data-catalog-integration\deploy.xml

 ------------------------------------------------------------------------------
   Found version 6.1 of the Endeca IAP installed in directory
   C:\Endeca\PlatformServices\6.1.3. If either the version or location are
   incorrect, type 'Q' to quit and adjust your ENDECA_ROOT environment variable.
   Press enter to continue with these settings.

   Continue?


 ------------------------------------------------------------------------------
     Deployment Template installation script.

     This script creates the directory structure for your
     deployment and installs configuration files and scripts
     into the directory structure.

05/24/2015 13:36:18 [deploy.pl] INFO:  Starting deployment template
   installation.

 ------------------------------------------------------------------------------
   The following app modules were specified on the command line argument:


   C:\Endeca\ToolsAndFrameworks\3.1.2\reference\discover-data-catalog-integration\deploy.xml


   Modules may be deployed standalone or on top of a base deployment type. Do
   you want to install a base deployment with the specified module(s)?

   Install base deployment? [Y/N]:
Y
05/24/2015 13:36:22 [AppDescriptorReader] INFO:  Parsing application descriptor
   file
   C:\Endeca\ToolsAndFrameworks\3.1.2\reference\discover-data-catalog-integration\deploy.xml.
05/24/2015 13:36:22 [AppDescriptorReader] INFO:  Parsing application descriptor
   file
   C:\Endeca\ToolsAndFrameworks\3.1.2\deployment_template\app-templates\base_descriptor.xml.

 ------------------------------------------------------------------------------
   Enter a short name for your application.

   Note: The name must conform to this regular expression: ^[a-zA-Z0-9]+$
    [default: Discover]

   Choosing a different application name may require additional configuration on
   your web application.

   Application name:
Store

 ------------------------------------------------------------------------------
   Specify the path into which the application will be deployed. The specified
   directory must exist and cannot contain spaces.

   For example, to deploy into c:\apps\Store, specify the path as c:\apps.

   Deployment directory:
C:\Endeca\apps

 ------------------------------------------------------------------------------
   Specify the port on which the Endeca Application Controller is running. This
   is configured in the server.xml file in the workspace of the Endeca software
   install and should be the same for all applications deployed in this
   environment. Ports must be in the range 1024-65535 [default: 8888].

   EAC port:
8888
05/24/2015 13:36:55 [deploy.pl] INFO:  Deploying application into
   C:\Endeca\apps\Store

 ------------------------------------------------------------------------------
   Please enter the CAS install directory using Unix-style forward slashes,
   including the version number (e.g. C:/Endeca/CAS/3.1.1).
C:/Endeca/CAS/3.1.2

 ------------------------------------------------------------------------------
   Please enter the version for CAS jar files (or hit 'enter' to use the
   default).  [Default: 3.1.1]


 ------------------------------------------------------------------------------
   Please enter the hostname where CAS is running (or hit 'enter' to use the
   default). [Default: localhost]


 ------------------------------------------------------------------------------
   Please enter the port where CAS is running (or hit 'enter' to use the
   default). [Default: 8500]


 ------------------------------------------------------------------------------
   Please enter the language code to use (or hit 'enter' to use the default).
   [Default: en]


 ------------------------------------------------------------------------------
   What port is the Workbench running? [Default: 8006]


 ------------------------------------------------------------------------------
   What port should be used for the Live Dgraph? [Default: 15000]


 ------------------------------------------------------------------------------
   What port should be used for the Authoring Dgraph? [Default: 15002]


 ------------------------------------------------------------------------------
   What port should be used for LogServer? [Default: 15010]

05/24/2015 13:37:33 [AppDescriptorReader] INFO:  Parsing application descriptor
   file
   C:\Endeca\ToolsAndFrameworks\3.1.2\deployment_template\app-templates\base_descriptor.xml.
05/24/2015 13:37:33 [deploy.pl] INFO:  Processing install with id 'Dgraph'
05/24/2015 13:37:34 [AppDescriptorReader] INFO:  Parsing application descriptor
   file
   C:\Endeca\ToolsAndFrameworks\3.1.2\reference\discover-data-catalog-integration\deploy.xml.
05/24/2015 13:37:34 [deploy.pl] INFO:  Processing install with id
   'DefaultTemplateManagerApp'
05/24/2015 13:37:35 [deploy.pl] INFO:  Application successfully deployed.
=============================================================================  

    3. Navigate to control directory of the newly created application.Then run initialize_services script.
     This will create required record stores and provision this application. Below is the command line output of this script.
=============================================================================
C:\Endeca\apps\Store\control>initialize_services.bat
Setting EAC provisioning and performing initial setup...
[05.24.15 13:47:12] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
[05.24.15 13:47:12] INFO: Setting definition for application 'Store'.
[05.24.15 13:47:13] INFO: Setting definition for host 'AuthoringMDEXHost'.
[05.24.15 13:47:13] INFO: Setting definition for host 'LiveMDEXHostA'.
[05.24.15 13:47:13] INFO: Setting definition for host 'ReportGenerationHost'.
[05.24.15 13:47:13] INFO: Setting definition for host 'WorkbenchHost'.
[05.24.15 13:47:13] INFO: Setting definition for host 'ITLHost'.
[05.24.15 13:47:13] INFO: Setting definition for component 'AuthoringDgraph'.
[05.24.15 13:47:13] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_-data-dgidx-output'.
[05.24.15 13:47:14] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_-data-partials-forge-output'.
[05.24.15 13:47:16] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_-data-partials-cumulative-partials'.
[05.24.15 13:47:17] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_-data-workbench-dgraph-config'.
[05.24.15 13:47:18] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_-data-dgraphs-local-dgraph-input'.
[05.24.15 13:47:19] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_-data-dgraphs-local-cumulative-partials'.
[05.24.15 13:47:20] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_-data-dgraphs-local-dgraph-config'.
[05.24.15 13:47:22] INFO: Setting definition for component 'DgraphA1'.
[05.24.15 13:47:22] INFO: Setting definition for script 'PromoteAuthoringToLive'.
[05.24.15 13:47:22] INFO: Setting definition for custom component 'WorkbenchManager'.
[05.24.15 13:47:22] INFO: Updating provisioning for host 'ITLHost'.
[05.24.15 13:47:22] INFO: Updating definition for host 'ITLHost'.
[05.24.15 13:47:22] INFO: [ITLHost] Starting shell utility 'mkpath_-'.
[05.24.15 13:47:23] INFO: [ITLHost] Starting shell utility 'mkpath_-data-workbench-temp'.
[05.24.15 13:47:24] INFO: [ITLHost] Starting shell utility 'mkpath_-data-processing'.
[05.24.15 13:47:26] INFO: Setting definition for custom component 'IFCR'.
[05.24.15 13:47:26] INFO: Updating provisioning for host 'ITLHost'.
[05.24.15 13:47:26] INFO: Updating definition for host 'ITLHost'.
[05.24.15 13:47:26] INFO: [ITLHost] Starting shell utility 'mkpath_-'.
[05.24.15 13:47:27] INFO: [ITLHost] Starting shell utility 'mkpath_-'.
[05.24.15 13:47:28] INFO: Setting definition for component 'LogServer'.
[05.24.15 13:47:28] INFO: [ReportGenerationHost] Starting shell utility 'mkpath_-reports-input'.
[05.24.15 13:47:30] INFO: Setting definition for script 'DaySoFarReports'.
[05.24.15 13:47:30] INFO: Setting definition for script 'DailyReports'.
[05.24.15 13:47:30] INFO: Setting definition for script 'WeeklyReports'.
[05.24.15 13:47:30] INFO: Setting definition for script 'DaySoFarHtmlReports'.
[05.24.15 13:47:30] INFO: Setting definition for script 'DailyHtmlReports'.
[05.24.15 13:47:30] INFO: Setting definition for script 'WeeklyHtmlReports'.
[05.24.15 13:47:31] INFO: Setting definition for component 'WeeklyReportGenerator'.
[05.24.15 13:47:31] INFO: Setting definition for component 'DailyReportGenerator'.
[05.24.15 13:47:31] INFO: Setting definition for component 'DaySoFarReportGenerator'.
[05.24.15 13:47:31] INFO: Setting definition for component 'WeeklyHtmlReportGenerator'.
[05.24.15 13:47:31] INFO: Setting definition for component 'DailyHtmlReportGenerator'.
[05.24.15 13:47:32] INFO: Setting definition for component 'DaySoFarHtmlReportGenerator'.
[05.24.15 13:47:32] INFO: Setting definition for script 'BaselineUpdate'.
[05.24.15 13:47:32] INFO: Setting definition for script 'PartialUpdate'.
[05.24.15 13:47:32] INFO: Setting definition for component 'ConfigurationGeneratorForge'.
[05.24.15 13:47:32] INFO: Setting definition for component 'Forge'.
[05.24.15 13:47:33] INFO: [ITLHost] Starting shell utility 'mkpath_-data-incoming'.
[05.24.15 13:47:34] INFO: Setting definition for component 'PartialForge'.
[05.24.15 13:47:34] INFO: [ITLHost] Starting shell utility 'mkpath_-data-partials-incoming'.
[05.24.15 13:47:35] INFO: Setting definition for component 'Dgidx'.
[05.24.15 13:47:36] INFO: Definition updated.
[05.24.15 13:47:36] INFO: Provisioning site from prototype...
[05.24.15 13:47:36] INFO: Finished provisioning site from prototype.
Finished updating EAC.
Initializing record stores...
These record stores exist:
NAME                    TYPE                    STATUS
CRS_en_schema                   RecordStore                     RUNNING
CRS_en_dimvals                  RecordStore                     RUNNING
CRS_en_prules                   RecordStore                     RUNNING
CRS_en_data                     RecordStore                     RUNNING
==================================================
Dropping old record stores (errors will occur on non-existent rs):
Error during execution (check log for more info): Component instance not found: Store_en_schema
Error during execution (check log for more info): Component instance not found: Store_en_dimvals
Error during execution (check log for more info): Component instance not found: Store_en_prules
Error during execution (check log for more info): Component instance not found: Store_en_data
==================================================
Creating fresh record stores:
Successfully created component: Store_en_schema
Successfully created component: Store_en_dimvals
Successfully created component: Store_en_prules
Successfully created component: Store_en_data
==================================================
Deploying rs configs:
Successfully set recordstore configuration.
Successfully set recordstore configuration.
Successfully set recordstore configuration.
Successfully set recordstore configuration.
==================================================
Finished initializing record stores
Importing sample content...
[05.24.15 13:48:33] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
[05.24.15 13:48:33] INFO: Definition has not changed.
[05.24.15 13:48:33] INFO: Packaging contents for upload...
[05.24.15 13:48:34] INFO: Finished packaging contents.
[05.24.15 13:48:34] INFO: Uploading contents to: http://JagdevSingh-PC:8006/ifcr/sites/Store/
[05.24.15 13:48:35] INFO: Finished uploading contents.
[05.24.15 13:48:37] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
[05.24.15 13:48:38] INFO: Definition has not changed.
[05.24.15 13:48:38] INFO: Packaging contents for upload...
[05.24.15 13:48:38] INFO: Finished packaging contents.
[05.24.15 13:48:38] INFO: Uploading contents to: http://JagdevSingh-PC:8006/ifcr/sites/Store/content
[05.24.15 13:48:39] INFO: Finished uploading contents.
[05.24.15 13:48:41] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
[05.24.15 13:48:42] INFO: Definition has not changed.
[05.24.15 13:48:42] INFO: Packaging contents for upload...
[05.24.15 13:48:42] INFO: Finished packaging contents.
[05.24.15 13:48:42] INFO: Uploading contents to: http://JagdevSingh-PC:8006/ifcr/sites/Store/pages
[05.24.15 13:48:43] INFO: Finished uploading contents.
Finished importing sample content
Importing media...
[05.24.15 13:48:45] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
[05.24.15 13:48:46] INFO: Definition has not changed.
[05.24.15 13:48:46] INFO: Packaging contents for upload...
[05.24.15 13:48:46] INFO: Finished packaging contents.
[05.24.15 13:48:46] INFO: Uploading contents to: http://JagdevSingh-PC:8006/ifcr/sites/Store/media
[05.24.15 13:48:49] INFO: Finished uploading contents.
Finished importing media
Importing editors configuration...
[05.24.15 13:48:51] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
[05.24.15 13:48:52] INFO: Definition has not changed.
[05.24.15 13:48:52] INFO: Packaging contents for upload...
[05.24.15 13:48:52] INFO: Finished packaging contents.
[05.24.15 13:48:52] INFO: Uploading contents to: http://JagdevSingh-PC:8006/ifcr/sites/Store/configuration/tools/xmgr
[05.24.15 13:48:52] INFO: Finished uploading contents.
Finished importing editors configuration
Importing templates...
Removing existing cartridge templates for Store
Setting new cartridge templates for Store
Finished setting templates
Finished importing templates

C:\Endeca\apps\Store\control>

=============================================================================
At this phase your Endeca application is ready. Next you need to configure ATG application to use this application.



Saturday, 13 June 2015

Resolving 'ConfigurationGeneratorForge' failed Issue

Endeca baseline indexing failed for newly created application. 


Here is the baseline error detail.

=======================================================================

C:\Endeca\apps\Store\control>baseline_update.bat
[06.13.15 19:35:25] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
[06.13.15 19:35:26] INFO: Updating provisioning for component 'ConfigurationGeneratorForge'.
[06.13.15 19:35:26] INFO: Updating definition for component 'ConfigurationGeneratorForge'.
[06.13.15 19:35:27] INFO: Updating provisioning for component 'Forge'.
[06.13.15 19:35:27] INFO: Updating definition for component 'Forge'.
[06.13.15 19:35:27] INFO: Updating provisioning for component 'PartialForge'.
[06.13.15 19:35:27] INFO: Updating definition for component 'PartialForge'.
[06.13.15 19:35:27] INFO: Definition updated.
[06.13.15 19:35:27] INFO: Starting baseline update script.
[06.13.15 19:35:27] INFO: Acquired lock 'update_lock'.
[06.13.15 19:35:27] INFO: [ITLHost] Starting shell utility 'cleanDir_processing'.
[06.13.15 19:35:29] INFO: [ITLHost] Starting shell utility 'cleanDir_forge-output'.
[06.13.15 19:35:30] INFO: [ITLHost] Starting shell utility 'cleanDir_dgidx-output'.
[06.13.15 19:35:31] INFO: [ITLHost] Starting shell utility 'move_-_to_processing'.
[06.13.15 19:35:32] INFO: [ITLHost] Starting copy utility 'fetch_config_to_input_for_forge_Forge'.
[06.13.15 19:35:34] INFO: [ITLHost] Starting backup utility 'backup_log_dir_for_component_ConfigurationGeneratorForge'.
[06.13.15 19:35:35] INFO: [ITLHost] Starting component 'ConfigurationGeneratorForge'.
[06.13.15 19:35:45] SEVERE: Batch component  'ConfigurationGeneratorForge' failed. Refer to component logs in C:\Endeca\apps\Store\config\script\..\..\.\logs\forges\ConfigurationGeneratorForge on host ITLHost.
Occurred while executing line 20 of valid BeanShell script:
[[

17|
18|        // Generate instance configuration
19|        ConfigurationGeneratorForge.archiveLogDir();
20|        ConfigurationGeneratorForge.run();
21|
22|        // archive logs and run ITL
23|        Forge.archiveLogDir();

]]

[06.13.15 19:35:45] SEVERE: Caught an exception while invoking method 'run' on object 'BaselineUpdate'. Releasing locks.

Caused by java.lang.reflect.InvocationTargetException
sun.reflect.NativeMethodAccessorImpl invoke0 - null
Caused by com.endeca.soleng.eac.toolkit.exception.AppControlException
com.endeca.soleng.eac.toolkit.script.Script runBeanShellScript - Error executing valid BeanShell script.
Caused by com.endeca.soleng.eac.toolkit.exception.EacComponentControlException
com.endeca.soleng.eac.toolkit.component.BatchComponent run - Batch component  'ConfigurationGeneratorForge' failed. Refer to component logs in C:\Endeca\apps\Store\config\script\..\..\.\logs\forges\ConfigurationGeneratorForge on host ITLHost.

[06.13.15 19:35:45] INFO: Released lock 'update_lock'.

C:\Endeca\apps\Store\control>


=======================================================================

Here is stack trace from ConfigurationGeneratorForge.start.log

=======================================================================

INFO    06/13/15 14:05:38.432 UTC (1434204338431)    FORGE    {config}: (AdapterRunner): Adapter class: com.endeca.itl.fcm.integration.cadk.ifdi.RecordPropertyRenamer  
ERROR    06/13/15 14:05:38.432 UTC (1434204338431)    FORGE    {config}: (AdapterRunner): java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
 com.endeca.edf.adapter.AdapterRunner$LoadAdapterException: java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
     at com.endeca.edf.adapter.AdapterRunner.loadAdapter(AdapterRunner.java:326)
     at com.endeca.edf.adapter.AdapterRunner.run(AdapterRunner.java:162)
     at com.endeca.edf.adapter.AdapterRunner.main(AdapterRunner.java:43)
 Caused by: java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
     at com.endeca.itl.fcm.integration.cadk.ifdi.RecordPropertyRenamer.<init>(RecordPropertyRenamer.java:44)
     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
     at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
     at java.lang.Class.newInstance0(Class.java:355)
     at java.lang.Class.newInstance(Class.java:308)
     at com.endeca.edf.adapter.AdapterRunner.loadAdapter(AdapterRunner.java:317)
     ... 2 more
 Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
     at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
     at java.security.AccessController.doPrivileged(Native Method)
     at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
     at java.lang.ClassLoader.loadClass(ClassLoader.java:303)
     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
     at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
     at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316)
     ... 10 more
    

 =======================================================================

Solution : Replace value recordstore-forge-adapter-3.1.1.jar with recordstore-forge-adapter-3.1.2.jar in CAS_RS_CLASSPATH property of environment.properties [This file ia available in <application directory>\config\script\].

Friday, 12 June 2015

Prevent Cross-Site Script attacks in ATG using SecurityServlet

Oracle ATG Web Commerce includes a component,  /atg/dynamo/servlet/dafpipeline/SecurityServlet, that monitors query parameters and stops processes if they appear suspicious. 

The SecurityServlet component uses the /atg/dynamo/servlet/security/ParameterValidator component to check query parameters.

The SecurityServlet component is enabled by default. You can disable it by removing /atg/dynamo/servlet/dafpipeline/SecurityServlet from the insertableServlets property of the /atg/dynamo/servlet/dafpipeline/DynamoHandler/ component.

 By default ParameterValidator  handle below.
  1.  illegal html tags.
  2.  illegal html attributes.
  3.  Parameter values with illegal pattern.
You can enhance the handling of  the suspicious  parameters values (point 3 above) by configuring below property in ParameterValidator. 

Default illegal regex is

illegalRegexes=javascript\\s*:,(^|[^a-zA-Z])x\\s*:,vbscript\\s*:,/\\s*>,^\\s*>


Now add alert and expression. Then expression will be.

illegalRegexes=javascript\\s*:,(^|[^a-zA-Z])x\\s*:,vbscript\\s*:,/\\s*>,^\\s*>,alert(\\(|%28),expression(\\(|%28)