Endeca SEVERE: Unauthorized (401): Unauthorized access to workbench

Endeca application initialization failed with below error.

=======================================================================
[01.23.16 10:12:03] SEVERE: Unauthorized (401): Unauthorized access to workbench. Please check your credentials.
Occurred while executing line 3 of valid BeanShell script:
[[

1|
2|
3|   IFCR.provisionSite();
4|   CAS.importDimensionValueIdMappings("MyStore-dimension-value-id-manager",
5|                   InitialSetup.getWorkingDir() + "/test_data/initial_dval_id_mappings.csv");
6|

]]

[01.23.16 10:12:03] SEVERE: Caught an exception while invoking method 'run' on object 'InitialSetup'. Releasing locks.

Caused by java.lang.reflect.InvocationTargetException
sun.reflect.NativeMethodAccessorImpl invoke0 - null
Caused by com.endeca.soleng.eac.toolkit.exception.AppControlException
com.endeca.soleng.eac.toolkit.script.Script runBeanShellScript - Error executing valid BeanShell script.
Caused by com.endeca.soleng.eac.toolkit.utility.IFCRUtility$HttpStatusException
com.endeca.soleng.eac.toolkit.utility.IFCRUtility execute - Unauthorized (401): Unauthorized access to workbench. Please check your credentials.
 
=======================================================================

Root Cause : This happened due to ToolsAndFramework installed with password [admin@1234] other then admin.

Update ToolsAndFramework password to admin using below steps.

C:\Endeca\ToolsAndFrameworks\11.1.0\credential_store\bin> manage_credentials.bat add --key ifcr --user admin

Enter password for user admin :

Re-enter password to confirm :

Credential of type [password] already exists for this key.

Do you want to replace it [yes/no] ?yes
23 Jan 2016 10:27:23,954  INFO CSFHandler:146 - Credential successfully updated for map : endecaToolsAndFrameworks.

C:\Endeca\ToolsAndFrameworks\11.1.0\credential_store\bin>

ProductCatalogSimpleIndexingAdmin component indexing phases and indexing buttons are missing

Sometimes when you are working on indexing related components. You may face this issue. In this case ProductCatalogSimpleIndexingAdmin component UI is looks like normal ATG component [indexing phases and indexing buttons are missing].

Below is the screen capture of erroneous ProductCatalogSimpleIndexingAdmin UI.

Root Cause : This happens when any one of the component listed in phaseToPrioritiesAndTasks is not able initialize/start.

Solution : Make sure that there is no error in all components in phaseToPrioritiesAndTasks .

After fixing issue here is the expected UI.

Write to record store failed due to handle invalid record

After creating new record store. Write to record store failed with below error.

=======================================================================
2016-01-02 20:35:29,565 ERROR [MyStore-schema] [1415899750@jetty-7] com.endeca.itl.recordstore.impl.RecordStoreImpl: Error executing method RecordStoreImpl.writeRecords()
java.lang.NullPointerException
    at com.endeca.itl.recordstore.impl.RecordStoreImpl.handleInvalidRecord(RecordStoreImpl.java:282)
    at com.endeca.itl.recordstore.impl.RecordStoreImpl.validateRecords(RecordStoreImpl.java:274)
    at com.endeca.itl.recordstore.impl.RecordStoreImpl.writeRecords(RecordStoreImpl.java:242)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:180)
    at com.endeca.itl.service.ServicePublisher$2.performInvocation(ServicePublisher.java:106)
    at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
    at org.apache.cxf.jaxws.AbstractJAXWSMethodInvoker.invoke(AbstractJAXWSMethodInvoker.java:178)
    at org.apache.cxf.jaxws.JAXWSMethodInvoker.invoke(JAXWSMethodInvoker.java:64)
    at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:75)
    at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at org.apache.cxf.workqueue.SynchronousExecutor.execute(SynchronousExecutor.java:37)
    at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:107)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:262)
    at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:122)
    at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:211)
    at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:213)
    at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:193)
    at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:129)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:187)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:110)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:166)
    at sun.reflect.GeneratedMethodAccessor102.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.endeca.itl.plugin.ThreadContextRunner$1$1.doWork(ThreadContextRunner.java:19)
    at com.endeca.itl.plugin.ThreadContextRunner.run(ThreadContextRunner.java:110)
    at com.endeca.itl.plugin.ThreadContextRunner$1.invoke(ThreadContextRunner.java:25)
    at $Proxy1.service(Unknown Source)
    at com.endeca.itl.bootstrap.CasBootStrapServlet.service(CasBootStrapServlet.java:105)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
    at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
    at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
2016-01-02 20:35:29,566 WARN [] [1415899750@jetty-7] org.apache.cxf.phase.PhaseInterceptorChain: Application {http://recordstore.itl.endeca.com/}RecordStoreService#{http://recordstore.itl.endeca.com/}writeRecords has thrown exception, unwinding now
org.apache.cxf.interceptor.Fault
    at org.apache.cxf.service.invoker.AbstractInvoker.createFault(AbstractInvoker.java:162)
    at org.apache.cxf.jaxws.AbstractJAXWSMethodInvoker.createFault(AbstractJAXWSMethodInvoker.java:213)
    at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:128)
    at org.apache.cxf.jaxws.AbstractJAXWSMethodInvoker.invoke(AbstractJAXWSMethodInvoker.java:178)
    at org.apache.cxf.jaxws.JAXWSMethodInvoker.invoke(JAXWSMethodInvoker.java:64)
    at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:75)
    at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at org.apache.cxf.workqueue.SynchronousExecutor.execute(SynchronousExecutor.java:37)
    at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:107)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:262)
    at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:122)
    at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:211)
    at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:213)
    at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:193)
    at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:129)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:187)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:110)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:166)
    at sun.reflect.GeneratedMethodAccessor102.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.endeca.itl.plugin.ThreadContextRunner$1$1.doWork(ThreadContextRunner.java:19)
    at com.endeca.itl.plugin.ThreadContextRunner.run(ThreadContextRunner.java:110)
    at com.endeca.itl.plugin.ThreadContextRunner$1.invoke(ThreadContextRunner.java:25)
    at $Proxy1.service(Unknown Source)
    at com.endeca.itl.bootstrap.CasBootStrapServlet.service(CasBootStrapServlet.java:105)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
    at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
    at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.lang.NullPointerException
    at com.endeca.itl.recordstore.impl.RecordStoreImpl.handleInvalidRecord(RecordStoreImpl.java:282)
    at com.endeca.itl.recordstore.impl.RecordStoreImpl.validateRecords(RecordStoreImpl.java:274)
    at com.endeca.itl.recordstore.impl.RecordStoreImpl.writeRecords(RecordStoreImpl.java:242)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:180)
    at com.endeca.itl.service.ServicePublisher$2.performInvocation(ServicePublisher.java:106)
    at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
    ... 44 more
=======================================================================

Solution : update record store configuration to handle(ignore) invalid records.

Step 1. Get record store configuration

recordstore-cmd.bat get-configuration -a MyStore-schema -f schema.xml

Step 2. Update recoed store configuration file with below line [edit xml file created in above step].

 <ignoreInvalidRecords>true</ignoreInvalidRecords>

Step 3. Set record store configuration using below command.

recordstore-cmd.bat set-configuration -a MyStore-schema -f schema.xml

Endeca Application Initialization error

After creating new endeca application successfully. Endeca application initialization failed with below error.
=========================================================================

SEVERE: Caught an exception while invoking method 'run' on object 'InitialSetup'. Releasing locks.

java.lang.reflect.InvocationTargetException

  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

  at java.lang.reflect.Method.invoke(Method.java:606)

  at com.endeca.soleng.eac.toolkit.Controller.invokeRequestedMethod(Controller.java:931)

  at com.endeca.soleng.eac.toolkit.Controller.execute(Controller.java:269)

  at com.endeca.soleng.eac.toolkit.Controller.main(Controller.java:137)

Caused by: com.endeca.soleng.eac.toolkit.exception.AppControlException: Unknown error executing a BeanShell script.

  at com.endeca.soleng.eac.toolkit.script.Script.runBeanShellScript(Script.java:184)

  at com.endeca.soleng.eac.toolkit.script.Script.run(Script.java:127)

  ... 7 more

Caused by: Sourced file: inline evaluation of: ``             IFCR.provisionSite();     CAS.importDimensionValueIdMappings("Store-d . . . '' : Error in method invocation: Method importDimensionValueIdMappings( java.lang.String, java.lang.String ) not found in class'com.endeca.soleng.eac.toolkit.component.CustomComponent' : at Line: 4 : in file: inline evaluation of: ``             IFCR.provisionSite();     CAS.importDimensionValueIdMappings("Store-d . . . '' : CAS .importDimensionValueIdMappings ( "Store-dimension-value-id-manager" , InitialSetup .getWorkingDir ( ) + "/test_data/initial_dval_id_mappings.csv" )

  at bsh.BSHMethodInvocation.eval(Unknown Source)

  at bsh.BSHPrimaryExpression.eval(Unknown Source)

  at bsh.BSHPrimaryExpression.eval(Unknown Source)

=========================================================================

Below is the error in application log file (MyStore.0.0.log) 

WARNING: Error parsing 'class' attribute:com.endeca.eac.toolkit.component.cas.ContentAcquisitionServerComponent

This attribute is in CAS custom component definition (DataIngest.xml).

Solution :  Add casStubs.jar from CAS_ROOT\lib\cas-dt\ to classpath in runcommand script.

Below is example from runcomman.bat file.

set CLASSPATH=%CLASSPATH%;C:\Endeca\CAS\11.1.0\lib\cas-dt\casStubs.jar

Use xslt to Index xml File into Endeca

Here is the small example with steps to use xslt while indexing xml file into Endeca. 

1. First of all configure xml file in Record Adapter of pipeline. 

 

2. Then click on Transformer tab.

Here configure below two fields.

a. Type : The file format of the transformation style sheet.The only valid value is XSLT.

b. URL :  Specifies the location of the .xsl file. The path is either an absolute path or a path relative to the location of the Pipeline.epx file. In this example transform.xsl file is located at the location of the pipeline.epx file.

In this example xslt is used to convert name to uppercase.

------------------------------------------------------------------------------------------------------------------------

 transform.xsl

------------------------------------------------------------------------------------------------------------------------

 <?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet  xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:variable name="smallcase" select="'abcdefghijklmnopqrstuvwxyz'" />
<xsl:variable name="uppercase" select="'ABCDEFGHIJKLMNOPQRSTUVWXYZ'" />
<xsl:template match="/">
<RECORDS>
<xsl:for-each select="employees/employee">
    <RECORD>
         <PROP NAME="P_Age">
             <PVAL><xsl:value-of select="age"/></PVAL>
          </PROP>
          <PROP NAME="P_Name">
             <PVAL><xsl:value-of select="translate(name, $smallcase, $uppercase)"/></PVAL>
          </PROP>
          <PROP NAME="Dep_ID">
             <PVAL><xsl:value-of select="depid"/></PVAL>
          </PROP>
    </RECORD>
</xsl:for-each>
</RECORDS>
</xsl:template>
</xsl:stylesheet>

 ------------------------------------------------------------------------------------------------------------------------

 ------------------------------------------------------------------------------------------------------------------------

Sample input xml file (data.xml)

 ------------------------------------------------------------------------------------------------------------------------

 <?xml version="1.0" encoding="UTF-8"?>
<employees>
   <employee>
      <name>Jon</name>
      <age>20</age>
      <depid>123</depid>
   </employee>
   <employee>
      <name>Ban</name>
      <age>30</age>
      <depid>456</depid>
   </employee>
   <employee>
      <name>Kim</name>
      <age>40</age>
      <depid>789</depid>
   </employee>
</employees>

 ------------------------------------------------------------------------------------------------------------------------

3. Run baseline indexing and see the resutls in jspref.

Resolved | Missing Discover application folders in Tools And Frameworks

After installing Endeca related softwares I tried to provision the Discover reference application. Then realized that discover application related folders are missing from installation.

Root Cause : ToolsAndFrameworks was installed with installation type Minimal option. This option does not includes the reference application.

Solution : Install ToolsAndFrameworks with installation type Complete option.

Interesting ATG Findclass Utility

While going through ATG documentation found one interesting utility. This is Findclass Utility. This will find the .class or JAR file from which a Java class has been loaded. It will also print the CLASS_VERSION information if found.

This utility can be accessed using below URL on your Oracle ATG Web Commerce server.

http://server:port/dyn/dyn/findclass.jhtml

Enter the name of the class in the Class Name field and click Find Class button.

For example ,here I have searched for atg.commerce.order.OrderImpl class.

Find Class

To print debug information append &debug=true to URL. Below is screen capture for OrderImpl with debug information.

Find Class with Debug Information

Class name field accepts class name in below formats.

• atg.commerce.order.OrderImpl
• atg.commerce.order.OrderImpl.class
• atg/commerce/order/OrderImpl.class
• /atg/commerce/order/OrderImpl.java