Endeca Implementing a Custom Java Record Manipulator

Follow below steps to write and configure custom java record manipulator.

1.   Write java manipulator using CADK (Content Adapter Development Kit) API classes.

2.   Compile this java manipulator into a JAR file.

3.   Configure manipulator in Forge pipeline.

Below is the example of simple java manipulator.

This record manipulator is doing two tasks.

1. Converting/updating one record property to lower case.

2. Adding one new record property.

=========================================================================

 package com.test.endeca.manipulator;

import com.endeca.edf.adapter.Adapter;
import com.endeca.edf.adapter.AdapterConfig;
import com.endeca.edf.adapter.AdapterException;
import com.endeca.edf.adapter.AdapterHandler;
import com.endeca.edf.adapter.PVal;
import com.endeca.edf.adapter.Record;

public class SimpleJavaManipulator implements Adapter {

    private static final String PASSTHROUGH_SOURCEPROP = "SOURCEPROP";
    private static final String PASSTHROUGH_TARGETPROP = "TARGETPROP";
  

    @Override
    public void execute(AdapterConfig adapterConfig, AdapterHandler adapterHandler)
            throws AdapterException {       
        String sourceprop = adapterConfig.first(PASSTHROUGH_SOURCEPROP);
        String targetprop = adapterConfig.first(PASSTHROUGH_TARGETPROP);
      
        if (sourceprop == null)
            throw new AdapterException(PASSTHROUGH_SOURCEPROP + " passthrough not specified");

        if (targetprop == null)
            throw new AdapterException(PASSTHROUGH_TARGETPROP + " passthrough not specified");

       
        for (int inp = 0; inp != adapterHandler.getNumInputs(); inp++)
        {          
            boolean hasMoreRecords = true;
            long currentRecord = 0;
          
            while(hasMoreRecords)
            {
       
               Record rec = adapterHandler.getRecord(inp);
               if (rec != null)
                {
                    ++currentRecord;
                    String sourcepropValue = null;

                   
                      for (PVal prop : rec)
                      {
                          if (prop.getName().equals(sourceprop))
                          {
                               sourcepropValue = prop.getValue();

                              // update current property to lower case
                              prop.setValue(sourcepropValue.toLowerCase());
                              break;
                          }
                       }
                    // Add new property
                    if (sourcepropValue != null)  {
                        rec.add(new PVal(targetprop,sourcepropValue));                                               
                    }                  
                    adapterHandler.emit(rec);
                } else {
                    hasMoreRecords = false;
                }
            }
        }      
    }

}

=========================================================================

Note : CADK API jar is available at %ENDECA_ROOT%\cadk\lib (adapter.jar).

Once manipulator is implemented and compiled into jar, now configure this manipulator into forge pipeline.

Steps to Configure java manipulator into forge pipeline.

1. Open Endeca application pipeline in developer studio.

2. Right Click then select New then Java Manipulator.

3. In manipulator pop up add general information like java home, manipulator class name and path of the manipulator compiled jar.

4. Add data sources.

5. Add Pass Throughs.

6. Add manipulator as data source in next pipeline component.

7. Now your manipulator is ready. Save the pipeline and run the baseline.

Create And Provision Endeca Application with the Deployment Template

Here are the steps to create an Endeca application from scratch and provision it using deployment template.

Application Creation 
 

      1. Open command line and change directory to ..\...\ToolsAndFrameworks\3.1.2  

          \deployment_template\bin and invoke deploy script. 

           For ex. C:\Endeca\ToolsAndFrameworks\3.1.2\deployment_template\bin\deply.bat

 

      2. Provide configuration parameter/values as script prompts you. First of all it will confirm

          IAP version. Then it will prompt following information.

• Application Name.
• Application Deployment directory.
• EAC port.
• Workbench port.
• Live dgraph port.
• Authoring Dgraph port.
• Log Server port.

Screen  Captures for application creation.

For port values you can use any available port on your machine, or use default one.At the end you will get message that appplication deployed successfully.

Now application creation is done. It is time to provision it. Once application provisioned it will be available for configuration in Endeca Workbench.

To provision an application.

Go to \control directory of your application you created earlier (above steps).

Invoke initialize_services.bat or initialize_services.sh script. This script won't prompt for any values.

Here is the screen capture of application provisioning.

 

 

Introduction to Endeca Content Acquisition System (CAS)

The Endeca Content Acquisition System (CAS) is a set of Components That add, configure, and crawl data sources for use in an application.The Endeca Content Acquisition System crawls data sources, Converts documents and files to Endeca records, and stores them for Use in an Forge pipeline.Output of CAS is number of Record Store instances (one per data source).

Below are Important Points Related to CAS.

The Endeca CAS service is a servlet container That runs the CAS Server, the Component Instance.Manager, and any number of Record Store instances (one per data source).

The CAS Server is the component That Manages all file systems and CMS crawling operations.

The CAS Console Endeca Workbench is a Web-based application used to crawl various data sources including file systems and content management systems.

The CAS Server API allows users to write programs That Communicate With The CAS server.The Server API has a WSDL interface and a CAS Server Command-Line Utility.

The Dimension Value Id Manager is a CAS Component That Creates, stores dimension value identifiers.

The Endeca Web Crawler manages all Web crawl-related operations.

The Component Instance Manager Creates, lists, and deletes Record Store instances. The Component Instance Manager has a WSDL interface and overpriced a CIM Command-Line Utility.

The Endeca Record Store Provides persistent storage for generation of Records. The Record Store has a WSDL interface and Also a Record Store Command-line Utility.The CAS Server writes crawl output from each data source to a unique Record Store instance.

Endeca CMS connectors are available for use in the CAS Console Endeca Workbench or the CAS Server API. CMS Connectors Provide a means to access and crawl data sources in a wide variety of CMS types.