Useful Linux Commands

Sometimes we need to work on Linux environment during development specially on Linux box hosting the server.As we know Linux is bundled with lots of useful commadline features, file handling is one of them.Here is small list of commands which I feel very useful while working on Linux.

Note : Command syntax/example given below are based on requirements/assignment I worked upon.Every command below may have multiple options,which can be used in different combination based on scenarios(requirement).For example I am redirecting output to a file which can also be displayed on console.

      1.  Secure File Copy.
          Using this feature you can securely copy file from one machine to another machine without login into any of the machine. 
            Syntax : scp user1@machine1:Path of source file user2@machine2:Path of the destination directory
            Example :scp admin@192.168.0.101:/opt/file/Test.jar jboss@192.168.0.109:/opt/jboss/app/lib

      2.  Fetching the specific line range from file and write result to another file.
           Syntax : sed -n 'startLineNumber,EndLineNumberp' fileToBeSearch > resultFile
           Example : sed -n '1,500p' Data.csv > SmallData.csv
 
      3. Fetching the line containing given text from file.
          Syntax :grep searchText SourceFile > FileContainingResultData
          Example : Search For single String
            grep nike BrandData.csv > NikeData.csv

          Example : Search For 2 Strings with OR operation
            grep 'nike\|adidas' BrandData.csv > NikeORAdidasData.csv
          Example : Search For 2 Strings with AND operation  
           grep nike BrandData.csv  | grep shoe > NikeANDShoeData.csv 
 
      4.  Changing file or directory (folder) permissions.

        Sometimes it required to change the permissions of an file or directory in order to perform certain operations.
            Syntax : chmod permissionOctalCode fileOrDirectoryName

            Example : chmod 777 data
            chmod 777 Record.csv
            Note : Use -R option to recursively change the permission of the directory.
           
      5.  Searching file by name.
            find command is very useful to search files.
            Syntax : find directoryToSearch -name fileName
            Example : find / -name 'eaccmd.sh'

      6.  Access environment as different user.
            Using below command you can access the environment as an different user.Mostly it is used to access the environment as more privileged user(root). 
            Syntax : sudo su - userName
            Example : sudo su - admin

      7.  Navigate to previous directory.
             Using below command you can navigate to previous directory from where you have navigated to current directory.
             Syntax : cd - 

      8. Create a zip file.
            Using below command you can add single or multiple files to zip.
            Syntax : zip nameOfZip.zip file1 file2 ...
            Example : zip Log.zip server.log console.log

      9. Unzip the zipped file.
           Use below command to unzip the zipped file.
           Syntax : unzip nameOfZip.zip
           Example : unzip Log.zip
         
    10.  Display content of zip file without extracting it.
           You can display the content of zip file without extracting it.
           Syntax : unzip -l nameOfZip.zip
           Example : unzip -l Log.zip

     11.  Delete all lines containing specific text from a file.Without opening the file.
           You can delete all lines containing 'some string' from a file without opening it.
           Syntax : sed --in-place '/some string/d' myfile
           Example : sed --in-place '/nike/d' Brand.csv

For more detail on any of the above commands use man command(manual pages or man pages).

Solution for signer information error (occuring at server startup or on browsing) for ATG 10.1 CRS

I have configured CRS (Commerce Reference Store) for ATG 10.1 using CIM (Configuration and Installation Manager). After server startup I tried to browse the application but only header was displaying.Server was throwing below exception.

ERROR [[jsp]] Servlet.service() for servlet jsp threw exception java.lang.SecurityException: class "atg.commerce.order.OrderHolder$$EnhancerByCGLIB$$e9ab456d"'s signer information does not match signer information of other classes in the same package
     at java.lang.ClassLoader.checkCerts(ClassLoader.java:807)
     at java.lang.ClassLoader.preDefineClass(ClassLoader.java:488)
     at java.lang.ClassLoader.defineClassCond(ClassLoader.java:626)
     at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
     at java.lang.reflect.Method.invoke(Method.java:597)
     at net.sf.cglib.core.ReflectUtils.defineClass(ReflectUtils.java:384)
     at net.sf.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:219) 

Solution :  Replace the jboss-eap-5.0/jboss-as/common/lib/cglib.jar with the unsigned version, which can be downloaded from the following URL : http://repository.jboss.com/maven2-brew/cglib/cglib/2.2/cglib-2.2.jar or http://search.maven.org/remotecontent?filepath=cglib/cglib/2.2.2/cglib-2.2.2.jar.
This is due to a problem with the JBoss cglib library.For more details see the JBoss bug https://jira.jboss.org/jira/browse/JBPAPP-2971.

Removing (uninstalling) a deployed Endeca application

Use below steps to remove Endeca application properly. Endeca application must be removed before deleting the application directory in order to avoid side effects.

1. Remove application from the EAC(Endeca Application Controller).

• List the current applications using below command.

             $ENDECA_ROOT/bin/eaccmd list-apps

• Remove the application by running runcommand --remove-app from application's control directory. 
• Verify the removed application by running above eaccmd list-apps script.Removed application won't be listed now.    

    2.   Navigate to apps directory and delete the selected(removed above) application directory.

Note : These steps I have used for application which is running without CAS. For application using CAS you need to remove related record stores.

Request Processing in ATG

In every web application request plays a very important role.User(browser) need to send request to server to access any web resource.

ATG performs below steps to process requests it receives.

1. When a user requests a page(send request to server).Web server parses the request and holds in an HTTPServletRequest object.
2. The Web server passes the HTTPServletRequest to the application server.It wraps the request in its own flavor of request around the generic one before passing it to the Web application.
3. If you defined custom filters and the request is for a JHTML page, the filters execute in the order you specified.
4. Any custom J2EE servlets are processed.
5. The Web application calls one of the following resources.

•  DynamoProxyServlet( DAS servlet pipeline for JHTML requests).
•  PageFilter (DAF servlet pipeline for JSP requests).

     6. The request is passed on to a pipeline of servlets.
     7. If you created custom filters and a JSP is being requested, they execute after the last pipeline servlet, but before the request returns to the application server.
     8. After the servlet pipeline reaches the end, it returns the request to the application server for final processing. 

Below diagram illustrates ATG request processing.
 

Note : DAS pipeline is invoked in the case of JHTML.Because JHTML is a proprietary language, it relies on the page compiler provided in the DAS servlet pipeline to generate JHTML into a servlet that is rendered as HTML by the application server.

Encrypting strings using java

Encryption : Encryption is the science of encoding and decoding secret messages.The message or information (referred to as plain text) is encrypted using an encryption algorithm, turning it into an unreadable cipher text . This is usually done with the use of an encryption key, which specifies how the message is to be encoded. Any adversary that can see the cipher text should not be able to determine anything about the original message. An authorized party, however, is able to decode the cipher text using a decryption algorithm, that usually requires a secret decryption key, that adversaries do not have access to. 

Here is the idea,how this can be done , for example password encryption.

• First accept user password.
• Encrypt password using random salt.
• Store the salt and encrypted password.
• For verification accept password from user.
• Fetch and decode salt for user.Already stored above.
• Encrypt user entered password using decoded salt.
• Now compare encrypted password with already stored encrypted password.Then take the action accordingly.

Below code demonstrates how to encrypt strings(password) using java.

package com.test;

import java.io.IOException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;

import sun.misc.BASE64Decoder;
import sun.misc.BASE64Encoder;

/**
 * This class contains code for encryption demo. 
 * 
 * @author Jagdev
 * 
 */

public class EncryptionTest {

/**
* @param args
* @throws NoSuchAlgorithmException
* @throws IOException
*/
public static void main(String[] args) throws NoSuchAlgorithmException,
IOException {
// String password
String password = "Jagdev";
// Hash function iterations
int iterationNb = 5000;
// SecureRandom object that implements the SHA1PRNG Random Number
// Generator (RNG) algorithm
SecureRandom random = SecureRandom.getInstance("SHA1PRNG");
// Salt generation 64 bits long
byte[] bSalt = new byte[8];
random.nextBytes(bSalt);
MessageDigest digest = MessageDigest.getInstance("SHA-1");
digest.update(bSalt);
byte[] input = digest.digest(password.getBytes("UTF-8"));

for (int i = 0; i < iterationNb; i++) {
input = digest.digest(input);
}

BASE64Encoder endecoder = new BASE64Encoder();
// encode the password using BASE64Encoder
String encodedPassword = endecoder.encode(input);
System.out.println("Encoded password is " + encodedPassword);
System.out.println("Salt is = " + new String(bSalt));
// encode the salt using BASE64Encoder
String encodedSalt = endecoder.encode(bSalt);
                //Code for encode string using decoded salt.
BASE64Decoder decoder = new BASE64Decoder();
// decode the encodedSalt
byte[] decodedSalt = decoder.decodeBuffer(encodedSalt);
// reset the digest
digest.reset();
// update the digest using decoded salt
digest.update(decodedSalt);
input = digest.digest(password.getBytes("UTF-8"));

for (int i = 0; i < iterationNb; i++) {
input = digest.digest(input);
}

String encodedpassword = endecoder.encode(input);
System.out.println("Encoded password is " + encodedpassword);
System.out.println("Decoded salt is = " + new String(decodedSalt));
}

}

Endeca “Failed to obtain lock” indexing error solution

While running baseline or partial indexing,you may encounter the below error.

INFO:Checking definition from AppConfig.xml against existing EAC provisioning.
INFO:Definition has not changed.
INFO:Starting baseline update script.
WARNING:Failed to obtain lock. 

Below is the reason behind this.

If running script breaks halfway through its execution due to an unhandled exception, or is manually interrupted 

by a user pressing Ctrl-C while it is running, the lock remains set within the EAC.  

To resolve this use below commands.

On Windows: .\runcommand.bat LockManager releaseLock update_lock
On UNIX: ./runcommand.sh LockManager releaseLock update_lock

Now run the indexing,it will work.

Endeca environment variables

Here I will explain how to set Endeca environment variables.Before discussing Endeca environment variables,I would like to introduce the term Environment variable. 

Environment variable

An environment variable is a dynamic "object" on a computer that stores a value, which in turn can be referenced by one or more software programs.It can be said that environment variables help to create and shape the environment  where a program runs.

Endeca environment variables

Below are the four types of the Endeca evironment variables.

1. MDEX Engine variables

• ENDECA_MDEX_ROOT : Path of the MDEX Engine root directory(default ../MDEX/version)

2. Platform Services variables

• ENDECA_ROOT : Path of the Platform Services root directory(default ../PlatformServices/version)

• ENDECA_CONF : Path of the workspace directory for the Endeca Http Service(default  ../PlatformServices/workspace)

3. Endeca Workbench variables

• ENDECA_TOOLS_ROOT : Path of the Endeca Workbench root directory(default ../Workbench/version). In latest installers \ToolsAndFrameworks\<version>.

• ENDECA_TOOLS_CONF : Path of the workspace directory for the Endeca Tools Service(default ../Workbench/workspace). In latest installers \ToolsAndFrameworks\<version>\server\workspace.

4. Other variables(Set and used by the Endeca Deployment Template)

• ENDECA_PROJECT_DIR : Path of the deployed application.(Default user supplied input at installation time).
• ENDECA_PROJECT_NAME : Project name that is used.(Default user supplied input at installation time).

Setting the environment variables ensures that the different Endeca components can communicate with each other properly.

Run  below scripts to set Endeca environment variables.
1. Windows Platform Script
On windows platform, Platform Services and Oracle Endeca Workbench environment variables are set at the time of installation.

You only need to set the MDEX Engine environment variable(ENDECA_MDEX_ROOT).Run the below script to achieve this.

\Endeca\MDEX\<version>\mdex_setup.bat

2. Linux Platform Scripts.
On Linux platform run the below scripts.
For MDEX Engine variables.
   source /endeca/MDEX/<version>/mdex_setup_sh.ini
For Platform Services variables.
   source /endeca/PlatformServices/workspace/setup/installer_sh.ini
For Endeca Workbench variables.
  source /endeca/Workbench/workspace/setup/installer_sh.ini