OOZIE Installation is Simplified

Oozie Installation

Hi All,

It took few hours for me to install Oozie :(. So I thought of making it simple for others. Below are steps & links for installing oozie.

A. Pre-Requisite:
1. Unix (Ubuntu, Redhat, Hp unix, etc ...)
2. Java 1.6+
3. Apache Hadoop (0.20 onwards)
4. Maven (mvn command should work from your terminal). To install Maven refer to : http://www.mkyong.com/maven/how-to-install-maven-in-ubuntu/
5. ExtJS Library (optional, it is to enable webconsole) - http://extjs.com/deploy/ext-2.2.zip (Download to /tmp)

B. Installation Steps:

1. Download latest version of Oozie from http://mirror.symnds.com/software/Apache/oozie/ (I have downloaded version 3.3.2)

2. Extract the downloaded tar.gz (tar -zxf oozie-3.3.2.tar.gz)

3. cd oozie-3.3.2/bin (Your version may be different!! )

4. ./mkdistro.sh -DskipTests (Run this command to build Oozie - It means you have downloaded oozie source & now need to build as a binary). Upon successful build it will create Oozie Binary. Note: It needs internet connection & will take some time.
Success build will show below message:
Oozie distro created, DATE[2013.08.13-12:54:51GMT] VC-REV[unavailable], available at [<Oozie downloaded path>/oozie-3.3.2/distro/target]

5. Change owner: sudo chown -R hduser:hadoop <oozie folder path>

6. Change directory name to oozie: sudo mv oozie-3.2.2 oozie

7. Move oozie folder from Downloads to other folder. In my case, moved to /home/hduser : Change directory name to oozie: sudo mv oozie /home/hduser/.

8. su hduser (login as hduser to continue setup) & cd

9. export OOZIE_HOME=<oozie path> (in my case: export OOZIE_HOME=/home/hduser/oozie)

10. Create libext folder under $OOZIE_HOME (mkdir libext)

11. Copy all jars from $HADOOP_HOME & $$HADOOP_HOME/lib to newly create libext folder (cp $HADOOP_HOME/lib/*jar $OOZIE_HOME/libext/. & cp $HADOOP_HOME/*jar $OOZIE_HOME/libext/.)

12. copy extJS-2.2.zip to $OOZIE_HOME/libext (before copying make sure the owner is same as oozie)

13. copy oozie war file:
- cp $OOZIE_HOME/distro/target/oozie-3.3.2-distro/oozie-3.3.2/oozie.war $OOZIE_HOME/webapp/srs/main/webapp/.

14. Change configuration as per below :
vi ./distro/target/oozie-3.3.2-distro/oozie-3.3.2/conf/oozie-site.xml (by default the valu is false. Change it to true)

<property>
        <name>oozie.service.JPAService.create.db.schema</name>
        <value>true</value>
        <description>
            Creates Oozie DB.
            If set to true, it creates the DB schema if it does not exist. If the DB schema exists is a NOP.
            If set to false, it does not create the DB schema. If the DB schema does not exist it fails start up.
        </description>
    </property>

15. Run the following command to create OOZIE DB:
- $OOZIE_HOME/distro/target/oozie-3.3.2-distro/oozie-3.3.2/bin/ooziedb.sh create -sqlfile oozie.sql -run

If it is successfull, the output should be:
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"

Validate DB Connection
DONE
Check DB schema does not exist
DONE
Check OOZIE_SYS table does not exist
DONE
Create SQL schema
DONE
Create OOZIE_SYS table
DONE

Oozie DB has been created for Oozie version '3.3.2'

16. To enable webconsole, we need to install the ext JS library. Also, oozie war file requires few other jar files like hadoop-core-<version>.jar & commons-configuration<version>.jar

- ./bin/addtowar.sh -inputwar oozie.war -outputwar oozie1.war -jars            /home/hduser/oozie/hadooplibs/target/oozie-3.3.2-hadooplibs/oozie-3.3.2/hadooplibs/hadooplib-1.1.1.oozie-3.3.2/*.jar -extjs $OOZIE_HOME/libext/ext-2.2.zip (make sure hadoop-core-<YOUR DISTRIBUTION VERSION>.jar & commons-configuration<VERSION>.jar are added)

- Run below commands to change the name of jar & deploy:
   i) rm -f oozie.war
   ii) mv oozie1.war oozie.war
   iii) cp oozie.war $OOZIE_HOME/distro/target/oozie-3.3.2-distro/oozie-3.3.2/oozie-server/webapps/.

17. Start the Oozie Server (oozied.sh start)

18. Run below command to check the status of oozie server:
bin/oozie admin -oozie http://localhost:11000/oozie -status
(It should return "System Mode: NORMAL")
19. To view the log file tail -100f logs/catalina.out


20. Finally configure hadoop core-site.xml. Open core-site.xml (located under $HADOOP_HOME/conf) and add below configuration:
<!-- OOZIE -->
  <property>
    <name>hadoop.proxyuser.hduser.hosts</name>
    <value>localhost</value>
  </property>
  <property>
    <name>hadoop.proxyuser.hduser.groups</name>
    <value>*</value>
  </property>
19. To access webconsole of oozie: http://localhost:11000/oozie .. - Able to view workflow status

20. To view the log file tail -100f logs/catalina.out


21. Finally configure hadoop core-site.xml. Open core-site.xml (located under $HADOOP_HOME/conf) and add below configuration:
<!-- OOZIE -->
  <property>
    <name>hadoop.proxyuser.hduser.hosts</name>
    <value>localhost</value>
  </property>
  <property>
    <name>hadoop.proxyuser.hduser.groups</name>
    <value>*</value>
  </property>

*************End of Installation - Enjoy Workflow Scheduler*****************************