Question 296621 in CentOS Question posted in
The official documentation can be found here.

Centos – Fail in Building Apache Tez from Source

I am setting up a cluster of nodes (CentOs 8) using Hadoop version 3.3.6. And I have tried to set up Apache Tez (version 0.9.2) for the execution engine of Hive (version 3.1.3) with Java 8. But after settin up hadoop version in pom.xml and hitting mvn clean package -DskipTests=true -Dmaven.javadoc.skip=true, it ends up being like this:

[INFO] 4 warnings
[INFO] -------------------------------------------------------------
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /usr/local/hadoop/apache-tez-0.9.2-src/tez-dag/src/main/java/org/apache/tez/state/StateMachineTez.java:[28,8] org.apache.tez.state.StateMachineTez is not abstract and does not override abstract method getPreviousState() in org.apache.hadoop.yarn.state.StateMachine
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] tez 0.9.2 .......................................... SUCCESS [  1.240 s]
[INFO] hadoop-shim ........................................ SUCCESS [  1.435 s]
[INFO] tez-api ............................................ SUCCESS [  5.870 s]
[INFO] tez-build-tools .................................... SUCCESS [  0.105 s]
[INFO] tez-common ......................................... SUCCESS [  0.560 s]
[INFO] tez-runtime-internals .............................. SUCCESS [  0.871 s]
[INFO] tez-runtime-library ................................ SUCCESS [  2.554 s]
[INFO] tez-mapreduce ...................................... SUCCESS [  1.232 s]
[INFO] tez-examples ....................................... SUCCESS [  0.258 s]
[INFO] tez-dag ............................................ FAILURE [  2.530 s]
[INFO] tez-tests .......................................... SKIPPED
[INFO] tez-ext-service-tests .............................. SKIPPED
[INFO] tez-ui ............................................. SKIPPED
[INFO] tez-plugins ........................................ SKIPPED
[INFO] tez-protobuf-history-plugin ........................ SKIPPED
[INFO] tez-yarn-timeline-history .......................... SKIPPED
[INFO] tez-yarn-timeline-history-with-acls ................ SKIPPED
[INFO] tez-history-parser ................................. SKIPPED
[INFO] tez-aux-services ................................... SKIPPED
[INFO] tez-tools .......................................... SKIPPED
[INFO] tez-perf-analyzer .................................. SKIPPED
[INFO] tez-job-analyzer ................................... SKIPPED
[INFO] tez-javadoc-tools .................................. SKIPPED
[INFO] hadoop-shim-impls .................................. SKIPPED
[INFO] hadoop-shim-2.7 .................................... SKIPPED
[INFO] tez-dist ........................................... SKIPPED
[INFO] Tez 0.9.2 .......................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17.025 s
[INFO] Finished at: 2023-09-07T19:24:55Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project tez-dag: Compilation failure
[ERROR] /usr/local/hadoop/apache-tez-0.9.2-src/tez-dag/src/main/java/org/apache/tez/state/StateMachineTez.java:[28,8] org.apache.tez.state.StateMachineTez is not abstract and does not override abstract method getPreviousState() in org.apache.hadoop.yarn.state.StateMachine
[ERROR]
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project tez-dag: Compilation failure
/usr/local/hadoop/apache-tez-0.9.2-src/tez-dag/src/main/java/org/apache/tez/state/StateMachineTez.java:[28,8] org.apache.tez.state.StateMachineTez is not abstract and does not override abstract method getPreviousState() in org.apache.hadoop.yarn.state.StateMachine

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:213)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:954)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
Caused by: org.apache.maven.plugin.compiler.CompilationFailureException: Compilation failure
/usr/local/hadoop/apache-tez-0.9.2-src/tez-dag/src/main/java/org/apache/tez/state/StateMachineTez.java:[28,8] org.apache.tez.state.StateMachineTez is not abstract and does not override abstract method getPreviousState() in org.apache.hadoop.yarn.state.StateMachine

    at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute (AbstractCompilerMojo.java:858)
    at org.apache.maven.plugin.compiler.CompilerMojo.execute (CompilerMojo.java:129)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:954)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[ERROR]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :tez-dag

I’ve downloaded an older version but it was totally in vain due to some errors like this:

[ERROR] /usr/local/hadoop/apache-tez-0.9.0-src/tez-mapreduce/src/main/java/org/apache/tez/mapreduce/client/NotRunningJob.java:[89,29] no suitable method found for 

newInstance(org.apache.hadoop.yarn.api.records.ApplicationId,org.apache.hadoop.yarn.api.records.ApplicationAttemptId,java.lang.String,java.lang.String,java.lang.String,java.lang.String,int,

<nulltype>,org.apache.hadoop.yarn.api.records.YarnApplicationState,java.lang.String,java.lang.String,int,int,org.apache.hadoop.yarn.api.records.FinalApplicationStatus,

<nulltype>,java.lang.String,float,java.lang.String,<nulltype>)

How can I resolve this problem?

Tags:

2

Answers


  1. 0

    Consider building newer version (if possible) like tez 10.1
    and before building apply patches from bigtop repository

    With this metod i succesfully build tez 10.1 on rocky8 (rhel compatible) and even created rpm for rhel8, although i was building against hadoop 3.3.4 and hive 3.1.3.

    Login or Signup to reply.
  2. 0

    After Hadoop YARN devs have fixed this issue https://issues.apache.org/jira/browse/YARN-11395 and backported it to a few previous release versions, Tez devs hit the same issue and has fixed it here.

    As you can see here, there is a new method in StateMachine interface which needs to be overridden in any class that implements it.

    StateMachineTez also implements StateMachine interface. So, what you need to do is to override the new method in StateMachineTez class before building it;

      @Override
  public STATE getPreviousState() {
    return realStatemachine.getPreviousState();
  }
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search