Flink1.10.1编译hadoop2.7.2 编译flink-shaded-hadoop-2-uber
从Flink 1.11开始,flink-shaded-hadoop-2-uberFlink项目不再正式支持使用发行版。
如果想建立flink-shaded对供应商特定的Hadoop版本,您必须首先描述配置特定供应商的Maven仓库在本地Maven安装在这里。
运行以下命令以flink-shaded针对所需的Hadoop版本(例如对于version 2.6.5-custom)进行构建和安装:
mvn clean install -Dhadoop.version=2.6.5-custom
1. 下载/flink-shaded
前往git 下载/flink-shaded 下载relese-10.0 因为10.0是包含hadoop-bin的,1.11 不包含hadoop
https://github.com/apache/flink-shaded.git
2. 解压至linux
tar -zxvf flink-shaded-release-10.0.tar.gz
3. 进入flink-shaded-hadoop-2-parent,修改pom
[ryxiong@node01 flink-shaded-release-10.0]$ cd flink-shaded-hadoop-2-parent/
修改flink-shaded-hadoop-2-uber下pom.xml文件, 执行zookeeper的版本
[ryxiong@node01 flink-shaded-hadoop-2-parent]$ cd flink-shaded-hadoop-2-uber/
[ryxiong@node01 flink-shaded-hadoop-2-uber]$ vim pom.xml
4. 进入flink-shaded-release-10.0下 修改pom
[ryxiong@node01 job]$ cd flink-shaded-release-10.0/
[ryxiong@node01 flink-shaded-release-10.0]$ vim pom.xml
在profiles标签中添加如下内容
<profile><id>vendor-repos</id><activation><property><name>vendor-repos</name></property></activation> <!-- Add vendor maven repositories --><repositories> <!-- Cloudera --><repository><id>cloudera-releases</id><url>https://maven.aliyun.com/repository/central</url><releases><enabled>true</enabled></releases><snapshots><enabled>false</enabled></snapshots></repository></repositories>
</profile>
添加阿里云镜像, 编译过程中下载块
5. 配置完毕,运行前提是需要linux配置maven
在flink-shaded-release-10.0执行
mvn clean install -DskipTests -Dhadoop.version=2.7.2 -Drat.skip=true
进入漫长的编译过程, 没有ERROR即成功
[INFO] Reactor Summary:
[INFO]
[INFO] flink-shaded ...................................... SUCCESS [1.266s]
[INFO] flink-shaded-force-shading ........................ SUCCESS [0.383s]
[INFO] flink-shaded-asm-7 ................................ SUCCESS [0.464s]
[INFO] flink-shaded-guava-18 ............................. SUCCESS [0.882s]
[INFO] flink-shaded-netty-4 .............................. SUCCESS [4.089s]
[INFO] flink-shaded-netty-tcnative-dynamic ............... SUCCESS [0.457s]
[INFO] flink-shaded-jackson-parent ....................... SUCCESS [0.015s]
[INFO] flink-shaded-jackson-2 ............................ SUCCESS [0.816s]
[INFO] flink-shaded-jackson-module-jsonSchema-2 .......... SUCCESS [0.595s]
[INFO] flink-shaded-hadoop-2-parent ...................... SUCCESS [0.017s]
[INFO] flink-shaded-hadoop-2 ............................. SUCCESS [8.014s]
[INFO] flink-shaded-hadoop-2-uber ........................ SUCCESS [10.657s]
[INFO] flink-shaded-zookeeper-parent ..................... SUCCESS [0.011s]
[INFO] flink-shaded-zookeeper-3.4 ........................ SUCCESS [41.310s]
[INFO] flink-shaded-zookeeper-3.5 ........................ SUCCESS [33.234s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:42.440s
[INFO] Finished at: Fri Apr 09 16:56:24 CST 2021
[INFO] Final Memory: 45M/1112M
[INFO] ------------------------------------------------------------------------
[mac@ryxiong flink-shaded-release-10.0]$ll
编译好的路径
[mac@ryxiong target]$pwd
~/flink-shaded-release-10.0/flink-shaded-hadoop-2-parent/flink-shaded-hadoop-2-uber/target
6. 编译好的jar包 放入flinke1.10.1的lib下
[ryxiong@node01 flink-shaded-release-10.0]$ cp flink-shaded-hadoop-2-uber-2.7.2-10.0.jar /opt/module/flink-1.10.1/lib/
我编译好的jar包,有需要的可以下载.
https://download.csdn.net/download/Ryxiong728/16593328