修改jar包中class文件并重新打包

修改jar包中class文件并重新打包

背景

使用parquet格式Avro数据序列化过程中报错，报错原因：Avro对字段名有校验，只支持字母和下划线开头[A-Za-z_]，本次报错Illegal initial character: $ip就是使用了$ip字段名，字段名$开头所以校验不通过报错

2023-12-11 19:17:43,738 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
    at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:181)
    at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
    at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.lang.RuntimeException: org.apache.seatunnel.shade.connector.file.org.apache.avro.SchemaParseException: Illegal initial character: $ip
    at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:206)
    at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:59)
    at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:39)
    at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:27)
    at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.handleRecord(IntermediateBlockingQueue.java:76)
    at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.collect(IntermediateBlockingQueue.java:51)
    at org.apache.seatunnel.engine.server.task.flow.IntermediateQueueFlowLifeCycle.collect(IntermediateQueueFlowLifeCycle.java:52)
    at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.collect(TransformSeaTunnelTask.java:73)
    at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:161)
    at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.call(TransformSeaTunnelTask.java:78)
    at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:526)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.seatunnel.shade.connector.file.org.apache.avro.SchemaParseException: Illegal initial character: $ip
    at org.apache.seatunnel.shade.connector.file.org.apache.avro.Schema.validateName(Schema.java:1562)
    at org.apache.seatunnel.shade.connector.file.org.apache.avro.Schema.access$400(Schema.java:91)
    at org.apache.seatunnel.shade.connector.file.org.apache.avro.Schema$Field.<init>(Schema.java:546)
    at org.apache.seatunnel.shade.connector.file.org.apache.avro.Schema$Field.<init>(Schema.java:585)
    at org.apache.seatunnel.shade.connector.file.org.apache.parquet.avro.AvroSchemaConverter.convertFields(AvroSchemaConverter.java:295)
    at org.apache.seatunnel.shade.connector.file.org.apache.parquet.avro.AvroSchemaConverter.convert(AvroSchemaConverter.java:279)
    at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.ParquetWriteStrategy.buildAvroSchemaWithRowType(ParquetWriteStrategy.java:344)
    at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.ParquetWriteStrategy.getOrCreateWriter(ParquetWriteStrategy.java:135)
    at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.ParquetWriteStrategy.write(ParquetWriteStrategy.java:96)
    at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:126)
    at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:43)
    at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:201)
    ... 15 more
  
    at org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:119)
    at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:174)

解决方案

1. 字段名换成可校验通过的字段名（使用全字母，例如：$ip换成ip）
2. 修改jar包中class文件并重新打包（由于我需要使用$开头的字段名，所以我选择了这种方式）
   1. 通过这种方式来记录【修改jar包中class文件并重新打包】的全过程操作

操作

使用到的工具

1. IntelliJ IDEA

概要步骤

1. 使用 idea 修改jar包中的class文件
2. 解压jar包
3. 替换class文件
4. 压缩jar包

详细步骤

修改jar包中class文件

1. 创建一个空的maven项目

2. 将对应的jar包导入到 src/main/resources/lib 目录下，并右键jar包选择【Add as Library】
   

3. 找到需要修改的.class文件（我这里以Schema.class为例）

4. 在 src/main/java 目录下创建一个跟上一步 .class 文件一样的 包目录和类名(.java)，将 .class 文件的内容复制到 .java 文件中
   

5. 解决依赖的问题（如果出现找不到对应的依赖，可能原因是jar包引用的是外部依赖，这个jar包没有对应的依赖，这个时候就需要在pom中先引入对应依赖，如果没有出现这个问题可以进行下一步）
   我的这个就是找不到slf4j依赖，我就在pom中引用了对应的依赖包
   

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-log4j12</artifactId>
    <version>1.7.36</version>
</dependency>

6. 修改.java程序中的逻辑代码

7. 修改后右键 Build Module 项目转成 .class 文件
   

重新打包成jar包

1. 解压jar包
   jar xf connector-file-hadoop-2.3.1.jar

2. 替换对应 .class 文件

   1. 找到对应 .class 文件目录
   2. 将编译出来的 .class 文件替换（特别注意??：如果一个 .java 程序编译出了多个 .class 文件，则需要将多个 .class 文件都替换，我这里是有多个 .class 文件，所以都要替换）
      

3. 重新打包
   jar cf connector-file-hadoop-2.3.1.jar ./*