当前位置: 代码迷 >> 综合 >> Win安装Spark问题:ERROR SparkContext: Error initializing SparkContext
  详细解决方案

Win安装Spark问题:ERROR SparkContext: Error initializing SparkContext

热度:31   发布时间:2023-12-15 17:25:49.0

问题背景:

Java版本:1.8.0.291
Spark版本:spark-3.2.0-bin-hadoop3.2
Windows操作系统64位
Spark初次安装

问题描述:

配置完Java环境和Spark路径之后,在cmd中执行Spark-shell时,出现如下错误

Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
21/11/28 20:34:27 ERROR SparkContext: Error initializing SparkContext.
java.lang.reflect.InvocationTargetExceptionat sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)at java.lang.reflect.Constructor.newInstance(Constructor.java:423)at org.apache.spark.executor.Executor.addReplClassLoaderIfNeeded(Executor.scala:909)at org.apache.spark.executor.Executor.<init>(Executor.scala:160)at org.apache.spark.scheduler.local.LocalEndpoint.<init>(LocalSchedulerBackend.scala:64)at org.apache.spark.scheduler.local.LocalSchedulerBackend.start(LocalSchedulerBackend.scala:132)at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:220)at org.apache.spark.SparkContext.<init>(SparkContext.scala:581)at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2690)at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:949)at scala.Option.getOrElse(Option.scala:189)at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:943)at org.apache.spark.repl.Main$.createSparkSession(Main.scala:106)at $line3.$read$$iw$$iw.<init>(<console>:15)at $line3.$read$$iw.<init>(<console>:42)at $line3.$read.<init>(<console>:44)at $line3.$read$.<init>(<console>:48)at $line3.$read$.<clinit>(<console>)at $line3.$eval$.$print$lzycompute(<console>:7)at $line3.$eval$.$print(<console>:6)at $line3.$eval.$print(<console>)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:747)at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1020)at scala.tools.nsc.interpreter.IMain.$anonfun$interpret$1(IMain.scala:568)at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:36)at scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:116)at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:41)at scala.tools.nsc.interpreter.IMain.loadAndRunReq$1(IMain.scala:567)at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:594)at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:564)at scala.tools.nsc.interpreter.IMain.$anonfun$quietRun$1(IMain.scala:216)at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:206)at scala.tools.nsc.interpreter.IMain.quietRun(IMain.scala:216)at org.apache.spark.repl.SparkILoop.$anonfun$initializeSpark$2(SparkILoop.scala:83)at scala.collection.immutable.List.foreach(List.scala:431)at org.apache.spark.repl.SparkILoop.$anonfun$initializeSpark$1(SparkILoop.scala:83)at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)at scala.tools.nsc.interpreter.ILoop.savingReplayStack(ILoop.scala:97)at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:83)at org.apache.spark.repl.SparkILoop.$anonfun$process$4(SparkILoop.scala:165)at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)at scala.tools.nsc.interpreter.ILoop.$anonfun$mumly$1(ILoop.scala:166)at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:206)at scala.tools.nsc.interpreter.ILoop.mumly(ILoop.scala:163)at org.apache.spark.repl.SparkILoop.loopPostInit$1(SparkILoop.scala:153)at org.apache.spark.repl.SparkILoop.$anonfun$process$10(SparkILoop.scala:221)at org.apache.spark.repl.SparkILoop.withSuppressedSettings$1(SparkILoop.scala:189)at org.apache.spark.repl.SparkILoop.startup$1(SparkILoop.scala:201)at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:236)at org.apache.spark.repl.Main$.doMain(Main.scala:78)at org.apache.spark.repl.Main$.main(Main.scala:58)at org.apache.spark.repl.Main.main(Main.scala)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.net.URISyntaxException: Illegal character in path at index 39: spark://windows10.microdone.cn:56140/C:\classesat java.net.URI$Parser.fail(URI.java:2845)at java.net.URI$Parser.checkChars(URI.java:3018)at java.net.URI$Parser.parseHierarchical(URI.java:3102)at java.net.URI$Parser.parse(URI.java:3050)at java.net.URI.<init>(URI.java:588)at org.apache.spark.repl.ExecutorClassLoader.<init>(ExecutorClassLoader.scala:57)... 70 more
21/11/28 20:34:27 ERROR Utils: Uncaught exception in thread main
java.lang.NullPointerExceptionat org.apache.spark.scheduler.local.LocalSchedulerBackend.org$apache$spark$scheduler$local$LocalSchedulerBackend$$stop(LocalSchedulerBackend.scala:173)at org.apache.spark.scheduler.local.LocalSchedulerBackend.stop(LocalSchedulerBackend.scala:144)at org.apache.spark.scheduler.TaskSchedulerImpl.stop(TaskSchedulerImpl.scala:927)at org.apache.spark.scheduler.DAGScheduler.stop(DAGScheduler.scala:2516)at org.apache.spark.SparkContext.$anonfun$stop$12(SparkContext.scala:2086)at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1442)at org.apache.spark.SparkContext.stop(SparkContext.scala:2086)at org.apache.spark.SparkContext.<init>(SparkContext.scala:677)at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2690)at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:949)at scala.Option.getOrElse(Option.scala:189)at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:943)at org.apache.spark.repl.Main$.createSparkSession(Main.scala:106)at $line3.$read$$iw$$iw.<init>(<console>:15)at $line3.$read$$iw.<init>(<console>:42)at $line3.$read.<init>(<console>:44)at $line3.$read$.<init>(<console>:48)at $line3.$read$.<clinit>(<console>)at $line3.$eval$.$print$lzycompute(<console>:7)at $line3.$eval$.$print(<console>:6)at $line3.$eval.$print(<console>)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:747)at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1020)at scala.tools.nsc.interpreter.IMain.$anonfun$interpret$1(IMain.scala:568)at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:36)at scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:116)at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:41)at scala.tools.nsc.interpreter.IMain.loadAndRunReq$1(IMain.scala:567)at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:594)at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:564)at scala.tools.nsc.interpreter.IMain.$anonfun$quietRun$1(IMain.scala:216)at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:206)at scala.tools.nsc.interpreter.IMain.quietRun(IMain.scala:216)at org.apache.spark.repl.SparkILoop.$anonfun$initializeSpark$2(SparkILoop.scala:83)at scala.collection.immutable.List.foreach(List.scala:431)at org.apache.spark.repl.SparkILoop.$anonfun$initializeSpark$1(SparkILoop.scala:83)at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)at scala.tools.nsc.interpreter.ILoop.savingReplayStack(ILoop.scala:97)at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:83)at org.apache.spark.repl.SparkILoop.$anonfun$process$4(SparkILoop.scala:165)at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)at scala.tools.nsc.interpreter.ILoop.$anonfun$mumly$1(ILoop.scala:166)at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:206)at scala.tools.nsc.interpreter.ILoop.mumly(ILoop.scala:163)at org.apache.spark.repl.SparkILoop.loopPostInit$1(SparkILoop.scala:153)at org.apache.spark.repl.SparkILoop.$anonfun$process$10(SparkILoop.scala:221)at org.apache.spark.repl.SparkILoop.withSuppressedSettings$1(SparkILoop.scala:189)at org.apache.spark.repl.SparkILoop.startup$1(SparkILoop.scala:201)at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:236)at org.apache.spark.repl.Main$.doMain(Main.scala:78)at org.apache.spark.repl.Main$.main(Main.scala:58)at org.apache.spark.repl.Main.main(Main.scala)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
21/11/28 20:34:27 WARN MetricsSystem: Stopping a MetricsSystem that is not running
21/11/28 20:34:27 ERROR Main: Failed to initialize Spark session.
java.lang.reflect.InvocationTargetExceptionat sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)at java.lang.reflect.Constructor.newInstance(Constructor.java:423)at org.apache.spark.executor.Executor.addReplClassLoaderIfNeeded(Executor.scala:909)at org.apache.spark.executor.Executor.<init>(Executor.scala:160)at org.apache.spark.scheduler.local.LocalEndpoint.<init>(LocalSchedulerBackend.scala:64)at org.apache.spark.scheduler.local.LocalSchedulerBackend.start(LocalSchedulerBackend.scala:132)at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:220)at org.apache.spark.SparkContext.<init>(SparkContext.scala:581)at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2690)at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:949)at scala.Option.getOrElse(Option.scala:189)at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:943)at org.apache.spark.repl.Main$.createSparkSession(Main.scala:106)at $line3.$read$$iw$$iw.<init>(<console>:15)at $line3.$read$$iw.<init>(<console>:42)at $line3.$read.<init>(<console>:44)at $line3.$read$.<init>(<console>:48)at $line3.$read$.<clinit>(<console>)at $line3.$eval$.$print$lzycompute(<console>:7)at $line3.$eval$.$print(<console>:6)at $line3.$eval.$print(<console>)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:747)at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1020)at scala.tools.nsc.interpreter.IMain.$anonfun$interpret$1(IMain.scala:568)at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:36)at scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:116)at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:41)at scala.tools.nsc.interpreter.IMain.loadAndRunReq$1(IMain.scala:567)at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:594)at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:564)at scala.tools.nsc.interpreter.IMain.$anonfun$quietRun$1(IMain.scala:216)at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:206)at scala.tools.nsc.interpreter.IMain.quietRun(IMain.scala:216)at org.apache.spark.repl.SparkILoop.$anonfun$initializeSpark$2(SparkILoop.scala:83)at scala.collection.immutable.List.foreach(List.scala:431)at org.apache.spark.repl.SparkILoop.$anonfun$initializeSpark$1(SparkILoop.scala:83)at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)at scala.tools.nsc.interpreter.ILoop.savingReplayStack(ILoop.scala:97)at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:83)at org.apache.spark.repl.SparkILoop.$anonfun$process$4(SparkILoop.scala:165)at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)at scala.tools.nsc.interpreter.ILoop.$anonfun$mumly$1(ILoop.scala:166)at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:206)at scala.tools.nsc.interpreter.ILoop.mumly(ILoop.scala:163)at org.apache.spark.repl.SparkILoop.loopPostInit$1(SparkILoop.scala:153)at org.apache.spark.repl.SparkILoop.$anonfun$process$10(SparkILoop.scala:221)at org.apache.spark.repl.SparkILoop.withSuppressedSettings$1(SparkILoop.scala:189)at org.apache.spark.repl.SparkILoop.startup$1(SparkILoop.scala:201)at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:236)at org.apache.spark.repl.Main$.doMain(Main.scala:78)at org.apache.spark.repl.Main$.main(Main.scala:58)at org.apache.spark.repl.Main.main(Main.scala)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.net.URISyntaxException: Illegal character in path at index 39: spark://windows10.microdone.cn:56140/C:\classesat java.net.URI$Parser.fail(URI.java:2845)at java.net.URI$Parser.checkChars(URI.java:3018)at java.net.URI$Parser.parseHierarchical(URI.java:3102)at java.net.URI$Parser.parse(URI.java:3050)at java.net.URI.<init>(URI.java:588)at org.apache.spark.repl.ExecutorClassLoader.<init>(ExecutorClassLoader.scala:57)... 70 more
21/11/28 20:34:27 ERROR Utils: Uncaught exception in thread shutdown-hook-0
java.lang.ExceptionInInitializerErrorat org.apache.spark.executor.Executor.stop(Executor.scala:333)at org.apache.spark.executor.Executor.$anonfun$stopHookReference$1(Executor.scala:76)at org.apache.spark.util.SparkShutdownHook.run(ShutdownHookManager.scala:214)at org.apache.spark.util.SparkShutdownHookManager.$anonfun$runAll$2(ShutdownHookManager.scala:188)at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:2019)at org.apache.spark.util.SparkShutdownHookManager.$anonfun$runAll$1(ShutdownHookManager.scala:188)at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)at scala.util.Try$.apply(Try.scala:213)at org.apache.spark.util.SparkShutdownHookManager.runAll(ShutdownHookManager.scala:188)at org.apache.spark.util.SparkShutdownHookManager$$anon$2.run(ShutdownHookManager.scala:178)at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)at java.util.concurrent.FutureTask.run(FutureTask.java:266)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerExceptionat org.apache.spark.shuffle.ShuffleBlockPusher$.<init>(ShuffleBlockPusher.scala:465)at org.apache.spark.shuffle.ShuffleBlockPusher$.<clinit>(ShuffleBlockPusher.scala)... 16 more
21/11/28 20:34:27 WARN ShutdownHookManager: ShutdownHook '$anon$2' failed, java.util.concurrent.ExecutionException: java.lang.ExceptionInInitializerError
java.util.concurrent.ExecutionException: java.lang.ExceptionInInitializerErrorat java.util.concurrent.FutureTask.report(FutureTask.java:122)at java.util.concurrent.FutureTask.get(FutureTask.java:206)at org.apache.hadoop.util.ShutdownHookManager.executeShutdown(ShutdownHookManager.java:124)at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:95)
Caused by: java.lang.ExceptionInInitializerErrorat org.apache.spark.executor.Executor.stop(Executor.scala:333)at org.apache.spark.executor.Executor.$anonfun$stopHookReference$1(Executor.scala:76)at org.apache.spark.util.SparkShutdownHook.run(ShutdownHookManager.scala:214)at org.apache.spark.util.SparkShutdownHookManager.$anonfun$runAll$2(ShutdownHookManager.scala:188)at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:2019)at org.apache.spark.util.SparkShutdownHookManager.$anonfun$runAll$1(ShutdownHookManager.scala:188)at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)at scala.util.Try$.apply(Try.scala:213)at org.apache.spark.util.SparkShutdownHookManager.runAll(ShutdownHookManager.scala:188)at org.apache.spark.util.SparkShutdownHookManager$$anon$2.run(ShutdownHookManager.scala:178)at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)at java.util.concurrent.FutureTask.run(FutureTask.java:266)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerExceptionat org.apache.spark.shuffle.ShuffleBlockPusher$.<init>(ShuffleBlockPusher.scala:465)at org.apache.spark.shuffle.ShuffleBlockPusher$.<clinit>(ShuffleBlockPusher.scala)... 16 more

 

问题解决

应该是启动时执行的java文件中有非法字符,导致启动失败,没能找到启动的是哪一个java文件出现错误,
尝试降低版本,使用spark-3.1.2-bin-hadoop3.2后成功启动
 

 

  相关解决方案