Cannot get Hive data

[JenniferOH]

Hi I’m Jennifer and I’m having trouble with getting Hive data using scala kernel.
I’m testing exact same code with same hive-site.xml(hive config file) on both spark-shell and jupyterlab spylon-kernel

Here’s my jupyterlab code:

Here’s my spark-shell code:

There weren’t many references, and the ones that I’ve tried are:

(Optional) Configuring Spark for Hive Access - Hortonworks Data Platform
https://groups.google.com/g/cloud-dataproc-discuss/c/O5wKJDW9kNQ
There’s no hive or spark related logs on JupyterLab. Here’s the logs

[W 2022-09-23 09:47:49.887 LabApp] Could not determine jupyterlab build status without nodejs
[I 2022-09-23 09:47:50.393 ServerApp] Kernel started: afa4234d-48ac-4505-b6a0-e3fa220161cd
[I 2022-09-23 09:47:50.404 ServerApp] Kernel started: 9404ff88-622f-4ba8-86b1-404d648588fc
[MetaKernelApp] ERROR | No such comm target registered: jupyter.widget.control
[MetaKernelApp] WARNING | No such comm: 94bab30b-35b1-48bf-bb51-6000d46df671
[MetaKernelApp] ERROR | No such comm target registered: jupyter.widget.control
[MetaKernelApp] WARNING | No such comm: 936a8763-707c-4076-8471-7ceed85ccb53
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
[I 2022-09-23 09:57:52.541 ServerApp] Saving file at /scala-spark/Untitled.ipynb
[I 2022-09-23 09:59:52.580 ServerApp] Saving file at /scala-spark/Untitled.ipynb
[I 2022-09-23 10:21:55.527 ServerApp] Kernel restarted: 9404ff88-622f-4ba8-86b1-404d648588fc
[MetaKernelApp] ERROR | No such comm target registered: jupyter.widget.control
[MetaKernelApp] WARNING | No such comm: 6eb29ba4-7dab-4314-ace4-88488935840b
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
I’ve also tried this on Jupyter Notebook, removed the kernel and reinstalled it but it was the same 😢

What could I’ve been missing?
Where should I check? Please help.

Thanks
Jennifer

[Bidek56]

@JenniferOH This code seems to work:

import org.apache.spark.sql.SparkSession
val spark = SparkSession.builder.config("spark.sql.legacy.createHiveTableByDefault", "false").master("local[*, 4]").appName("Spark").getOrCreate()

but this repo is dead so I would suggest posting your questions on SO.