[BUG][Spark Connector]When using FullFidelity change feed mode, the spark job failed with NullPointerException

[xinlian12]

Issue:

Job aborted due to stage failure: Task 3 in stage 22.0 failed 1 times, most recent failure: Lost task 3.0 in stage 22.0 (TID 79) ```
(d154638bc000009.abqsg4x0r5nu5avvstxro4uyoe.dx.internal.cloudapp.net executor driver): java.lang.NullPointerException 
 at com.azure.cosmos.spark.CosmosRowConverterBase.parseId(CosmosRowConverterBase.scala:708) 
 at com.azure.cosmos.spark.CosmosRowConverterBase.$anonfun$convertStructToChangeFeedSparkDataTypeV1$1(CosmosRowConverterBase.scala:835) 
 at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286) 
 at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36) 
 at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33) 
 at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198) 
 at scala.collection.TraversableLike.map(TraversableLike.scala:286) 
 at scala.collection.TraversableLike.map$(TraversableLike.scala:279) 
 at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:198) 
 at com.azure.cosmos.spark.CosmosRowConverterBase.convertStructToChangeFeedSparkDataTypeV1(CosmosRowConverterBase.scala:831) 
 at com.azure.cosmos.spark.CosmosRowConverterBase.fromObjectNodeToChangeFeedRowV1(CosmosRowConverterBase.scala:93) 
 at com.azure.cosmos.spark.ChangeFeedPartitionReader.com$azure$cosmos$spark$ChangeFeedPartitionReader$$changeFeedItemFactoryMethodV1(ChangeFeedPartitionReader.scala:110) 
 at com.azure.cosmos.spark.ChangeFeedPartitionReader$$anon$2.deserialize(ChangeFeedPartitionReader.scala:156) 
 at com.azure.cosmos.CosmosItemSerializer.deserializeSafe(CosmosItemSerializer.java:95) 
 at com.azure.cosmos.CosmosItemSerializer$1.deserializeSafe(CosmosItemSerializer.java:212) 
 at com.azure.cosmos.implementation.Utils.parse(Utils.java:645) 
 at com.azure.cosmos.implementation.RxDocumentServiceResponse.getQueryResponse(RxDocumentServiceResponse.java:188) 

Possible root cause:

The issue happens for delete operation, when current and previous node are both being null (whether the previous image will be returned could be controlled by a backend flag config)

Fix

• Follow up when the previous image will be returned
• Fix in spark connector code to check whether previous image exists, and handle null cases correctly (for example return null for the properties need to be parsed)

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @kushagraThapar @pjohari-ms @TheovanKraay.