Get from server immediately after a document is updated gives old cached data when snapshot is open

[rorystephenson]

Description

NOTE: I have been directed to share this here after opening an issue in the flutter firebase sdk.

The short version is that when a snapshot listener for a document is already open and the app calls get(GetOptions(source: Source.server)) for that document, instead of fetching fresh data from the server cloud_firestore is returning the most recently fetched snapshot data. Moreover the isFromCache and hasPendingWrites metadata values are both false. This means we do not have a way to fetch fresh data from firestore when we know that there has been updated.

I believe this is a variant of this issue: firebase/flutterfire#10153. The difference being that our write is done by the backend. We have verified that the backend waits for the transaction to complete and that the mobile app waits for the backend to respond before calling get().

The linked issue has been closed as 'will not fix' using firebase/firebase-js-sdk#6915 (comment) as justification however this seems incorrect for the following reasons:

• Firestore goes to great efforts to be a strongly consistent database that guarantees that reads which happen after a transaction is committed will return the latest data. What is the point of such a guarantee if the client short-cuts it by making it impossible to actually force a fetch of fresh data from the database.
• The documentation for Source.server in GetOptions makes no reference to this behaviour.
• In fact that same documentation emphasises that the data is being fetched from the server by mentioning that failure to fetch the data will result in an error. In my testing this does not happen, I can turn the internet off and the get() request returns the latest data from the snapshot listener.
• The only workaround would seem to be stopping snapshots whilst performing the get(). In a large app which may listen to documents in various places for various reasons this is not feasible and amounts to maintaining a global state somewhere to make sure that snapshots are off before a given document is fetched via get() to be sure that the returned data is up to date.
• Even if we didn't know that the data has just been changed for our purposes we still need to get the latest version of the data as we will be updating it via the backend API and we don't want to do that from stale data. Note that making the write from the frontend is not feasible as the backend needs to do checks on the data which the frontend cannot make.

Reproducing the issue

Same as firebase/flutterfire#10153

In our case the transaction is run from a backend system but the outcome is the same.

Firebase SDK Version

11.8.0

Xcode Version

16.3

Installation Method

CocoaPods

Firebase Product(s)

Firestore

Targeted Platforms

iOS

Relevant Log Output

If using Swift Package Manager, the project's Package.resolved

No response

If using CocoaPods, the project's Podfile.lock

Expand Podfile.lock snippet (NOTE: Irrelevant parts removed to fix issue char limit)

PODS:
  - cloud_firestore (5.6.5):
    - Firebase/Firestore (= 11.8.0)
    - firebase_core
    - Flutter
  - connectivity_plus (0.0.1):
    - Flutter
  - CTNotificationContent (0.2.7)
  - CTNotificationService (0.1.7)
  - device_info_plus (0.0.1):
    - Flutter
  - facebook_app_events (0.0.1):
    - FBAudienceNetwork (= 6.16)
    - FBSDKCoreKit (~> 17.0)
    - Flutter
  - FBAEMKit (17.4.0):
    - FBSDKCoreKit_Basics (= 17.4.0)
  - FBAudienceNetwork (6.16.0)
  - FBSDKCoreKit (17.4.0):
    - FBAEMKit (= 17.4.0)
    - FBSDKCoreKit_Basics (= 17.4.0)
  - FBSDKCoreKit_Basics (17.4.0)
  - Firebase/Analytics (11.8.0):
    - Firebase/Core
  - Firebase/Auth (11.8.0):
    - Firebase/CoreOnly
    - FirebaseAuth (~> 11.8.0)
  - Firebase/Core (11.8.0):
    - Firebase/CoreOnly
    - FirebaseAnalytics (~> 11.8.0)
  - Firebase/CoreOnly (11.8.0):
    - FirebaseCore (~> 11.8.0)
  - Firebase/Crashlytics (11.8.0):
    - Firebase/CoreOnly
    - FirebaseCrashlytics (~> 11.8.0)
  - Firebase/Firestore (11.8.0):
    - Firebase/CoreOnly
    - FirebaseFirestore (~> 11.8.0)
  - Firebase/Messaging (11.8.0):
    - Firebase/CoreOnly
    - FirebaseMessaging (~> 11.8.0)
  - Firebase/RemoteConfig (11.8.0):
    - Firebase/CoreOnly
    - FirebaseRemoteConfig (~> 11.8.0)
  - firebase_analytics (11.4.4):
    - Firebase/Analytics (= 11.8.0)
    - firebase_core
    - Flutter
  - firebase_app_check (0.3.2-4):
    - Firebase/CoreOnly (~> 11.8.0)
    - firebase_core
    - FirebaseAppCheck (~> 11.8.0)
    - Flutter
  - firebase_auth (5.5.1):
    - Firebase/Auth (= 11.8.0)
    - firebase_core
    - Flutter
  - firebase_core (3.12.1):
    - Firebase/CoreOnly (= 11.8.0)
    - Flutter
  - firebase_crashlytics (4.3.4):
    - Firebase/Crashlytics (= 11.8.0)
    - firebase_core
    - Flutter
  - firebase_messaging (15.2.4):
    - Firebase/Messaging (= 11.8.0)
    - firebase_core
    - Flutter
  - firebase_remote_config (5.4.2):
    - Firebase/RemoteConfig (= 11.8.0)
    - firebase_core
    - Flutter
  - FirebaseABTesting (11.8.0):
    - FirebaseCore (~> 11.8.0)
  - FirebaseAnalytics (11.8.0):
    - FirebaseAnalytics/AdIdSupport (= 11.8.0)
    - FirebaseCore (~> 11.8.0)
    - FirebaseInstallations (~> 11.0)
    - GoogleUtilities/AppDelegateSwizzler (~> 8.0)
    - GoogleUtilities/MethodSwizzler (~> 8.0)
    - GoogleUtilities/Network (~> 8.0)
    - "GoogleUtilities/NSData+zlib (~> 8.0)"
    - nanopb (~> 3.30910.0)
  - FirebaseAnalytics/AdIdSupport (11.8.0):
    - FirebaseCore (~> 11.8.0)
    - FirebaseInstallations (~> 11.0)
    - GoogleAppMeasurement (= 11.8.0)
    - GoogleUtilities/AppDelegateSwizzler (~> 8.0)
    - GoogleUtilities/MethodSwizzler (~> 8.0)
    - GoogleUtilities/Network (~> 8.0)
    - "GoogleUtilities/NSData+zlib (~> 8.0)"
    - nanopb (~> 3.30910.0)
  - FirebaseAppCheck (11.8.0):
    - AppCheckCore (~> 11.0)
    - FirebaseAppCheckInterop (~> 11.0)
    - FirebaseCore (~> 11.8.0)
    - GoogleUtilities/Environment (~> 8.0)
    - GoogleUtilities/UserDefaults (~> 8.0)
  - FirebaseAppCheckInterop (11.10.0)
  - FirebaseAuth (11.8.1):
    - FirebaseAppCheckInterop (~> 11.0)
    - FirebaseAuthInterop (~> 11.0)
    - FirebaseCore (~> 11.8.0)
    - FirebaseCoreExtension (~> 11.8.0)
    - GoogleUtilities/AppDelegateSwizzler (~> 8.0)
    - GoogleUtilities/Environment (~> 8.0)
    - GTMSessionFetcher/Core (< 5.0, >= 3.4)
    - RecaptchaInterop (~> 100.0)
  - FirebaseAuthInterop (11.10.0)
  - FirebaseCore (11.8.1):
    - FirebaseCoreInternal (~> 11.8.0)
    - GoogleUtilities/Environment (~> 8.0)
    - GoogleUtilities/Logger (~> 8.0)
  - FirebaseCoreExtension (11.8.0):
    - FirebaseCore (~> 11.8.0)
  - FirebaseCoreInternal (11.8.0):
    - "GoogleUtilities/NSData+zlib (~> 8.0)"
  - FirebaseCrashlytics (11.8.0):
    - FirebaseCore (~> 11.8.0)
    - FirebaseInstallations (~> 11.0)
    - FirebaseRemoteConfigInterop (~> 11.0)
    - FirebaseSessions (~> 11.0)
    - GoogleDataTransport (~> 10.0)
    - GoogleUtilities/Environment (~> 8.0)
    - nanopb (~> 3.30910.0)
    - PromisesObjC (~> 2.4)
  - FirebaseFirestore (11.8.0):
    - FirebaseCore (~> 11.8.0)
    - FirebaseCoreExtension (~> 11.8.0)
    - FirebaseFirestoreInternal (= 11.8.0)
    - FirebaseSharedSwift (~> 11.0)
  - FirebaseFirestoreInternal (11.8.0):
    - abseil/algorithm (~> 1.20240116.1)
    - abseil/base (~> 1.20240116.1)
    - abseil/container/flat_hash_map (~> 1.20240116.1)
    - abseil/memory (~> 1.20240116.1)
    - abseil/meta (~> 1.20240116.1)
    - abseil/strings/strings (~> 1.20240116.1)
    - abseil/time (~> 1.20240116.1)
    - abseil/types (~> 1.20240116.1)
    - FirebaseAppCheckInterop (~> 11.0)
    - FirebaseCore (~> 11.8.0)
    - "gRPC-C++ (~> 1.65.0)"
    - gRPC-Core (~> 1.65.0)
    - leveldb-library (~> 1.22)
    - nanopb (~> 3.30910.0)
  - FirebaseInstallations (11.8.0):
    - FirebaseCore (~> 11.8.0)
    - GoogleUtilities/Environment (~> 8.0)
    - GoogleUtilities/UserDefaults (~> 8.0)
    - PromisesObjC (~> 2.4)
  - FirebaseMessaging (11.8.0):
    - FirebaseCore (~> 11.8.0)
    - FirebaseInstallations (~> 11.0)
    - GoogleDataTransport (~> 10.0)
    - GoogleUtilities/AppDelegateSwizzler (~> 8.0)
    - GoogleUtilities/Environment (~> 8.0)
    - GoogleUtilities/Reachability (~> 8.0)
    - GoogleUtilities/UserDefaults (~> 8.0)
    - nanopb (~> 3.30910.0)
  - FirebaseRemoteConfig (11.8.0):
    - FirebaseABTesting (~> 11.0)
    - FirebaseCore (~> 11.8.0)
    - FirebaseInstallations (~> 11.0)
    - FirebaseRemoteConfigInterop (~> 11.0)
    - FirebaseSharedSwift (~> 11.0)
    - GoogleUtilities/Environment (~> 8.0)
    - "GoogleUtilities/NSData+zlib (~> 8.0)"
  - FirebaseRemoteConfigInterop (11.10.0)
  - FirebaseSessions (11.8.0):
    - FirebaseCore (~> 11.8.0)
    - FirebaseCoreExtension (~> 11.8.0)
    - FirebaseInstallations (~> 11.0)
    - GoogleDataTransport (~> 10.0)
    - GoogleUtilities/Environment (~> 8.0)
    - GoogleUtilities/UserDefaults (~> 8.0)
    - nanopb (~> 3.30910.0)
    - PromisesSwift (~> 2.1)
  - FirebaseSharedSwift (11.10.0)

SPEC CHECKSUMS:
  cloud_firestore: e61acbf808607d2c88ee32b00cd3aec027b38b3c
  Firebase: d80354ed7f6df5f9aca55e9eb47cc4b634735eaf
  firebase_analytics: 4e93dbe66872104d28ae9750fec1800e1fd66858
  firebase_app_check: cb20bddfd2664003e55c1dc29545cd4a6512ee34
  firebase_auth: 9ebbd83276bf977dea1c74dfc199acf9d2a2f42f
  firebase_core: 8d552814f6c01ccde5d88939fced4ec26f2f5510
  firebase_crashlytics: 05519be6b623981a77fe54fb52e6061956cb6047
  firebase_messaging: 8b96a4f09841c15a16b96973ef5c3dcfc1a064e4
  firebase_remote_config: ca499f96ddbcc63db863b941bf9f5e6e9a81ceba
  FirebaseABTesting: 7d6eee42b9137541eac2610e5fea3568d956707a
  FirebaseAnalytics: 4fd42def128146e24e480e89f310e3d8534ea42b
  FirebaseAppCheck: f6648d6d2b321ecf94cf72f6737fc68d4fddc010
  FirebaseAppCheckInterop: 9664c858489710f682766ef54e2b6741d3b62070
  FirebaseAuth: ad59a1a7b161e75f74c39f70179d2482d40e2737
  FirebaseAuthInterop: 01a804fb074424fd58b92dd50dd0272277199356
  FirebaseCore: 99fe0c4b44a39f37d99e6404e02009d2db5d718d
  FirebaseCoreExtension: 3d3f2017a00d06e09ab4ebe065391b0bb642565e
  FirebaseCoreInternal: df24ce5af28864660ecbd13596fc8dd3a8c34629
  FirebaseCrashlytics: a1102c035f18d5dd94a5969ee439c526d0c9e313
  FirebaseFirestore: 563a4ab1a65e2858f05e150bb4c31b0f8f79248b
  FirebaseFirestoreInternal: 8c5921c360a70e447bfeefb245f450e8b50e750b
  FirebaseInstallations: 6c963bd2a86aca0481eef4f48f5a4df783ae5917
  FirebaseMessaging: 487b634ccdf6f7b7ff180fdcb2a9935490f764e8
  FirebaseRemoteConfig: f63724461fd97f0d62f20021314b59388f3e8ef8
  FirebaseRemoteConfigInterop: 7c9a9c65eff32cbb0f7bf8d18140612ad57dfcc6
  FirebaseSessions: c4d40a97f88f9eaff2834d61b4fea0a522d62123
  FirebaseSharedSwift: 1baacae75939499b5def867cbe34129464536a38

COCOAPODS: 1.16.2

[google-oss-bot]

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

[rizafran]

Thanks for the report, @rorystephenson. Could you share the steps in reproducing the issue or code snippet?

[MarkDuckworth]

This was also reported in the Android repo, see firebase/firebase-android-sdk#6974. Repro steps were provided there. @milaGGL has been investigating and posted updates there.