Connection lost issue when calling long running function

[megarg]

Question

In NiceGUI, I've encountered the following issue:
In the main thread, I need to call a function—let's name it funcA. Inside funcA, there are numerous operations involving reading and processing files, as well as calling OpenAI and handling embedding-related tasks. Consequently, funcA is time-consuming. Upon completion of funcA in the main thread, we need to display its returned result. However, directly invoking funcA, due to its lengthy execution time, leads to a "Connection Issue." While funcA is running, my NiceGUI page becomes unresponsive, displaying the message "Connection lost. Trying to Connect."

I have considered two solutions to address this problem:

### Solution 1: Utilize Multi-Threading
I create a new thread for funcA and use ui.timer to check whether the result has been set. If the result has been set, then I will simply display the result; otherwise, I will wait. I tested this solution, and it works.

### Solution 2: Asynchronous Programming
Currently, funcA operates as a synchronous function. I propose changing the entire process to asynchronous mode. However, this might pose challenges, especially when third-party libraries used in funcA do not support asynchronous operations.

I would like to determine which solution is preferable, or if there are any other recommended approaches to the issue described above.

[rodja]

Please read https://github.com/zauberzeug/nicegui/wiki/FAQs#why-is-my-long-running-function-blocking-ui-updates. It should answer your question pretty well.

[megarg]

Thanks rodja. i understand the reason why this is happening though I am trying to figure out the best/simple solution. Looks like the solution proposed is solution#2 above which has challenges if we want to leverage 3rd party libraries which do not support asycn ops.

However, I also got another solution which I want to get feedback from community, if they see any problem with this:

### Solution3: Use the asyncio library

The asyncio library provides a run_in_executor function, which allows running synchronous code within an asynchronous function.
The code I used is as below:

# asyncio calling code
loop = asyncio.get_event_loop()
await loop.run_in_executor(None, long_running_operation)

Additionally, I've used a ui.timer to monitor whether the result has been determined, similar to Solution 1 mentioned above
The only issue I've encountered with Solution 3 arises depending on the placement of the ui.timer relative to the asyncio calling code.

When the ui.timer is positioned above the asyncio calling code, everything functions smoothly.
However, placing the ui.timer below the asyncio calling code leads to the "500 server timeout error" and I'm not sure why this occurs.

This give me concerns if solution#3 has any unintended consequences that I am not grasping.
Would appreciate communities help on the best & simplest way to do this.

[megarg]

Thank you for the information.
Even if I remove ui.timer, I still encounter this Timeout issue when using Solution3.
This problem can be replicated using the test code provided below. When I click the Test Link in the main page, I would encounter the Timeout Error.

main.py

from nicegui import app, ui, events
import os
import function_test

@ui.page("/")
def index_page() -> None:
    ui.markdown("**This is Homepage.**")
    ui.link("Test Link", "/test")

function_test.create()
ui.run(title='Test', port=8001)

function_test.py

from nicegui import ui
import time
import asyncio

def create() -> None:
    @ui.page("/test")
    async def func1():
        ui.label("This is Partner Onboarding File Page.")
        status = ui.label()
        status.set_text("This is inital")
        def sync_func(): 
            time.sleep(10)
            status.set_text("This is after func2")    
        loop = asyncio.get_event_loop()
        await loop.run_in_executor(None, sync_func)
        status.set_text("This is final")

[falkoschindler]

The problem is that you're awaiting the executor, while blocking the page creation. You'd need to move the async call to a background task like this:

def create() -> None:
    @ui.page("/test")
    def func1():
        ui.label("This is Partner Onboarding File Page.")
        status = ui.label()
        status.set_text("This is inital")

        def sync_func():
            time.sleep(5)
            status.set_text("This is after func2")

        async def load_data():
            loop = asyncio.get_event_loop()
            await loop.run_in_executor(None, sync_func)
            status.set_text("This is final")

        background_tasks.create(load_data())

An alternative would be to await the client connection explicitly and await the executor afterwards. This allows the page creation to finish before starting long-running tasks:

def create() -> None:
    @ui.page("/test")
    async def func1():
        ui.label("This is Partner Onboarding File Page.")
        status = ui.label()
        status.set_text("This is inital")

        await ui.context.client.connected()

        def sync_func():
            time.sleep(5)
            status.set_text("This is after func2")

        loop = asyncio.get_event_loop()
        await loop.run_in_executor(None, sync_func)
        status.set_text("This is final")

[megarg]

Thanks for providing the solutions.
Regarding the first solution, could I inquire about the origin of background_tasks? I am new to NiceGUI, and also I didn't notice any call to sync_func function within the first solution.

I have tried the second solution, and it functions correctly. Previously, I used the below code to avoid the Timeout Error. It seems ui.timer function before the await will get the page created first. Please correct me if I understand it wrong. Thanks.

def create() -> None:
    @ui.page("/test")
    async def func1():
        ui.label("This is Partner Onboarding File Page.")
        status = ui.label()
        status.set_text("This is inital")
        def sync_func(): 
            time.sleep(10)
            status.set_text("This is after func2")
        def check_status():
            pass
        ui.timer(1, check_status)   
        loop = asyncio.get_event_loop()
        await loop.run_in_executor(None, sync_func)
        status.set_text("This is final") 

[falkoschindler]

Sorry, I should have mentioned the import:

from nicegui import background_tasks

Oh and I forgot to copy&paste the run_in_executor line which actually calls sync_func. I updated my code above.

Yes, ui.timer is also a common method to trigger a background task. But make sure to set once=True to avoid repeating the task every 1 second:

ui.timer(0, load_data, once=True)

[megarg]

Thank you very much for all the solutions provided. They all work now.
Including the multi-threading solution. among these, which solution would you recommend as the best?

[falkoschindler]

Personally I like ui.timer for its simplicity, but background_tasks.create is a bit more explicit about its purpose. Technically they are almost equivalent for this usecase.

[husain-akridata]

A hack that worked for me, was to just increase the default timeout duration when that pop-up shows on the UI using reconnect_timeout parameter.
ui.run(reload=False, port=8080, host='0.0.0.0', reconnect_timeout=60)

[evnchn]

Sidenote: there is a performance/security consequence to this. When the timeout is high, it is easy for people to perform a DoS by spamming requests to the server.

The issue stems from Garbage Collection, not NiceGUI, and could probably apply to other libraries.

The core of the issue, I have found GC time to grow with the amount of left-open connections.

To speed test my NiceGUI website, I simply send ~200 requests to the page (single-threaded because I'm lazy) and time the response time. I found that every 20 requests or so, there would be a spike, and that spike would grow with the later requests.

Normally, with the low timeout, the amount of client connections left in memory would be purged rapidly, and GC takes a short while (maybe 200ms).

With a high timeout, in my testing I found that the high amount of connections in memory made GC unacceptably slow. We're talking seconds.

Bear in mind that I was using a single-threaded script to make many requests to the server, and it was not at all optimized for maximum request/sec.

Also, it'll be worse if you follow any kind of GC tweak where allocs, gen1 or gen2 is increased, especially the one which claims a 20% speed improvement (intentionally not linking it here...)