feat(simu): cross-platform connection of host serial ports to simu radio

[nrw505]

Summary of changes:

• Replaces the old way of connecting simulated radio AUX ports to the host's serial ports.

It's a bit convoluted because QSerialPort has to belong to the main event loop's thread, not the simulator's thread. We rely on QT's signal management to let us shunt data between the threads nicely.

[wimalopaan]

Would this allow to connect an external module (e.g. ELRS RX-as-TX bidirectional connected) to the simulator?

[rotorman]

The task was explicitly not to require to communicate within the simulator with physical internal or external RF modules, as the serial communication with them has too tight timing constraints, which typical computer OS, w/o real-time capabilities, would not be able to adhere to: https://edgetx.org/contest_companion_phy_ser/

[nrw505]

Would this allow to connect an external module (e.g. ELRS RX-as-TX bidirectional connected) to the simulator?

If ELRS RX-as-TX currently works on AUX1 or AUX2, then maybe, with absolutely no guarantees of reliability given the impossibility of guaranteeing timing on a general purpose OS. If you're going to try it for god's sake make sure you've got your props off.

[wimalopaan]

I tried this PR unter Linux: selecting a serial device like ttyUSB0, then OK and then reentering the serial device dialog shows not assigned again.

[wimalopaan]

I tried this PR unter Linux: selecting a serial device like ttyUSB0, then OK and then reentering the serial device dialog shows not assigned again.

I tried both the prebuilt AppImage as well as a self-compiled version: in both variants I can't successfully choose a serial device.

[nrw505]

I tried this PR unter Linux: selecting a serial device like ttyUSB0, then OK and then reentering the serial device dialog shows not assigned again.

I tried both the prebuilt AppImage as well as a self-compiled version: in both variants I can't successfully choose a serial device.

Did you see if data flowed between the simulator and the serial port, or did you just check to see if the dropdown had the correct value selected when you opened the serial device dialog again?

I've just fixed the issue where the dialog wouldn't select the currently-connected port when it opens :)

[wimalopaan]

Just tried with yout latest update (both pre-built app-image as well as self-build)

1. Now the device selection is permanent ;-) But not across restarts of the simulator.

2. Unfortunately there is no data flow on the serial device (checked that with LA).

[wimalopaan]

And I want to say that it would be a real cool feature if simu could talk to an ELRS Rx-as-TX. At least sending CRSF_CHANNEL packets should be possible with loose timing requirements. Maybe elrsv3.lua would also be possible (no need for elrsbuddy anymore).

[nrw505]

But not across restarts of the simulator.

It's not expected to persist across restarts of the simulator.

2. Unfortunately there is no data flow on the serial device (checked that with LA).

Interesting, it's working for me (on linux and windows) with a GPS plugged in to the host.

a) How are you configuring the simulated radio?
b) What do you have plugged into the host's serial port?
c) Can you see any errors in the log with HostSerialConnector::serialStart() ?

[nrw505]

And I want to say that it would be a real cool feature if simu could talk to an ELRS Rx-as-TX. At least sending CRSF_CHANNEL packets should be possible with loose timing requirements. Maybe elrsv3.lua would also be possible (no need for elrsbuddy anymore).

It might be cool, but it won't happen in this PR. (I've had a poke through and it looks like you'd need to wait for #4102 to go in)

[wimalopaan]

But not across restarts of the simulator.

It's not expected to persist across restarts of the simulator.

Ok, no problem ;-)

2. Unfortunately there is no data flow on the serial device (checked that with LA).

Interesting, it's working for me (on linux and windows) with a GPS plugged in to the host.

a) How are you configuring the simulated radio? b) What do you have plugged into the host's serial port? c) Can you see any errors in the log with HostSerialConnector::serialStart() ?

Maybe we are talking of different things.
You are reading from the serial port.
My objective is to use the serial port to get the data sent to the external module. So, my hope was to set AUX1 to external module and set in the model config external module to crsf.

[nrw505]

If ELRS RX-as-TX currently works on AUX1 or AUX2

I've just had a poke around, and it doesn't.

Maybe we are talking of different things. You are reading from the serial port. My objective is to use the serial port to get the data sent to the external module. So, my hope was to set AUX1 to external module and set in the model config external module to crsf.

If you set AUX1 to External module and set the model config external module to CRSF then the radio does not actually try to use AUX1 to talk to the external module (at least not on any radio that has an external module). This is why the simulator doesn't send anything over the serial port when you try to do that in the simulator - it's doing the same thing that a real radio does in that situation which is not even initialising AUX1.

[wimalopaan]

If you set AUX1 to External module and set the model config external module to CRSF then the radio does not actually try to use AUX1 to talk to the external module (at least not on any radio that has an external module). This is why the simulator doesn't send anything over the serial port when you try to do that in the simulator - it's doing the same thing that a real radio does in that situation which is not even initialising AUX1.

Mmh, that's a pity!
Then my understanding of the setting AUX1 -> external module was completely bogus!

[nrw505]

Mmh, that's a pity! Then my understanding of the setting AUX1 -> external module was completely bogus!

I suspect that the point of AUX1 -> External module is to allow UART-based modules e.g. multi / CRSF on radios whose module bay doesn't have a UART, just PPM. But I can't say for sure.

[rotorman]

@wimalopaan See: #2562

[wimalopaan]

The baudrate should be user settable I think, and not fixed 9600Bd

[nrw505]

The baudrate should be user settable I think, and not fixed 9600Bd

The baudrate on the host will be set to whatever the radio has set it to.

opentxsimulator.cpp:167 -> opentxsimulator.cpp:144 -> opentxsimulator.cpp:678 -> simulatormainwindow.cpp:160 -> hostserialconnector.cpp:136

[rotorman]

Here the files and sections from previous post with links for easier reading:

https://github.com/nrw505/edgetx/blob/097968dfdc636245d7bdfd01894aca2bdfad02a7/radio/src/targets/simu/opentxsimulator.cpp#L167

https://github.com/nrw505/edgetx/blob/097968dfdc636245d7bdfd01894aca2bdfad02a7/radio/src/targets/simu/opentxsimulator.cpp#L144-L151

https://github.com/nrw505/edgetx/blob/097968dfdc636245d7bdfd01894aca2bdfad02a7/radio/src/targets/simu/opentxsimulator.cpp#L678-L681

https://github.com/nrw505/edgetx/blob/097968dfdc636245d7bdfd01894aca2bdfad02a7/companion/src/simulation/simulatormainwindow.cpp#L160

https://github.com/nrw505/edgetx/blob/097968dfdc636245d7bdfd01894aca2bdfad02a7/companion/src/simulation/hostserialconnector.cpp#L136-L151

[wimalopaan]

The baudrate on the host will be set to whatever the radio has set it to.

Yes, I was aware of that fact but did not find the link to your new code ...
Thanks.

[wimalopaan]

Here the files and sections from previous post with links for easier reading:

Many thanks ;-)

[wimalopaan]

I still wonder about the best way to achieve the goal to attach an ELRS-module to the simu ...

[nrw505]

I still wonder about the best way to achieve the goal to attach an ELRS-module to the simu ...

Something like the approach in this PR but instead of connecting the simulated AUX1/AUX2 to a host serial port, you'd be connecting the simulated internal or external module to the host hardware in some way?

OR you could extend the module handling code to support having a third (or fourth) serial-connected (no module heartbeat for sync, for example) module on AUX1/AUX2 and use this PR to connect it to the host.

[wimalopaan]

Actually I hestitate to use a serial device. I think of a pipe/socket or shared-memory to communicate with a kind of module-simulator ...

[pfeerick]

Sorry it's taken so long to come back to this one.

My intended route to test this was to simply use a USB Uart (i.e. CH340/FTDI) and do a loopback on the TX/RX, thus could test Lua serialRead and serialWrite. However, it has not been co-operating, and so first had to debug my own setup. But now it looks like a bug in the implementation, as it looks like Lua is only sending the last byte of a string? And I don't know if it is able to read anything at all, as it appears to be reading an empty string.

A secondary bug appears to be the serial port is not released/returned when you close the simulator, so you need to restart Companion in order to use the simulator with a serial port on a second run. This was on Windows, not sure about Linux. It would also be nice it if there was message out to the user that serial port was already in use.

Later this week I'll dig out a GPS, and try it with that.

i..e this should have been "EdgeTX"

and this should have been "Eddie"

I used a terminal app just to sanity check "EdgeTX"

This was all on Windows. Linux appears to be the same, but I've not verified after confirming my test setup is indeed correct.

Stupid simple lua test (which was working on TX16S handset with a longer test string):

-- toolName = "TNS|_PR5584 EdgeTX|TNE"

local firstRun = true

local function init()
end

local function run(event)
  if firstRun == true then
    firstRun = false
    -- setSerialBaudrate(9600)
    lcd.clear();
    lcd.drawText(10,10,"PR5584 Serial Loop test")
    lcd.drawText(10,30,"Press ENTER to write to serial")
  end
  
  if event == EVT_VIRTUAL_EXIT then
    firstRun = true
  elseif event == EVT_VIRTUAL_ENTER then
    lcd.clear();
    lcd.drawText(10,50,"Writing to serial")
    serialWrite("EdgeTX")
  end

  local output = serialRead()

  if output ~= nil and output ~= '' then
    lcd.drawText(10,70,output)
    print("serialRead: " .. output)
  end

  return 0
end

return { run=run, init=init }

[nrw505]

A secondary bug appears to be the serial port is not released/returned when you close the simulator, so you need to restart Companion in order to use the simulator with a serial port on a second run.

Whoops, this is what happens when I test with simulator211 not companion211. Fixed in latest push.

re: the LUA loopback test: turns out that QByteArray::fromRawData doesn't copy the data it's given but points directly to it. And with the timing for the the GPS driver it just happened to be that the data got sent before the buffer got overwritten. But when sending data from LUA, the buffer got overwritten for each character while the queue of QT signals grew. Also fixed in latest push, at least that's what it looks like on my machine :)

[pfeerick]

Looking good so far... both fixes working fine here on Linux :)

edit: and Windows

[nrw505]

Now just need to find someone with a mac 😆

[pfeerick]

Welp... github blocked my email reply...

Got someone in mind already, just need to send him a reminder to test this as we were discussing this just before EdgeTX fest so would have been forgotten in the chaos of getting ready for that.