Android support

[corrados]

This issue has been migrated from Sourceforge. Created: 2015-12-13
Creator: Volker Fischer

Add support for the Android operating system. The software can already be compiled for Android but the audio interface does not work right now. So the first step is to finish the audio interface implementation. As a second step, all GUI functionality must be checked that it is possible to use it with a touch screen only.

[corrados]

Some progress reported here: https://sourceforge.net/p/llcon/discussion/533517/thread/cfd165e0/#452b/6e3b

[corrados]

See here: https://www.facebook.com/groups/507047599870191/permalink/521878088387142/

[j-santander]

Is there any instructions on how to build this? or any specific branch/forked repository I could refer to? From the previous links there seems to be some implementation working to some extent. I might be able to give a hand, at least testing
Thanks

[corrados]

The code on master is from Simon. But it does not work correctly. Recently, I have started a branch where I do some testing. But also, nothing working yet.

[j-santander]

Ok, thanks. I got it to compile and started testing.

There's an mistake on android/sound.cpp: on CSound::onAudioRead() when converting from incoming float buffer to int16_t:

                pSound->vecsTmpAudioSndCrdStereo[frmNum * oboeStream->getChannelCount() + channelNum] =
                    (short) floatData[frmNum * oboeStream->getChannelCount() + channelNum] * _MAXSHORT;

should be

                pSound->vecsTmpAudioSndCrdStereo[frmNum * oboeStream->getChannelCount() + channelNum] =
                    (int16_t)(floatData[frmNum * oboeStream->getChannelCount() + channelNum] * _MAXSHORT);

otherwise the precedence of the casting operator will make the input always 0.

Changing that if I write to a file the content pSound->vecsTmpAudioSndCrdStereo vector I got a correct signal.

To continue progressing I might need to understand better how the whole things work (I'm not at all familiar with neither your code or even the oboe library). A few doubts:

• Is it assumed that the sound card is always stereo? It seems so, but the oboe library seems to be configured in Mono. Changing that I seemed to started receiving a more-or-less-ok signal on another client. What other assumptions should be made for the signal coming from CSound -> CClient (sampling rate, frame size....)?
• You got oboe setup for float samples. Is there any reason to not use int16_t samples? (we're doing the conversion by hand).
• I'm not sure how it works on the playing side. What I believe I see is that at CClient::ProcessAudioDataIntern after the recording data is processed, the same vector used for the recording data is filled with the playing data received from the network. However, my expectation is that I would be receiving an independent call to CSound::onAudioRead() for the playing channel, which might even indicate that the number of frames available for writing is not the same as the number of frames that we've written (however there seem to be an oboe builder call: setFramesPerCallback() that might help in synchronizing).
• What are the expectations for the CClient -> CSound data?
• How the expected buffer sizes work? (both for playing and recording)

In this last case, I'd go probably for a temporary circular buffer where the onAudioRead() processing of input data will write the recording data left in the buffer and the call of onAudioRead() for the output data will read. It probably would need to be mutex protected (unless we can be sure that the oboe callbacks are always run on the same thread)

[corrados]

There's an mistake on android/sound.cpp: on CSound::onAudioRead() when converting from incoming float buffer to int16_t

I'll merge from a branch soon, where the int 16 type is replaced by float.

Is it assumed that the sound card is always stereo?

Yes, the input/output audio vector for the audio interfaces is assumed to be always stereo. If you have only a mono stream available, just write the same audio data in both stereo channels.

On my branch https://github.com/corrados/jamulus/tree/android_testing I tried to use the full duplex oboe example class. But it is not yet finished. There is function onBothStreamsReady which should have the same size for input and output (which is what Jamulus expects).

[j-santander]

I've got some working code at j-santander@a3558aa
(Unfortunately I was mistakenly working from the master code, not from the android_testing branch.. and only when I was ready to submit I realized of my mistake).

In my tests I can hear other clients and they can hear me. Not sure of the overall quality of sound.

Key modification was to setup the callback only on the input stream and do explicit write to the output stream after processing the incoming data.

I'll try to have a look at the code in the android_testing branch and incorporate it.

Let me know what you think or how should I proceed.

Cheers

[corrados]

My code on the android_testing branch is not working. I was just experimenting a bit. You do not need to use any of that code. If you have something working, then you could create a pull request. I can then test it on my Android phone. But be aware that some minutes ago I applied a big change to the code on master.

[j-santander]

Created PR #661.

Thanks

[ann0see]

How’s the quality? I assume it would be best to try it with a LAN adapter

[j-santander]

How’s the quality? I assume it would be best to try it with a LAN adapter

I might be able to test with more people today.

[dzpex]

i'm interested to test the android code but i do not have experience to compile qt5 project for andoid can you help me?

[corrados]

@j-santander I tested your code and it crashed with "AudioStreamRecord: maybeConvertDeviceData() conversion size 256 too large for buffer 128". I had the same error message on my Android testing branch. This is a known bug in the Oboe library. We have to use a newer version. Just to let you know: I'll now update the Oboe Git submodule to the newest Oboe code now to avoid this issue.

[corrados]

I just updated the Oboe library version but unfortunately, it still crashes, now with a different error message: " InputEventReceiver: Attempted to finish an input event but the input event receiver has already been disposed."

[j-santander]

Oh, I'll test it more on my side. I didn't experience neither of the errors. I'm on Android 11, not sure if it has anything to do.

[corrados]

The UI on Android looks the same as on the PC. No need for QML.

[ann0see]

Ok.
Also, there was no need to change the code, I assume?
On iOS is that I don't see anything but a black screen, so it's probably an iOS only problem.

I hope, my problem can be fixed in some way without having to rewrite the UI for iOS

[corrados]

Qt should give you the same GUI on all platforms.

[nefarius2001]

Hi,

• Is anyone currently working on This?
• had anyone experienced a „known to work“ configuration?

I got the code compiled and the GUI running, but not Audio transmission. Will try a different device as next step...

[nefarius2001]

I have the audio part well working on my tablet (X10605). Some UI issues are still open (mainly using the "back" button)

I am thinking about a source for "nightly" APKs, for those eager to have at least anything.
@corrados Any plans or existing infrastructure to provide "official" builds, except the released installers?

[ann0see]

I am thinking about a source for "nightly" APKs, for those eager to have at least anything.

Any plans or existing infrastructure to provide "official" builds, except the released installers?

GH actions should be able to build it automatically. Once #828 is merged and tested, it should be quite easy to add an action which automatically builds apk files.

[corrados]

That would be creat. Since the Action searches for a "r*" tag, I could easily create beta versions by simply tagging the Git repo with something like r3.6.2beta1.

[ann0see]

Yes, that’s possible but will also move the latest tag.

I tried to setup an Android build environment on a clean Debian 10 VM but didn’t succeed unfortunately.
If anybody succeeds to create a shell script which installs the dependencies and starts the build, it‘s easy to upload the apk. You will then just need to move the .apk file to /deploy in the Shell script and edit the GitHub Action.

[nefarius2001]

I tried to setup an Android build environment on a clean Debian 10 VM but didn’t succeed unfortunately.

I tried with an Ubuntu, but also no success yet.
Weird, setting it up on windows with QtDesigner was easy, now Ubuntu is quite hard...

[nefarius2001]

Until an official build is available, the APK of my pull request can be found here:
http://116.203.80.188:8000/jamulus-024ec6a8402048c13b03a31a13504feed1de3635.apk

[ann0see]

I've now created a project board on this topic, since I think it's a larger one. It would be great if we could close this issue and document needed changes in separate issues and in https://github.com/jamulussoftware/jamulus/projects/6

[ann0see]

Just a quick reminder here: Thanks to nefarius we have working .apk android builds here: https://github.com/jamulussoftware/jamulus/releases/tag/r3_7_0beta3

Once 3.7.0 is out, GitHub actions will automatically build apk files for every new release.

[hoffie]

Once Jamulus 3.7.0 is out, there will be official apk files for every new release (hopefully).

I'm wondering if we want to bless them as "official" already or rather mark them as experimental for now?

I gave them a quick try and while the UI worked, I did not manage to get usable sound out of it on a Moto G4 Play (distortion in short frequency, not the typical Wifi-related unreliability). If this is supposed to work, I should probably open a bug report about it. Until now I assumed that this was some kind of proof of concept...? :)

[ann0see]

Yes. You're right. I just edited my post. Some people have reported "usable" results: #1142

[gilgongo]

Moving this to discussion in the interests of keeping issues to actionable work items.