Summary of findings (both implementation and documentation)

[drummer1154]

Prefaces:
I: I am the drummer in a 3 peace advanced jazz band "fuTunes" and a retired SW developer/project manager, located in Munich, Germany. I have also been working as a sound engineer (FoH/Monitor/Studio).
II: Currently I am struggling for our band to get a local Jamulus server running on a DS-Lite connection so the work on this Issue deducts me from essential research for our band.
III: The more I dig into Jamulus topics the more I question myself: Are we going the right way to support the community (also regarding the current Corona situation)? From the (few) threads I was able to follow I see two separate movements: 1) Keep it simple and work as expected => optimize the current implementation without caring too much for documentation ("bottom-up"). 2) Update the documentation to match the implementation and improve the implementation only after it is clear how the pieces should work together and what to change where and why ("top-down"). Personally I support 2) but I admit that it will take more time in the beginning.
IV: For me "local" always means "related to the (physical) location of my equipment (HW and SW)".

Referring to https://jamulus.io/wiki/Software-Manual ["the Manual"], to http://llcon.sourceforge.net/PerformingBandRehearsalsontheInternetWithJamulus.pdf ["the Case Study", February 2015], and to #64 (comment) ["the Audio Diagram", May 2020], there are features which from my PoV do not work as documented or do not work as expected. I list the issues which have caused the biggest waste of time for me so far. Everything was tested with Jamulus V3.6.1. Once the root causes are clear, we can split up this summary into separate issues.

Signal Flow Diagram: In the Case Study we find "Figure 2: Simplified Jamulus block diagram" which shows the data processing but cannot be taken as an audio flow diagram. It shows the input from "other client" but does not show the server output to other clients and the text below says "This mix is ... transmitted to all connected clients ..." suggesting that there is only a single mix which is distributed to all clients - I was completely misled by this when I started working with Jamulus. Meanwhile I was pointed to the Audio Diagram which explains the audio flow but still suffers from missing information (see my points below).

• Feature request: Provide two up-to-date diagrams to be taken up into the Manual (maybe into a new "Technical Reference" section): 1) data flow and 2) audio signal flow. Rationale: "A picture paints a thousand words" - Jamulus is too complex to be quickly understood by simply reading the Manual, and the currently required "trial and error" method is a waste of time.

I believe that most every musician will be able to read an audio signal flow diagram. From my PoV, 1) could be easily achieved by updating the block diagram from the Case Study, but 2) is more complicated because - at least for me - there are a lot of things which are not working as expected (see my points below). What is also required: Select a drawing tool - are there any preferences? Personally I would start with Visio.

Input level: The Manual: "This shows the level of the two stereo channels for your audio input." According to the Audio Diagram (congruent to the Manual), this display is expected to be always working. Issues:

• Input level is not displayed while the local client is not connected to a Jamulus server.
• There is only one stereo channel, not two.
• In the Manual the wording "for your audio input" is ambiguous - does it mean "from your Jamulus audio input" (i.e. the output of your audio interface to the input of the Jamulus client) or "received by the Jamulus server" (i.e. the output of the Jamulus client sent to the Jamulus server if there is an active connection)? [I think the current implementation is the latter - OK from my PoV.]
• Feature request: Input level shall be displayed permanently. Rationale: The generation of the local audio to be eventually sent to the Jamulus server is influenced by a huge number of parameters (audio interface, drivers, OS settings, Jamulus settings, maybe more). A permanent monitoring of this signal would drastically reduce the time required to achieve a correct setting of all local parameters without the need to additionally set up a local Jamulus server (on the same computer or on another computer in your LAN, while you are not yet 100% familiar with the SW). The Manual then shall read "This shows the level of the stereo channel which will be sent to the Jamulus server."

Mute Myself button: The Manual: "Cuts your audio stream to the server so that you will be able to hear yourself and see your own input levels, but other musicians will not." According to the Audio Diagram, the local user's signal bypasses the Jamulus server and is fed back to the local audio interface directly (together with the mix coming from the server). If the Audio Diagram is correct, I see the following issues:

• Why is this working only while connected to the server - the server is bypassed anyway.
• The signal remains completely on the local computer, not affected by any network delay (verified to the extent I can w/o disturbing remote musicians), so it does not make much sense to mix-in the server output as the "golden principle" to "always listen to your signal from the server" is violated.
• The Audio Diagram cannot be correct because the fader and mute button (but not the pan) within the "Server audio mixer" affect the signal (see also Honour own fader and Mute button in “Mute Myself” #148 but there I do not see a solution for my finding).
• Feature request: 1) Split "Mute Myself" into two separate functions: 1a) "Local Audio Loop" (Client): Enable to listen only to your own signal "as if it would come from the server" but without any network delay, i.e. feed the signal which would go to the server directly into the local client's input jitter buffer and do not send any audio to the server. In this mode the "Server audio mixer" display shall be emptied (as it is now while not connected but of course the "Input level" needs to be displayed - see above). 1b) "Mute me for others" (Server): Introduce a new info bit to inform the the server which will then set 1b1) your own signal to Mute and Solo *) and 1b2) the signal of all other users to [Edit] Mute [/Edit] for your personal mix. According to the Audio Diagram, this would be sufficient to create the correct mixes without the need to implement new mixing methods in the server and the GUI remains unchanged. 2) Remove the red bar "Muted (...)" on top of the "Server audio mixer" which currently causes the single channel strips to jump. The status of each user's channel for their personal mix would be displayed correctly anyway. 3) Update the Audio Diagram accordingly.

In the GUI of course a new button is required. I think the input level display can be shrunk to get the space for the new button. Local Audio Loop and Mute me for others shall be mutual exclusive (one activated releases the other).

[Edit] *) "Mute and Solo" does not work as you would expect from the Audio Diagram - Mute takes precedence :-( So this needs to be also changed so that for your own client Solo takes precedence.

• Proposal by dakhubgit (https://github.com/jamulussoftware/jamuluswebsite/issues/174#issuecomment-744040593):
  "Ok, what I consider useful for startup is, assuming you are unconnected, to behave as follows:
  a) as if "mute myself" is active, namely route audio to the mixer directly rather than through a server
  b) add a mixer strip for oneself as if one were connected on one's own to a server
  That allows playing around with quite a bit of things in a private sandbox before even going online."
• My comment: The behavior is exactly what I want, but I think the GUI should be visually different in the two connection states - if you see a fader, you may think you are connected while you are not. The input level meter is sufficient to check for input overload (once my proposal above is implemented), the playback volume can be set with the volume control of the audio interface.

Reverb effect: The Manual: "Reverb can be applied to one local mono audio channel or to both channels in stereo mode ...". This together with the following description ("... a microphone signal is fed in to the right audio channel of the sound card and a reverb effect needs to be applied ...") suggests that in mono mode, the reverb is applied only to the channel selected by the radio buttons. Issues:

• The reverb always generates a stereo signal, also if "Settings/Misc/Audio Channels" is set to Mono. Is this the designed and wanted behavior?
• In the GUI it is not clear that the reverb directly belongs to the Input.
• Feature request: 1) Update The Manual and the Audio Diagram to reflect the implementation. 2) GUI: Move the vertical separator which is currently on the left side of the reverb slider to the right to clearly show the relationship.

Local audio pan / balance control: To me it is completely unclear how this is supposed to work, see also #353, and I think it is wrong what I wrote in #778 (comment) (I will edit this comment after things have been clarified). Issues:

• If the vertical pan affects the final signal sent to the client, then it is related to the input and this not clearly seen in the GUI - same as with Reverb effect above, so moving the vertical separator to the right side would remove this problem.
• There are two separate elements labelled "Pan": 1) The Local audio pan / balance control with the vertical slider and 2) the rotary pan above the channel fader (which - for some reason - disappears when I switch "Settings/Misc/Audio Channels" to Mono).
• Depending on "Settings/Misc/Audio Channels" the effect of the vertical slider is different:

1. When I input a mono signal (i.e. I feed audio only into the left or right input of the audio interface) then 1a) Mono: The mix is a stereo signal with the input signal equally distributed to the left and right channel (centered), the vertical pan just affects the level and there is no possibility to pan the signal => useless. 1b) Mono in/Stereo out: The mix is a stereo signal, the rotary pan above the fader works as expected, but the vertical pan just affects the level => useless. 1c) Stereo: The mix is a stereo signal with the input signal present only at the channel where it is input. Both the vertical and the rotary pans just affect the level => both are useless.
2. When I input a stereo signal then 2a) Mono: The mix is a stereo signal with the left and right input signals mixed together and then equally distributed to the left and right channel (centered). The vertical pan attenuates the left or right input signal prior to being mixed together, but the reading above the vertical fader (top position: R-50, bottom position: L-50) has the channels exchanged (bug, I think). 2b) Mono in/Stereo out: The mix is a stereo signal, the vertical pan attenuates the left or right input signal prior to being mixed together (same bug regarding the channel reading) and the rotary pan works as expected. 2c) Stereo: The mix is a stereo signal, the vertical pan attenuates the left or right input signal (same bug regarding the channel reading) and the rotary pan works as expected.

• Conclusion from my tests above:

1. The vertical slider labelled "Pan" is not a pan but an attenuator for the left and right input signals. Therefore a better label would maybe be "LR Attenuator", the channel attenuation reading is exchanged and "Center" in the middle position is wrong (it should simply read 0).
2. Why is the rotary pan not available if "Settings/Misc/Audio Channels" is set to Mono?

• I will not make any concrete request for implementation and/or documentation update unless it has been clarified how the "Pan" features are supposed to function. Maybe it is possible to change at least the GUI so as to reflect the relationship of Local audio pan / balance control and Reverb effect to the lefthand Input section (just move the vertical bar), change the label of the vertical slider from "Pan" to "LR Attenuator" and correct the attenuation number display.

So far for the documentation of the quirks I have found.
@corrados @gilgongo @dakhubgit @WolfganP @Matzix: Could you please kindly review what I have written and direct me how to continue, thanks.

[gilgongo]

Thanks for this - very interesting.

Provide two up-to-date diagrams to be taken up into the Manual ... Jamulus is too complex to be quickly understood by simply reading the Manual

I think this depends entirely on what you mean by "understand". There's no problem creating such diagrams and documentation if we intend to keep that away from normal users :-) But if you mean you think Jamulus can't be used without these things, then I'm afraid that's objectively incorrect.

BTW quite who would create such diagrams, I don't know. I did a high-level one (and it's on the Getting Started page) but we concluded as you did after a while that it wasn't really very helpful so we demoted it to the bottom of the page. I doubt anyone actually looks at it. You are the first person to ask about it as far as I know. Also BTW I used a free online tool, the "source" file for which I posted on a ticket here if that's any help. #64 (comment)

The vertical slider labelled "Pan" is not a pan but an attenuator for the left and right input signals.

Could we call the Local Pan slider "Local L/R Volume" with labels "+ 0 +" vertically then? Non-zero numbers seems fairly pointless to me.

Why is the rotary pan not available if "Settings/Misc/Audio Channels" is set to Mono?

I've wondered this too. Perhaps others will know.

Side note: At least one of these confusions is to do with a mis-match between the user's understanding of a mixing desk and associated terms (eg "panning") and the way Jamulus actually works.

[drummer1154]

@gilgongo

I think this depends entirely on what you mean by "understand".

By "understand" I mean to be able to use the SW intuitively for its intended purpose without the need to RTFM: To make music where the musicians are at home, located away from each other. OK, so you find Jamulus, you install it (not to mention the safety warnings), you open the GUI and then? On first glance it looks much like any other audio recording SW, only there are no channel strips, only a confusing "Pan" strip in the middle and an empty "Server" area to the right (I think I do not need to explain the initial client look any further).

The problems start directly after installing Jamulus client: It is not obvious that - conversely to any audio recording SW I am aware of - you will not be able to hear your own instrument. This is due to the client/server architecture which appears to be brilliant to me from a technical PoV and makes the platform performant. BUT:

Do you really want to create a user guide which forces every newbee to install not only the client but also the server? Does their computer have enough computing power to allow both client and server in parallel? If not, then what? E.g. myself as a "DOS / Windows Veteran" am forced to use an iMac (given to me by my bandmate) because my bandmates have been using this platform already for years, and in the first Corona lockdown we successfully started offline recording with Garageband (GB) which is included in the MacOS and imposes no additional cost. For Jamulus I need to say that I spent hours (if not days) to drill down the "Audio MIDI Setup" settings until I succeeded in hearing my instrument (mostly caused by samle rate and unclear settings in Jamulus Settings/Soundcard/Device). No such problem with GB: Just specify the interface and that's it - the USB LED on my Lexicon Omega ist permanently on (according to the Owner’s Manual: "the application" has configured the interface correctly), conversely to Jamulus where it fast blinks all the time even if it works somehow (and Volker told me it is the same thing on his setup).

So if there would be an up-to-date audio flow diagram, if you run into such difficulties, you would be able to identify the root cause more easily.

who would create such diagrams

I have already said that I am willing to create them - if we agree on a common tool. OK, I will check out https://online.visual-paradigm.com/.

Could we call the Local Pan slider "Local L/R Volume" with labels "+ 0 +" vertically then? Non-zero numbers seems fairly pointless to me.

Sorry but it is not the "Volume". It is an Attenuation (at least what I was able to get from my time consuming tests).

[drummer1154]

@ann0see I do not agreee to your change of my title. Please revert back to my initial title and discuss a title change here until we agree. How come you override my PoV???

[ann0see]

Sorry about that. Usually issues here in the documentation repo have a language tag.

Since it will refer to the English documentation first, I added [en].

Since (at least some parts) some changes might refer to the manual, I added manual. But I can revert it.

Edit: done. I didn't want to insult you. Sorry.

[dakhubgit]

Ok, what I consider useful for startup is, assuming you are unconnected, to behave as follows:
a) as if "mute myself" is active, namely route audio to the mixer directly rather than through a server
b) add a mixer strip for oneself as if one were connected on one's own to a server
That allows playing around with quite a bit of things in a private sandbox before even going online.

[drummer1154]

@ann0see OK, NP, execution accepted. As this is my 1st Issue created here, please give me a hint regarding the language tag (didn't see any yet) and tell me why you are thinking I am complaining only about documentation ("Manual") where I explicitly stated both Implementation AND Documentation.

Every input is welcome!

-hg

[gilgongo]

By "understand" I mean to be able to use the SW intuitively for its intended purpose without the need to RTFM

So this isn't about the docs, it's about the UI?

Do you really want to create a user guide which forces every newbee to install not only the client but also the server?

We don't. Do we?

[drummer1154]

So this isn't about the docs, it's about the UI?

Both, if you would please read everything.

Do you really want to create a user guide which forces every newbee to install not only the client but also the server?

We don't. Do we?

No, of course not. But you will be forced to if you want to keep the audience satisfied.
Edit ... with the current implemetation and documentation.

[ann0see]

please give me a hint regarding the language tag

The process in the documentation is a bit complicated (don't want to bother you with that).

We agreed that at least every pull request should have a language tag (not sure about issues though) to keep organised. We suffered a lot of discussion being spread across multiple issues, PRs, etc. which makes it difficult to follow up.

Since this issue is in the documentation repository, I thought everything related to documentation changes would be discussed here, while everything related to the software on corrados' repository.

I added "Manual" to the title since (at least what I understand, you would like quite some changes in this file).

I must admit that the CONTRIBUTING file is not yet up to date and I acted too fast.

[drummer1154]

Just to emphasize: For me this Issue is only a starting point for discussion. Most likely it can be closed later if we have created additional Issues in their correct repository which cover my findings.

@ann0see OK, I see - I am not at all familiar with the github process, sorry. I was not aware in which repository my new Issue was created and I just followed the advice given in #778 (comment).

@dakhubgit Please try to evaluate whether your proposal at https://github.com/jamulussoftware/jamuluswebsite/issues/174#issuecomment-744040593 is really less impacting the current implementation than mine. Edit: From my PoV there is no need for a fader for oneself while not connected - the input level meter is enough to check for eventual overload. Also from a user perspective I believe that if you display a fader while not connected, you will get more or less the same picture in the GUI as when connected while your client is the only user. I believe this could be confusing.

[gilgongo]

OK. So you want:

1. Some diagrams. Which you will make. I have no objection to that, just as long as we don't present them to normal users as being something they need to examine before using the software :-)

2. Some changes to the UI. I'm a bit unclear on these though as there seem to be a variety of suggestions. Probably best to raise them as separate tickets on the software repo issues list.

3. Some changes to the function - perhaps such that a user can play about on their own without joining a server? Again, best raised as separate tickets.

Lastly, and forgive me if I'm wrong, but you seem to imply there is some crisis of understanding between the maintainers of the software and documentation and the users of Jamulus. I would be the first to admit that improvements could be made for ease of use in both areas, which I suspect are not optimal. These things never are.

However, I would also point out that of 229 responses to our usage survey so far, answers to the question "How easy was it to understand how Jamulus worked?" are currently:

58% for "Easy - installed and got if going almost immediately"
32% for "A little tricky - needed a bit of work"
6% for "Hard - had to read the docs/forums"

This is not to imply complacency - and only 2.6% identify themselves as having "Basic" computer skills - and things could of course be improved.

[ann0see]

OK, I see - I am not at all familiar with the github process, sorry.

No problem. This is an issue we should talk about in another GitHub issue. gilgongo already said that the information/development structure is a bit unclear.

[drummer1154]

Since this issue is in the documentation repository,

OK, I see, I was not aware of the nature of the repository, I just blindly followed the advice I received in #778 (comment):

Please open a ticket to address this in the documentation

After I started editing (and testing), I decided to document all weaknesses I stumbled upon, and my proposal is to discuss the single points here and open new Issues (of course each then in its correct repository :-) once the solution is clear. I am going to copy dakhubgit's proposal. [Done and commented.]

[ann0see]

Still a bit confused about this issue, but as far as I understand, there are multiple smaller and bigger issues (also in the documentation).

Provide two up-to-date diagrams to be taken up into the Manual (maybe into a new "Technical Reference" section): 1) data flow and 2) audio signal flow. Rationale: "A picture paints a thousand words" - Jamulus is too complex to be quickly understood by simply reading the Manual, and the currently required "trial and error" method is a waste of time.

I also think that a a more technical explanation of Jamulus which is not part of Volker's paper would be good. We should maybe include this in the knowledge base (not yet online) since not every user would like to have to read this.

This page would describe how Jamulus works (with technical details including the fact that the Server has multiple mixes, etc.) and be linked from the getting started page.

The diagram on the getting started page: https://jamulus.io/wiki/Getting-Started#how-jamulus-works-in-general could stay (almost) the same but with another explanation

Jamulus works on the client server principle. Everybody’s audio is sent to a server, mixed and processed there. Afterwards the audio is sent to every client. If a server is made public and registered on a central server, its information will be broadcasted to all clients.

could be changed to

Jamulus works on the client server principle. Everybody’s audio is sent to a server, mixed and processed there for every client. Afterwards, the server sends every client their own mix. If a server is made public and registered on a central server, its information will be broadcasted to all clients.

(not yet sure about the exact wording)

This would go with @gilgongo s answer:

Some diagrams. Which you will make. I have no objection to that, just as long as we don't present them to normal users as being something they need to examine before using the software :-)

Input level, Mute myself etc. are still to be discussed.

[drummer1154]

The diagram on the getting started page ...

is of course well suited for a "normal" user, thanks @gilgongo the sources are available so if it needs tweaking just go on.

As already said earlier I am really willing to create more technical drawings explaining the "inner" functioning so as to enable troubleshooting, but not required to be studied in the "normal case", when everything works well right from the beginning. I have an older Visio version which I will use. But I am sure that I will need help, and I also need to say that I do not have much time the next days. So most likely I will be starting over next weekend.

Input level, Mute myself etc. are still to be discussed.

Correct, and I hope the Jamulus architects are really willing to take these proposals into account without throwing them into the bin immediately, I am not the only one who says it is worth to improve the current methods.

[ann0see]

@drummer1154 I know you worked on some proposed diagrams etc. Can you open a PR with them if they are finished?

[gene96817]

Very confusing. This thread was opened Dec 13, 2020 and it was mailed to me April 12, 2021 0700 UTC.

[ann0see]

Just came across this issue again. After being part of the community, I think I now understand some of your questions. Especially this one:

Why is the rotary pan not available if "Settings/Misc/Audio Channels" is set to Mono?

That’s a technical issue. If Jamulus is set to mono only one channel is sent to the server. Same goes for the receiving side. Mono in/Stereo out sends two similar channels to the server and is basically a hack (to remain backwards compatible ??) to get a stereo signal back from the server.

Stereo sends two independent channels to the server.

[dakhubgit]

ann0see ***@***.***> writes:

Just came across this issue again. After being part of the community, I think I now understand some of your questions. Especially this one: > Why is the rotary pan not available if "Settings/Misc/Audio Channels" is set to Mono? That’s a technical issue. If Jamulus is set to mono only one channel is sent to the server. Same goes for the receiving side. Mono in/Stereo out sends two similar channels to the server and is basically a hack (to remain backwards compatible ??) to get a stereo signal back from the server.

The problem is that it's a stupid hack: the Opus decoder does not even care if the sent signal is stereo or mono: a mono decoder will happily create a mono signal from a stereo encoded one and vice versa. And even the "custom" Opus decoder will not care about mono or stereo or the bitrate: give it the actual packet size, and it will decode whatever it is given. Unfortunately, the Jamulus server itself refuses to decode a packet with a size it does not like and does not even pass on the packet size to the Opus decoder. If the server (and likely also the clients) just passed the size of every received packet to the decoder, this would be a lot more flexible.

Stereo sends two independent channels to the server.

Nope, to the encoder. The server pretty much always receives a mono and a side channel. If VBR were active, Mono-In/Stereo-Out would dial down the bandrate significantly because of the trivial side channel.

…

-- David Kastrup

[dakhubgit]

Sending control data via TCP would have the advantage of guaranteed and in-order arrival. It would make it more important that data packets sent via UDP can be interpreted standalone, without depending on the delivery of control data. Using their packet size will facilitate that, but when not the stereosity or the quality but the buffer size that a packet corresponds to is changed, this requires switching to a different decoder when using custom encodings. Making the UDP data packets self-descriptive would be warranted when switching to a TCP control protocol, but doing so could well be done independently ahead of the larger changes that a TCP control protocol (that would also require firewall changes for server administrators) would require.

[ghost]

I think that the design idea is that Jamulus does not even distinguish data and control packets, decides it is a data packet if the size

The beauty in this design is that everything is serialized before transmission and is deserialized later. There is a huge advantage in sending a serialized stream vs multiple parallel streams, as synchronization is maintained. Jamulus is somehow robust against packet loss also.

Almost 15 years of experience and debugging are nothing to be sneezed at.

Jamulus is obviously Intelligent design, not Evolutionary design.

[dakhubgit]

The beauty in this design is that everything is serialized before transmission and is deserialized later.
Which is sort of what you do not want if one stream is realtime-critical, and the other is delivery-critical. Synchronisation is not maintained since UDP is a protocol admitting packet loss. For the control stream, Jamulus maintains a complex resend-and-acknowledge stream that is basically reimplementing TCP, only worse so.

I don't actually see the point in the "everything must stay the same" kind of discussions since old versions of course remain available. At the same point of time, there is not much of a point in agreeing on what should be done when there is nobody around that is going to do it. When some concrete proposal gets implemented for concrete reasons, there will be ample time to test it out, including checking for backwards compatibility with both server and clients. And if it is done for concrete tangible reasons, those can be verified as well.

There is more of a judgment call for functionally identical changes intended to lead to more flexibility and better extensibility and maintainability. The readily apparent observation in these discussions that very few people actually have a grasp of what Jamulus does internally seems like a good enough reason to try letting its code reflect its design and workings more transparently.

C++ is a plague boil of a language for essentially one reason: enable modularisation and generalisation without an arguable loss of performance. Would be a pity to take its disadvantages while leaving the advantages alone.

[gene96817]

there is not much of a point in agreeing on what should be done when there is nobody around that is going to do it.

There are places in Jamulus that can be improved. Some functions are better served by TCP. Since Jamulus works fairly well, we could take the attitude that it is good enough. It is all academic if the developers are interested in improving other aspects of Jamulus.

[dcorson-ticino-com]

there is not much of a point in agreeing on what should be done when there is nobody around that is going to do it.

Then I would suggest, volunteer to do it.
You seem to know C++ very well from your comments. Go to it!
This is a FOSS project, we all do what we can and what we find is important.
Please join in.