Support for TP-Link VIGI Two-Way Audio

[fllash333]

Hey there,

I have a few TP-Link VIGI cameras. Unfortunately all the VIGI cameras only support ONVIF Profile S, so no back-channel audio.
However the cameras themselves do support it via the VIGI app and also via an "Open API".

In the API docs (attached), there is the talk endpoint for sending audio (with return audio sent via the preview stream).

If support is possible that would be excellent!

I'm happy to help in any way I can.
VIGI-API.docx

[AlexxIT]

Very rare cameras. Not worth wasting time on homemade protocols. I'll leave this issue in case someone wants to implement it.

[fllash333]

Understandable. I'll do some research and see about working on this myself.

[redelacruz]

I want to get in on this party, as I already have 2 Vigi cameras (really good cameras, BTW, despite the non-reference ONVIF implementation and custom protocols) and I plan to add more. The only thing is I haven't ever used Go yet, but I might have some spare time over the holidays to learn a new language. Will look into it.

@fllash333 API doc helps a ton, thanks for putting it out there! I didn't know they released something like it.

[fllash333]

I only found out they had an API at all in the product guide which had a single dot point on one of the pages. I don’t think they want to be on the hook for supporting unofficial integrations so it’s not easily accessible.

I have maybe 6 or 7 cameras, four different models.
I’m not the most experienced coder so if it’s something you think you can do I’m happy to take a back seat and help where I can.

[n1mda]

@redelacruz @fllash333 Im happy to donate if you add support for 2-way communication with VIGI cameras

[svodbno]

Hi guys. Any of you succesfully connected with vigi camera according VIGI API document? I'm not succesful by using methods listed in API doc. Even doAuth does not work. I found https://github.com/yetanothercarbot/vigi_camera_lighting where method 'do' is used for auth which worked for me. Maybe @yetanothercarbot can help to find actualised API doc?

[yetanothercarbot]

Hey there, thanks for pinging me.

I wasn't aware of this API doc. Those scripts were developed by reverse engineering the obfuscated JS served by the cameras.

I'm happy to have a poke to see if I can get audio documented. Unfortunately that's only available on the app, not on the web interface.

---

Edit: I've had a gander at the document in the OP. It's most likely just old. On a quick skim, most of the settings endpoints are different from what is specified.

E.g. the video capability request which returns resolutions - in the document, it uses {"method": "getVideoCapability"}.

The current interface seems to use:

{
    "video": {
        "name": [
            "main",
            "minor"
        ]
    },
    "method": "get"
}

As far as I can tell, the method field will always be one of "get", "set" or "do", with the other fields controlling the behaviour further.

---

@fllash333 would you have a link to where you found this apidoc by any chance? I wonder if there's a newer version available.

I've had a poke at the protocol with mitmproxy but the camera feed seems to break when accessed through the proxy (regardless of web version, desktop app or mobile app).

[fllash333]

Hey @yetanothercarbot, the doc came directly from TP so I'd assume it's the newest available. Bit of a bummer if it's not even current. Can you point me in the right direction - how were you authenticating against the API? The doc says port 20443 but that's closed.

[yetanothercarbot]

Even with some searching I wasn't able to locate the source of doc - is it possibly only relevant to older VIGI cameras?

---

The API takes the form of HTTPS POST requests on port 443. I included the Accept: application/json and Content-Type: application/json; charset=UTF-8 headers. For authentication, the requests just go to /. For other functions, they seem to go to /ds.

The web server uses the AES256-GCM-SHA384 cipher for HTTPS (which is noteworthy because python's urllib3 doesn't include the cipher by default, perhaps other libraries don't either) and uses an expired self-signed certificate. At least the certificate is for the correct domain!

Authentication is actually pretty simple. Clients are authenticated by holding a stok token, which is valid for ~30 minutes.

A smaller example of how to retrieve the stok value is on this older gist.

Essentially:

1. The user password is modified by:
   1. prefixing with a fixed string ("TPCQ75NF2Y:")
   2. MD5 Hash
   3. Hexdigestion (convert to a string only containing hex digits - py docs)
   4. Converting to uppercase
   5. I'll call this pw_hash (referenced in step 3)
2. The script retrieves a nonce (random one-time-use value) and encryption key from the camera, using get_encrypt_info ({"user_management": {"get_encrypt_info": null}, "method": "do"})
3. The string from step 1 is used - the string {pw_hash}:{nonce} is encrypted using the key received from the camera, and encoded in base64. I'll call this step3 (referenced in step 4). The purpose of this is to presumably prevent replay attacks.
4. A second API request is made with the payload {"method":"do", "login": {"username": username, "password": step3, "passwdType":"md5","encrypt_type":"2"}} (where username is the username and step3 is the output of string in step 3).
5. Assuming that the credentials are valid, the response JSON will include a stok field which can be used to authenticate future requests. Keep in mind it expires every 30 minutes or so, and will need to be re-requested.

This was reverse-engineered from using the network tab in the F12 debugger and picking apart the absolute mess of JavaScript code in the camera's web interface.

The mobile app might use a different method - it specifies encrypt method 1 where my script (and the web UI) use encrypt method 2. Superficially it seems similar however.

---

However

I don't think this particular API needs to be explored much further. As far as I can tell, the following ports are open:

• 80: HTTP, just redirects to HTTPS
• 443: HTTPS, exposes uhttpd which serves the web UI
• 554: RTSP
• 2020: ONVIF
• 8800: HTTP, exposes vhttpd
• 8443: HTTPS, exposes vhttpsd

The vhttpd and vhttpsd daemons seem interesting. This is what is doing the actual streaming of the camera - the web UI seems to reach out to the encrypted 8443 version and the mobile app tries to access the unencrypted 8800 version.

I don't think using these two ports to stream video is that interesting - RTSP is available and works fine for main and sub streams. However, it seems likely that this also where audio is streamed to the camera.

This seems to be a type of HTTP API it's using, but I can't tell much more unfortunately. I've tried to intercept the API calls using mitmproxy, but was not successful sadly. Connecting without mitmproxy works, but connecting with mitmproxy results in the camera returning a 401 request even in transparent mode.

It seems that vhttpsd returns some data that could be used for authentication in the headers of the 401 response, but the lack of a follow-up request makes me doubt that it is intended as such.

One of the requests had this body - unfortunately I don't remember if this is from one of the (multiple!) desktop apps or the web version.

{"type":"request","seq":1,"params":{"method":"get","preview":{"channels":[0],"resolutions":["vga"]}}}

The OPTIONS request is similar to what RTSP does, but this has the HTTP/1.1 tag at the end...it's certainly a bit bizarre.

Note that if you want to try with mitmproxy yourself, you'll need to set the ssl_insecure flag to true, since the certs are self-signed and expired. I think the main goal is to see if it's possible to get the camera feed working while capturing traffic. I was using mitmproxy because it decodes HTTPS requests. Wireshark would work too if you can capture the tls sessions - easy enough on desktop, but seems to be difficult to impossible on Android.

I was surprised at the lack of information on these cameras tbh, they are fairly common and I would have assumed there was more info already floating around.

[AlexxIT]

It's very similar to Tapo protocol:
https://github.com/AlexxIT/go2rtc/blob/master/pkg/tapo/

[yetanothercarbot]

Ah, so it is. I probably shouldn't be surprised.

I hadn't looked at that since trying to see if the authentication on that was at all relevant to the VIGI cameras.

[yetanothercarbot]

OK, I seem to have gotten the gist of the authentication steps for port 8800/8443. Haven't tested it with a separate script but I believe this should be accurate.

(Apologies for formatting. Just figured it out and wanted to share lol)

Authentication Steps

Retrieve nonce, opaque, qop from camera (HTTP header)
Let cnonce be random 16 character string from "0123456789abcdefghijklmnopqrstuvwxyz"
Let nc be "00000001"
Let password hash be same as from UI: hashlib.md5(f"TPCQ75NF2Y:{password}".encode()).hexdigest().upper() (python snippet)
Let f be calculated using security encode function (below)

Let a be MD5 hash of "{username}:{realm}:{f}"
Let b be MD5 hash of "POST:/stream"
Let response be equal to md5 hash of "{a}:{nonce}:{nc}:{cnonce}:auth:{b}"

The new digest header then becomes:
Digest username="{userName}", realm="{realm}", nonce="{nonce}", uri="/stream", response="{response}", opaque="{opaque}", qop="{qop}", nc="{nc}", cnonce="{cnonce}"

It's really just a modified http digest function at the end of the day. The security encode function was the annoying part to dig out.

Security Encode function:

securityEncode = function (c) {
    let b = "RDpbLfCPsJZ7fiv";
    let c = "yLwVl0zKqws7LgKPRQ84Mdt708T1qQ3Ha7xv3H7NyU84p21BriUWBU43odz3iP4rBL3cD02KZciXTysVXiV8ngg6vL48rPJyAUw0HurW20xqxv9aYb4M9wK1Ae0wlro510qXeU07kV57fQMc8L6aLgMLwygtc0F10a0Dg70TOoouyFhdysuRMO51yY5ZlOZZLEal1h0t9YQW0Ko7oBwmCAHoic4HYbUyVeU3sfQ1xtXcPcf1aT303wAQhv66qzW";
    var d = ''
    var g, e, f, h
    var k = 187
    var l = 187
    e = c.length;
    f = b.length;
    h = a.length;
    g = e > f ? e : f; // Set g to larger of e or f
    // for (var m = 0; m < g; m++) {l = k = 187, m >= e ? l = b.charCodeAt(m) : m >= f ? k = c.charCodeAt(m) : (k = c.charCodeAt(m), l = b.charCodeAt(m)), d += a.charAt((k ^ l) % h);

    for (var m = 0; m < g; m++) {
        let charB, charC;
        
        // Initialize l and k to 187
        let l = 187;
        let k = 187;
        
        // Determine the values of l and k based on the value of m
        if (m >= e) {
          charB = b.charCodeAt(m);
          l = charB;
        } else if (m >= f) {
          charC = c.charCodeAt(m);
          k = charC;
        } else {
          charB = b.charCodeAt(m);
          charC = c.charCodeAt(m);
          k = charC;
          l = charB;
        }
        
        // Perform the XOR operation and append the result to d
        let xorResult = charB ^ charC;
        let charA = a.charAt(xorResult % h);
        d += charA;
      }

    return d
  };

[MiAutomations]

It's very similar to Tapo protocol: https://github.com/AlexxIT/go2rtc/blob/master/pkg/tapo/

Yes probably, but once is little different, the "Tapo" team doesn't support this new TP-Link Brand :( VIGI

JurajNyiri/HomeAssistant-Tapo-Control#743

[MiAutomations]

Very rare cameras. Not worth wasting time on homemade protocols. I'll leave this issue in case someone wants to implement it.

The VIGI is getting more and more camera models in this last days ( Current Year) and I thing is a question of time until this Sub TP-LINK Brand start to be more and more used. The cameras are very good and have a several models Types available right now

[fllash333]

Just as an update, I heard from the support team that not all Vigi cameras support the API yet.
C540: Update due in two weeks
C540s/C240: Update due next April
Probably others but I only asked about the models I have.
That's a little bit sneaky as it's listed on the product marketing material as a feature.

[svodbno]

@fllash333 dont you know if there is some beta to test with?

[fllash333]

Not that I know of unfortunately, I'm just passing on what support told me.
I did ask for a list of all Vigi cameras and when API support is expected.
And just for good measure I asked what the chances are of them adding support for ONVIF Profile T since the cameras clearly have capable hardware. Ahh, we can dream.

[yetanothercarbot]

It's very similar to Tapo protocol: https://github.com/AlexxIT/go2rtc/blob/master/pkg/tapo/

Yes probably, but once is little different, the "Tapo" team doesn't support this new TP-Link Brand :( VIGI

It's probably worth a try whether the Tapo protocol works when modified with the Vigi's authentication method (which is documented above). At a skim of the Tapo code linked, it seems both series are using the same endpoints with similar parameters.

I must admit that I'm not particularly familiar with Go, and have no need to for 2way audio (only one camera is installed in a location where I'd consider audio useful, and that one has no speaker!).

Anyway, the best option here for everyone is if TP-Link implements ONVIF backchannel. That would greatly broaden support and not require any of this messing about. As an aside, does anyone know if PTZ control (on the cameras that support it) is exposed through ONVIF?

Unfortunately with how slow TP-Link had been on getting the local control toggle rolled out on the Tapo cameras, any manufacturer solution will take a while to arrive.

[fllash333]

All the VIGI cameras that I’ve tested (that have the ability to move!) seem to support ONVIF profile S and allow PRZ via ONVIF.

[AlexxIT]

If someone gives me remote access to their camera - I can try to screw the new authorization to the tapo protocol. My contacts are in my profile.

[yetanothercarbot]

That's an incredibly generous offer AlexxIT! Hopefully it would be that straightforward, should someone take you up on your offer.

[fllash333]

I'll organise access to one of my cameras. Will reach out directly.

[AlexxIT]

Well. I have managed to login to port 8800. But I can't decode video data. The algorithm seems to be different from tapo.

[AlexxIT]

Thanks to @n1mda and @fllash333 - VIGI cameras now supported. Including two way audio. In master (dev) version. ONLY admin account can be used! Auth same as for RTSP.

Tested with:

• C240I - only video H264
• C230I Mini - video H264 with two way audio

streams:
  camera1: vigi://admin:[email protected]

There may be problems with H265. Still needs some improvements.

[AlexxIT]

And of course thanks @yetanothercarbot for secure function. There are some inaccuracies in the description, but I found the correct way to handle it.

[PetePeter]

I have Vigi C350 (or maybe it was C340) 2.8mm i can test once go2rtc gets a new release

[yetanothercarbot]

Pulled in the latest, can confirm it works with my cameras! Thank you AlexxIT :)

Purely to satiate my curiosity, would you mind expanding on what the inaccuracies were?

[MiAutomations]

By the way it will be great if the Vigi cameras (TP-link sub brand) It will supported in Home Assistant as a integration in a future

https://community.home-assistant.io/t/tp-link-vigi-smart-ip-cameras/780540

[AlexxIT]

@yetanothercarbot

• password hash (with TPCQ75NF2Y) is unnecessary step
• let c should be let a

[yetanothercarbot]

Ah, thank you very much! I'm glad you got it working.

[MiAutomations]

vigi://admin:[email protected]

Hello

I configured the stream from my VIGI camera using the addon go2rtc master (dev) version and the admin account for the camera , and I'm using also the webrtc card configured like this:

type: custom:webrtc-camera
url: TP-Link VIGI C345_Sub
mode: webrtc,webrtc/tcp,mse,hls,mjpeg
media: video,audio
ui: true

But I still not able to use the 2way audio ... Can you please confirm that I also need to access over the HTTPS?

Thank you

[redelacruz]

Can you please confirm that I also need to access over the HTTPS?

At least in modern browsers, HTTPS (or a directive to your browser to treat an otherwise unsecure site as secure, e.g., the #unsafely-treat-insecure-origin-as-secure flag in Chrome) is always required to allow use of your microphone.

[MiAutomations]

Can you please confirm that I also need to access over the HTTPS?

At least in modern browsers, HTTPS (or a directive to your browser to treat an otherwise unsecure site as secure, e.g., the #unsafely-treat-insecure-origin-as-secure flag in Chrome) is always required to allow use of your microphone.

Ok, is there any way to use the HTTPS to test?

Or is there any way to use this over the frigate once in the machine that I'm using HTTPS access is installed the Frigate and I can't use at the same time the addon go2rtc ?

Thank you

[redelacruz]

@AlexxIT Great work on the update! I can confirm that it works.

However, I encountered an issue while trying it out. With the VIGI source enabled, one camera's set of streams (not the one using the VIGI module) starts to develop weird issues. Here's my config file for context:

api:
  origin: '*'
hass:
  config: /config
log:
  format: text
  rtsp: debug
  streams: debug
  webrtc: debug
ngrok:
  command: ngrok tcp 8555 --authtoken [redacted]
rtsp:
  default_query: mp4
streams:
  Outdoor_Front:
  - ffmpeg:outdoor_front_sd
  Outdoor_Pedestrian_Gate:
  - ffmpeg:outdoor_pedestrian_gate_sd
  Outdoor_Side_Passageway: # <<<----- Affected by issues caused by adding VIGI source
  - ffmpeg:outdoor_side_passageway_sd
  birdseye: exec:ffmpeg -hide_banner -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device
    /dev/dri/renderD128 -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -r 10 -i
    /tmp/cache/birdseye -c:v h264_vaapi -g 50 -bf 0 -profile:v high -level:v 4.1 -sei:v
    0 -an -vf format=vaapi|nv12,hwupload -rtsp_transport tcp -f rtsp {output}
  indoor_homeserver_webcam: ffmpeg:device?video=0&video_size=1280x720#video=h264#raw=-pix_fmt
    yuv420p#hardware
  outdoor_front_hd:
  - rtsp://remote_user:[redacted]@192.168.2.11:554/user=admin_password=_channel=0_stream=0&protocol=unicast&onvif=ture.sdp?real_stream
  - ffmpeg:outdoor_front_hd#audio=aac#video=copy
  outdoor_front_sd:
  - rtsp://remote_user:[redacted]@192.168.2.11:554/user=admin_password=_channel=0_stream=1&protocol=unicast&onvif=ture.sdp?real_stream
  - dvrip://remote_user:[redacted]@192.168.2.11:34567?backchannel=1
  - ffmpeg:outdoor_front_sd#audio=aac#video=copy
  outdoor_pedestrian_gate_hd:
  - rtsp://admin:[redacted]@192.168.2.13:554/stream1
  - ffmpeg:outdoor_pedestrian_gate_hd#audio=aac#video=copy

# ---- VIGI Source
  outdoor_pedestrian_gate_sd:
  - rtsp://admin:[redacted]@192.168.2.13:554/stream2#backchannel=0
  - vigi://admin:[redacted]@192.168.2.13/stream2
  - ffmpeg:outdoor_pedestrian_gate_sd#audio=aac#video=copy
# ---- 

  outdoor_side_passageway_hd: # <<<----- Affected by issues caused by adding VIGI source
  - rtsp://admin:[redacted]@192.168.2.12:554/stream1
  - ffmpeg:outdoor_side_passageway_hd#video=copy
  outdoor_side_passageway_sd: # <<<----- Affected by issues caused by adding VIGI source
  - rtsp://admin:[redacted]@192.168.2.12:554/stream2
  - ffmpeg:outdoor_side_passageway_sd#video=copy
webrtc:
  candidates:
  - 192.168.2.28:8555
  listen: :8555

In the above context, Outdoor_Side_Passageway, outdoor_side_passageway_sd and/or outdoor_side_passageway_sd streams begin to fail randomly after the VIGI (vigi://...) stream is started by go2rtc.

Here's a pastebin of the logs when issues happen. I'll continue to update it as the issues occur.
https://pastebin.com/gMqvwEDC

Edited to add: If you want me to open a new issue for this for tracking purposes, let me know.

[redelacruz]

Ok, is there any way to use the HTTPS to test?

As I mentioned above, in Chrome, enable the #unsafely-treat-insecure-origin-as-secure flag and add the URL to your go2rtc and/or your Frigate instance in the whitelist. In Firefox, you can access site permissions from the padlock.

Or is there any way to use this over the frigate once in the machine that I'm using HTTPS access is installed the Frigate and I can't use at the same time the addon go2rtc ?

I really can't understand what you're trying to ask here.

[AlexxIT]

@redelacruz maybe your camera can't handle so many simultaneous connections.

[redelacruz]

@redelacruz maybe your camera can't handle so many simultaneous connections.

I may be wrong, but if that's the case, shouldn't adding the VIGI source affect the camera that source is connected to, not a different camera altogether?

Edited to add: I edited the config file I provided above to make it more illustrative of the issue I'm facing.

Edited again to add: I've also ruled out network issues by adding one of my spare cameras into the mix with the same general configuration as my other cameras above (one HD and one SD stream with ffmpeg:{stream_name}#audio=aac#video=copy for each stream).

[MiAutomations]

Ok, is there any way to use the HTTPS to test?

As I mentioned above, in Chrome, enable the #unsafely-treat-insecure-origin-as-secure flag and add the URL to your go2rtc and/or your Frigate instance in the whitelist. In Firefox, you can access site permissions from the padlock.

Or is there any way to use this over the frigate once in the machine that I'm using HTTPS access is installed the Frigate and I can't use at the same time the addon go2rtc ?

I really can't understand what you're trying to ask here.

Just to update, I have successfully #unsafely-treat-insecure-origin-as-secure

My second question is related to the Frigate "over go2RTC" but this one is not yet in the same version, and if is possible to use on the frigate?

[redelacruz]

My second question is related to the Frigate "over go2RTC" but this one is not yet in the same version, and if is possible to use on the frigate?

Yes. You have to either build the go2rtc binary yourself by cloning the repository and building it or download a suitable nightly. Then, follow the instructions in the Frigate documentation to use it.

[AlexxIT]

https://github.com/AlexxIT/go2rtc/releases/tag/v1.9.8

[yetanothercarbot]

Should this be added to the readme as a valid source?

[AlexxIT]

You right. Readme update in plans.

[yetanothercarbot]

Awesome, good to know. Thank you!