ISAPI audio input not working to Hikvision DS-2CD2386G2-ISU/SL

[f1d094]

Although I am able to send audio to the camera via curl, audio via the ISAPI connection via go2rtc fails to send. Modifying client.go to call '/ISAPI/System/TwoWayAudio/channels/1/close' before '/ISAPI/System/TwoWayAudio/channels/1/open' allows the connection to actually open...but it isn't reliable.

curl --digest "http://$USER:[email protected]/ISAPI/System/TwoWayAudio/channels"

Output:

<?xml version="1.0" encoding="UTF-8"?>
<TwoWayAudioChannelList version="2.0" xmlns="http://www.hikvision.com/ver20/XMLSchema">
<TwoWayAudioChannel version="2.0" xmlns="http://www.hikvision.com/ver20/XMLSchema">
<id>1</id>
<enabled>false</enabled>
<audioCompressionType>G.711ulaw</audioCompressionType>
<speakerVolume>100</speakerVolume>
<noisereduce>true</noisereduce>
<audioInputType>MicIn</audioInputType>
</TwoWayAudioChannel>
</TwoWayAudioChannelList>

curl --digest -X PUT "http://$USER:[email protected]/ISAPI/System/TwoWayAudio/channels/1/close"

Output:

<?xml version="1.0" encoding="UTF-8"?><ResponseStatus version="2.0" xmlns="http://www.hikvision.com/ver20/XMLSchema">
<requestURL></requestURL>
<statusCode>1</statusCode>
<statusString>OK</statusString>
<subStatusCode>ok</subStatusCode>
</ResponseStatus>

curl --digest -X PUT "http://$USER:[email protected]/ISAPI/System/TwoWayAudio/channels/1/open"

Output:

<?xml version="1.0" encoding="UTF-8"?>
<TwoWayAudioSession version="2.0" xmlns="http://www.hikvision.com/ver20/XMLSchema">
<sessionId>2135689571</sessionId>
</TwoWayAudioSession>

Finally, sending audio file which works:

curl --digest -X PUT -H "Content-Length: 0" -H "Content-Type: application/octet-stream" -d @duck.ulaw "http://$USER:[email protected]/ISAPI/System/TwoWayAudio/channels/1/audioData"

Review of .pcaps revealed constant authorization errors. After a lot of poking, I found that if I did:

curl --digest -X PUT "http://$USER:[email protected]/ISAPI/System/TwoWayAudio/channels/1/close"

Before opening the MicOn view then it would work. This is the card config:

type: custom:webrtc-camera
ui: true
streams:
  - url: Front_Door-Live
    mode: webrtc
    media: video,audio
    muted: true
    name: Mic Off
    style: '.pictureinpicture {display: none} .screenshot {display: none}'
  - url: Front_Door-Intercom
    mode: webrtc
    muted: false
    media: video,audio,microphone
    name: Mic On
    style: '.pictureinpicture {display: none} .screenshot {display: none}'

I tried to modify /pkg/isapi/client.go to do a 'close' before 'open' and that kindof works but isn't reliable, go2rtc sometimes not returning with a stream when switching the from microphone off/on.

[Cold-Lemonade]

@f1d094 I have an Anpviz camera with two-way audio that I think uses ISAPI. But I am having trouble getting any access to it via the command line using curl. Any suggestions? I can connect to the camera using the go2rtc instance in Frigate, but not the go2rtc add-on in Home Assistant.

[f1d094]

@Cold-Lemonade: I don't know anything about Anpviz cameras. The ISAPI I am referring to is unique to Hikvision. Unless the Anpviz cameras are hikvision clones and use the same API I would guess it is entirely different. I cobbled together those commands from the Hikvision ISAPI reference, which I will note has several typos. For example, Using GET in one example where it should be PUT and entirely failing to state that they require digest authentication.

But if you can't get it to work on the command line using curl, then it is certain to not work via HASS.

Frigate has no microphone support.

[f1d094]

@Cold-Lemonade: I just realized this is under my original issue. I created my own workaround and put in a PR for the fix PR#967. I think it might be done better but it works. I am now using my own version of go2rtc 1.8.5 and I can chat with people at the front door via HASS using the Frigate Card. Check the frigate docs on how to use an external go2rtc. It is pretty straightforward.

[f1d094]

Additional Note: Using the WebRTC card trick of unloading and reloading when you turn the microphone on and off was too much for it to handle and everything winds up in an unusable state. I think the Frigate Card keeps the channel open and just mutes/enables the microphone client side...maybe not, but it is stable in any case. The WebRTC card was highly problematic.

[pergolafabio]

Hey @f1d094 , i have same issue on my intercom
When i check the channels command, when not using the card, i get this

<?xml version="1.0" encoding="UTF-8"?>
<TwoWayAudioChannelList version="2.0" xmlns="http://www.isapi.org/ver20/XMLSchem                                                                                                                                                             a">
<TwoWayAudioChannel version="2.0" xmlns="http://www.isapi.org/ver20/XMLSchema">
<id>1</id>
<enabled>false</enabled>
<audioCompressionType>G.711ulaw</audioCompressionType>
</TwoWayAudioChannel>
</TwoWayAudioChannelList>

When i start two way audio with the card:

<?xml version="1.0" encoding="UTF-8"?>
<TwoWayAudioChannelList version="2.0" xmlns="http://www.isapi.org/ver20/XMLSchema">
<TwoWayAudioChannel version="2.0" xmlns="http://www.isapi.org/ver20/XMLSchema">
<id>1</id>
<enabled>true</enabled>
<audioCompressionType>G.711ulaw</audioCompressionType>
</TwoWayAudioChannel>
</TwoWayAudioChannelList>

If i then press the mic button again, to stop two way audio AND i send this command afterwards:

curl -i --digest -u admin:XXX-X PUT  http://192.168.0.70/ISAPI/System/TwoWayAudio/channels/1/close

<?xml version="1.0" encoding="UTF-8"?>
<ResponseStatus version="1.0" xmlns="http://www.std-cgi.com/ver10/XMLSchema">
<requestURL></requestURL>
<statusCode>1</statusCode>
<statusString>OK</statusString>
<subStatusCode>ok</subStatusCode>
</ResponseStatus>

I still see the enabled = true when i do the curl to see the channels, is that normal?

curl -i --digest -u admin:XXX http://192.168.0.70/ISAPI/System/TwoWayAudio/channels


<?xml version="1.0" encoding="UTF-8"?>
<TwoWayAudioChannelList version="2.0" xmlns="http://www.isapi.org/ver20/XMLSchema">
<TwoWayAudioChannel version="2.0" xmlns="http://www.isapi.org/ver20/XMLSchema">
<id>1</id>
<enabled>true</enabled>
<audioCompressionType>G.711ulaw</audioCompressionType>
</TwoWayAudioChannel>
</TwoWayAudioChannelList>

It seems it goes back to false, when i browse away from the lovelace card, maybe its because the RTSP stream is still active

[f1d094]

Hi @pergolafabio. It is my understanding that the channel will stay open, expecting data, until it either receives an explicit call to close or it receives audio data and the audio stream terminates. Either of these then closing the channel, which should result in <enabled>false</enabled> being returned.

Just testing this now I see that if I explicitly close the homeassistant app after enabling the mic (iPhone/iPad apps) the channel appears to stay open until I reopen homeassistant or visit via my browser on PC...

[pergolafabio]

Gonna go a real test tomorrow when I'm home, gonna start two way audio with frigate, the stop the audio using the mic button on frigate card, then send the curl to close the channel...
Let's hope if I can start a two way audio again, without reopening/ refreshing HA!!

[pergolafabio]

It's strange that I notice the issue only with frigate card... With the webrtc card I can start/stop two way audio without an issue

[f1d094]

I'm afraid I had already abandoned the WebRTC card in favor of the Frigate Lovelace Card well-before I had any idea of any of the underlying problems...largely because of cosmetic reasons on how I wanted to integrate into my setup. The Frigate Card has more flexibility where I needed it. I do however remember that as long as I was super slow-motion about my changing state of the microphone that the WebRTC card worked well. I suspect that is because it fully closes the audio channel each time you turn off the mic, which is an alternate trigger to cause the ISAPI interface to close.

I no longer have a WebRTC card setup to test with however, I'll be curious to see what you find.

[pergolafabio]

Will test tomorrow!! :-)
I have a real Hikvision intercom, not just a camera, so on a real incoming call, I first need to fake an answer command to make the Hikvision device stop ringing (bringing back the interxom to idle) then I start the two way audio... Frigate is perfect for that , since you can customize the buttons with services

[f1d094]

@AlexxIT - FYI...I've been using my fork for a while now and it seems solid, except as noted here: #967 (comment)

I think there is an issue with the way the Frigate Lovelace Card enables the mic but have not found the bandwidth to do any testing. YMMV.

[f1d094]

I think there is an issue with the way the Frigate Lovelace Card enables the mic

@AlexxIT - Following up - @dermotduffy clued me in on setting the mic to always-enabled which has resolved my remaining issue. Using my patched go2rtc the microphone consistently and smoothly loads and works as expected

[AlexxIT]

https://github.com/AlexxIT/go2rtc/releases/tag/v1.9.0

[pergolafabio]

Hey, can this ticket openend again? I just updated to 1.9.0 , but the audio is not open again the second time
After, the two way audio was in use with the intercom itself (not the card), i still need to send this command to stop the audio, so the card (frigate) can be used again??

"curl -i --digest -u admin:XXX-X PUT http://192.168.0.70/ISAPI/System/TwoWayAudio/channels/1/close"

how can i debug this?

[AlexxIT]

I don't have this cameras, so can't help

[pergolafabio]

Hey @f1d094 , how can I troubleshoot this ? Seems the close command is not used?
Am I right that it always sends the close first before opening two way audio? Want that the purpose of the fix?

[f1d094]

@pergolafabio The proposed fix was accepted and merged :)

https://github.com/AlexxIT/go2rtc/releases/tag/v1.9.0

[pergolafabio]

Yes, I tried 1.9.0, but it doesn't solve the problem for me... When intercom was in use with an two way audio session , if I stop the intercom from ringing, and I start two way audio, I doesn't initiate it... I still need to send the curl before I start the two way audio

[f1d094]

I'm afraid I only have a regluar camera with microphone...my doorbell is just a doorbell.

Looks like it might be time to get some .pcap dumps and use wireshark to look over what is actually happening. The process is probably different. Also have a look at the Hikvision ISAPI docs; somewhere in there should tell you what the expected steps are for the doorbell function and how it works with the TwoWayAudio.

[pergolafabio]

Yeah, I know the docs... The process is like..
Intercom is ringing , two way audio is open...
Then you have a choice, of you can pickup the call with the Hikvision app, but that's what we don't want...
So therefore I send a curl, to stop the Hikvision from ringing, it's also an isapi command...
Afterwards , I want to start initiate the two way audio with frigate... But that only works the first time...

If the second call comes in, I stop the ringing, if I then start two way audio with frigate, I can't hear the audio, unless I send the close channel command..

unless, if I close or browse away from the frigate card, and I refresh the card, then I can initiate again the two way audio.,. If I don't do that, it never works the second time... So it's still related my guess to go2rtc or frigate...

[f1d094]

It is impossible to say at this point. The only way to find out is...to find out. Do a tcpdump of the traffic and open the capture file in wireshark and see what is actually happening. On your Frigate box, try: tcpdump -i $ETH -w isapi-capture.pcap "host $DOORBELL_IP and (dst port 80 or dst port 80)" of course setting ETH and DOORBELL_IP accordingly.

The way my setup works the card is closed and/or opened when I browse to it via the app and it opens and ISAPI two-way audio connection which it keeps open until I close the card. This now works great.

One other thing I can think of to do: Be certain to restart your frigate a couple of times after changing go2rtc version. There is definitely some caching of some sort going on and I find that Frigate will not run smoothly unless I bounce it twice after any major update. No idea why...

[pergolafabio]

Ok, I will do that later... Thnx..

I don't use frigate add-on, only the frigate card

[pergolafabio]

ok, i did a wireshark, and i see indeed ISAPI commands
But on the second try when the camera card is still open, what i do, i toggle the MIC button, or i do unmute the mic
But i dont see any ISAPI commands fired?? i tought toggling the mic button, opens/closes the twowayaudio? But thats not doing that at all ... thats probably the issue for me?

[AlexxIT]

@pergolafabio have you using webrtc card? There is good example in first message.

[pergolafabio]

No, I use the frigate card...
I tested the webrtc card before, that worked... I guess your mic button actually restarts the connection, while frigate only mutes/unmutes

[AlexxIT]

The webrtc card is purposely made to have the stream stops when switching to another stream. Exactly because of these problems with two way audio.

[pergolafabio]

Ok, makes sense, I think the issue can be closed again, it's indeed related to frigate... I need to find a way to actually restart the stream when toggling the mic button

[dermotduffy]

Ok, makes sense, I think the issue can be closed again, it's indeed related to frigate... I need to find a way to actually restart the stream when toggling the mic button

FYI: The Frigate card does restart the stream on the first press of the unmute button. This is how the microphone stream gets attached into it in the first place. The only exception is if you use always_connected: true in which case the microphone stream is attached on initial connection. Perhaps the difference is in stopping the old stream differently.

If you'd like to discuss further, lets discuss over here: https://github.com/dermotduffy/frigate-hass-card/issues

[pergolafabio]

Ok, I had an issue open here:
dermotduffy/frigate-hass-card#1356

[Martinvdm]

@f1d094 i have the exact same camera but behind the hikvision nvr. I setup the webrtc card but no mic is getting to the camera. It is recording in app or browser well, but no sound. Even tested with curl, everything look likes oke (opening/closing just status OK and sessionid), after sending 8khz ulaw, but do not hear anything.
Is this the same issue? still your fork needed?
Can you share some go2rtc etc code?