Denon webpage - xml

Intro

Using a github python package from ol-iver I have been able to implement the main amplifier-related Denon functions I want on a web page.

Unfortunately I want more!

It must be possible to obtain from Denon, all the information which is available in the Denon iPad/Android app.   In particular, the app shows a list of all radio station presets from which you can select.  In the protocol specification I did find a telnet command to change preset but there isn't a command to list all presets.

There is a mention in ol-ivers github page of how he used  wireshark to identify XML commands which retrieve information from his Denon AVR.  I took this as a hint that I must use Network packet capture to discover further commands which we can use.  There was also a reference in a Home Assistant Denon forum to the "unofficial port 80 HTTP protocol" which may provide XML using an http request. 

Packet Capture

If I start up wireshark and monitor all packets coming to / from Denon I see nothing useful.  Both the iPad and Denon are on the wifi network and wireshark is running on the PC with an ethernet card or wifi.  Windows isn't very good at putting network cards into monitor / promiscuous mode so I wasn't sure it was capable of doing the capture I want.

Previously I have had better success with a packet analyser called Fiddler.  In particular, it is possible to set up Fiddler on the PC as a proxy server and then configure the iPad to send all its packets via Fiddler.

My copy of Fiddler hasn't been used for a few years and doesn't seem to work.  "Fiddler Everywhere" is a new version of their software which is available for a 10-day free trial, it can also capture https traffic.
I installed Fiddler Everywhere and found it is quite easy to use and works very well.  In fact I will subscribe for a couple of months ($12pcm) as I think it is worthwhile.

Fiddler shows exactly what I want to see, a lot of http requests/responses to/from Denon together with the appropriate message headers and bodies.  The message body is in XML format.  

Packet Analysis

Most of the packets are in POST format but a few are GET requests.  The good thing about GET requests is that you can enter them directly into the browser.  I tried submitting a sample request and Denon responded with an answer written in XML 😀😀

POST requests typically need to be sent using a browser HTML form, but I use an excellent little utility, postman which submits requests and receives responses.  Now I can submit a POST request and receive a response.

Looking through the (hundreds of) requests I saw that most of them are covered by ol-iver's functions.  The POST request which I really want is "GetTunerStatus" which I will describe further below.

Some of the GET requests are interesting.  As shown on the screen shot below, commands within the Denon Protocol specification can be submitted as  a GET request, which is potentially much simpler than ol-ivers telnet interface.  Packet 1449 shows a HTTP GET request to change the radio station preset to number 6.

I can easily add some buttons on the webpage to change the preset value:

Packet 1238 shows a POST request to list all presets and the xml response which is provided

Web page

When I tried any POST requests in my web page I received messages to say that they were blocked owing to CORS (Cross Origin Resource Sharing) security weakness.

If I wanted to I could circumvent this issue by starting chrome on windows with a switch to allow CORS, but I felt this wasnt the way to go and I abandoned the idea.

Instead I returned to python, found a geeksforgeeks explanation for submitting HTTP POST requests and tested whether this would resolve the problem.  Success 😀😀, Python responds with a chunk of XML containing the information I need.

Initially I imported XML to JSON (xmltodict) to make the data accessible but I ended up just searching on the <name> tag to extract the current radio station.

CGI

Rather than submitting web requests using the websocket interface I felt it would be more sensible to use the lighttpd web server python CGI interface.  My python script needed some tweaking to work in CGI (xmltodict didn't work) but I was soon able to report the tuner preset back to the web page.  This was inserted in my draft web page into an iframe.  An early draft is shown below (before formatting).

Result

We now have a properly functional single, simple web page which has all my Denon controls easily accessible and controllable.