iPython
iPython is a must for any python programming. It is simple to install (with pip3), can be used just like the standard interpreter with no learning curve and is packed with useful features. Features which made my life better from day 1 include:
1) being able to a stop a program at a given point and inspect the variables
2) Hitting tab after an obect name to display a list of its properties.
Python CLI
python command line programs are useful when using linux. In particular Python is great for simple utilities but it is a nuisance to have to run them from specific directories. Python CLI makes this easier. Thomas Stringer has provided a simple example. I also found an explanation on Stack Overflow on usage helpful.
Sunday 17 February 2019
Monday 11 February 2019
XWindows on WSL
I generally use linux headless as the functions I need can be initiated and controlled from the command line. It occured to me that there are potentially some GUI functions which would be useful and my initial reaction is to have an RPI build with a desktop.
However the idea is very uninspiring and it then occured to me that using Xwindows software on Windows I could get GUI sessions without an RPI desktop.
Xming provides a simple popular solution which works as expected.
Mobaxterm also provides an Xserver which is integrated into its other functions.
Clearly an Xwindows server is not much use without software. As my RPIs dont have desktop programs installed I cant do much with them by default.
However the idea is very uninspiring and it then occured to me that using Xwindows software on Windows I could get GUI sessions without an RPI desktop.
Xming provides a simple popular solution which works as expected.
Mobaxterm also provides an Xserver which is integrated into its other functions.
Clearly an Xwindows server is not much use without software. As my RPIs dont have desktop programs installed I cant do much with them by default.
It occured to me that WSL (Windows Sub-system for Linux) would be much better as a source for client applications at least for playing/testing. WSL doesn't include a GUI so Xming Xserver can be used on Windows show windows, the server should be started automatically at startup or before using any sessions.
To get a terminal window you can use Putty. Simply create a new putty session and in Connection>SSH>X11 tick "enable X11 forwarding" and specify "X display location" as localhost 0:0. The connection userid and password are john/secret.
WSL doesn't have any GUI software either so we have to install some for testing. I found this article very helpful. I can get some sample GUI programs by installing x11-apps. Before I did that I had to do an apt update/upgrade on Windows and this took a couple of attempts and quite a lot of CPU time for installation.
Once x11-apps is installed I can add "export DISPLAY=:0" at the end of .bashrc.
xeyes is a nice first program and xclock is useful.
xeyes is a nice first program and xclock is useful.
I then installed a terminal window (xterm) and editor (gedit), both of which required significant further software additions to WSL.
I now have a basic useful Xwindows system to use a basis for understanding better how it works.
As I run RPIs headless I dont have x-windows installed cant run x-windows based programs. However as I now have x-windows on Windows-10 and x-windows is famous for being able to run windows on remote machines.
1) On Windows-10 WSL install x-utilities: apt-get install x11-xserver-utils
2) On Windows-10 WSL permit RPI to start sessions: xhost +
3) On RPI install some x-windows apps: apt-get install x11-apps xterm
4) On RPI command line send remote display: export DISPLAY=192.168.0.5:0
5) On RPI command line run x-app: xeyes &
6) App now runs and displays a window on Windows-10
I tested this successfully with paprefs, which is one of the programs I have "missed"
This article maybe useful
As I run RPIs headless I dont have x-windows installed cant run x-windows based programs. However as I now have x-windows on Windows-10 and x-windows is famous for being able to run windows on remote machines.
1) On Windows-10 WSL install x-utilities: apt-get install x11-xserver-utils
2) On Windows-10 WSL permit RPI to start sessions: xhost +
3) On RPI install some x-windows apps: apt-get install x11-apps xterm
4) On RPI command line send remote display: export DISPLAY=192.168.0.5:0
5) On RPI command line run x-app: xeyes &
6) App now runs and displays a window on Windows-10
I tested this successfully with paprefs, which is one of the programs I have "missed"
This article maybe useful
Multiroom audio using DLNA devices
The story so far
Initial investigations to stream audio from MPD to multiple DLNA devices using Linux centred on finding a Linux utility which already implemented the necessary functionality. When that failed a node.js solution was investigated. Again I came to a dead end - although it turned out in retrospect to be a working solution. On the basis that a programmed solution is required it was sensible to concentrate on python as there is a huge amount of software available, it is easy to implement and generally easier to understand the programs.
Looking for a python solution
Initially I looked for linux-based python audio players to familiarise myself with considerations for playing files and streams using python. There wasn't as much choice as I expected, pygame, pyaudio and pyglet looked more complicated than they needed to be but playsound was a good simple solution to play files.
Looking for a program to play on dlna devices didn't have many candidates but nanodlna seemed promising. It is a command line utility written by Gabriel Magno which lists DLNA devices and plays a video. It was installed using pip but didn't work initially. I don't have a solid python programming environment so there wasn't much thought put into installation. The source programs were simple and I adjusted them so I could run nanodlna using the python3 command. For the first test I just specified a music track and no destination which causes nanodlna to pick a device at random. I was surprised and excited that it worked perfectly and played good quality music via a jongo.
nanodlna investigation - request content
nanodlna comprises 4 source files:
cli.py the main program which parses the command and calls other sources.
devices.py identify dlna devices attached to local network
streaming.py reads a music file so it can be played
dlna.py sends instructions to DLNA device for playing music.
I hacked the code extensively to see how it works. In particular I printed variables and replaced variables by their values in the program to see exactly what information was used. dlna.py sends 2 http instructions to a Jongo. The first includes the URI of the the file to be played and the second is an instruction to start playing it. The standard python http request urllib.request is used to send the information to the jongo. It comprises an http header and an xml body formatted for SOAP.
headers = {
'Connection': 'close',
'Content-Type': 'text/xml; charset="utf-8"',
'Content-Length': '465',
'SOAPACTION': '"urn:schemas-upnp-org:service:AVTransport:1#SetAVTransportURI"'}
action_data = b'<?xml version=\'1.0\' encoding=\'utf-8\'?>\n \
<s:Envelope s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" \
xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">\n \
<s:Body>\n \
<u:SetAVTransportURI xmlns:u="urn:schemas-upnp-org:service:AVTransport:1">\n \
<InstanceID>0</InstanceID>\n \
<CurrentURI>http://192.168.0.33:9000/file_video/kylie.mp3</CurrentURI>\n \
<CurrentURIMetaData></CurrentURIMetaData>\n \
</u:SetAVTransportURI>\n \
</s:Body>\n</s:Envelope>\n'
headers = {
'Connection': 'close',
'Content-Type': 'text/xml; charset="utf-8"',
'SOAPACTION': '"urn:schemas-upnp-org:service:AVTransport:1#Play"',
'Content-Length': '337'}
action_data = b'<?xml version=\'1.0\' encoding=\'utf-8\'?>\n \
<s:Envelope s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" \
xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">\n \
<s:Body>\n \
<u:Play xmlns:u="urn:schemas-upnp-org:service:AVTransport:1">\n \
<InstanceID>0</InstanceID>\n \
<Speed>1</Speed>\n \
</u:Play>\n \
</s:Body>\n \
</s:Envelope>\n'
This is extremely helpful. I could have found the same information by looking at TCP traffic using tcpdump or wireshark but this is much easier to understand. All requests to Jongos to play a track are identical, the only information which changes in these requests is the name of the track.
Previously I supposed that the python program streamed all chunks of a track to the Jongo. Clearly what actually happens is that the python program only has to send the name of the track to be played in a SetAVTransportURI request and then send a Play request. It doesn't seem necessary to know any more about the requests but Gabriel Magno provides a link to the official spec. The DLNA control device (remote/phone/RPI etc) doesn't need to communicate or even be switched on whilst the Media Renderer (Jongo) is playing music. The controller is only required to change instructions.
The program streaming.py uses a framework called Twisted Web to load a file into an internal web server so that it can be streamed. In practice I don't need that I have a lot of files already hosted on RPI lighttpd webservers which I can use. I was thus able to dispense with al the Twisted Web functionality and choose a URL directly.
nanodlna investigation - addressing
These dlna requests need to be sent to one or more Jongos. The address information discovered by nanodlna in devices.py is:
# print("video_data");print(video_data);
# deviceJon = {'st': 'urn:schemas-upnp-org:service:AVTransport:1', \
# 'friendly_name': 'Jon', 'hostname': '192.168.0.102', \
# 'action_url': 'http://192.168.0.102:48567/Control/org.mpris.MediaPlayer2'+ \
# '.mansion/RygelAVTransport', \
# 'location': 'http://192.168.0.102:48567/93b2abac-cb6a-4857-b891-0019f5844dd8.xml'}
# deviceJoe = {"st": "urn:schemas-upnp-org:service:AVTransport:1",
# "action_url": "http://192.168.0.122:55746/Control/org.mpris.MediaPlayer2.mansion/RygelAVTransport",
# "friendly_name": "Joe",
# "hostname": "192.168.0.122",
# "location": "http://192.168.0.122:55746/93b2abac-cb6a-4857-b891-0019f584c8f8.xml"
# }
From this we can see that a "random" port is used on the Jongo as the destination and there is some sort of file structure within the jongo. The port number changes from time to time, perhaps when a Jongo is rebooted. This information which is discovered using SSDP protocol. It was a simple job to change the program to send requests to two Jongos and I ascertained that they synchronise quite well. Listening to them in the same room gives a slight echo but was not irritating.
Streaming - curl
At this stage I stumbled across an excellent article which shows how to submit requests to uPnP devices using the Linux command line utility curl. The example provided closely resembles what I have found with python. I was quickly able to setup simple scripts / files assisted by this blog to allow me to send various requests to Jongos much more quickly than amending a python program each time.
In particular it seemed reasonable that the jongo should accept stream URLs. I tried a radio station and it worked fine. I tried an httpd stream from MPD and I had exactly the same problem as previously with node.js - the stream started for a moment then stopped. This time around, I realised that MPD was the problem and I looked more closely at setting up MPD shout cast streaming. I found a good RPI centric article and realised I needed to install icecast before MPD shout output would work. Once I had done this jongos played my mpd out perfectly. The solution doesn't require pulseaudio at all so it is simpler. I did note that changing tracks or streams within the muse application requires Jongo streams to be restarted.
uPnP discovery - udp multicast investigation
upnp hacks provides a good explanation of how uPnP devices find each other on a LAN. A new device uses the UDP protocol to send a multicast request which all others must reply with a UDP unicast message. Ideally we would like a linux command line tool to do this for us. netcat can easily send requests but I couldn't find an easy way to view the responses.
The most common solution appears to use Python UDP capabiilities to send UDP multicast and receive replies just like devices.py in nanodlna. The electric monk provides a lot more detail.
The best way to carry out a command line discovery is gssdp-discover from gupnp-tools:
gssdp-discover -i wlan0 --timeout=3 --target=urn:schemas-upnp-org:device:MediaRenderer:1
This will show media render devices on the LAN :
Using network interface wlan0
Scanning for resources matching urn:schemas-upnp-org:device:MediaRenderer:1
Showing "available" messages
resource available
USN: uuid:93b2abac-cb6a-4857-b891-0019f5844dd8::urn:schemas-upnp-org:device:MediaRenderer:1
Location: http://192.168.0.102:48567/93b2abac-cb6a-4857-b891-0019f5844dd8.xml
resource available
USN: uuid:93b2abac-cb6a-4857-b891-0019f584c8f8::urn:schemas-upnp-org:device:MediaRenderer:1
Location: http://192.168.0.122:55746/93b2abac-cb6a-4857-b891-0019f584c8f8.xml
resource available
USN: uuid:93b2abac-cb6a-4857-b891-0019f584dcf0::urn:schemas-upnp-org:device:MediaRenderer:1
Location: http://192.168.0.118:56405/93b2abac-cb6a-4857-b891-0019f584dcf0.xml
Initial investigations to stream audio from MPD to multiple DLNA devices using Linux centred on finding a Linux utility which already implemented the necessary functionality. When that failed a node.js solution was investigated. Again I came to a dead end - although it turned out in retrospect to be a working solution. On the basis that a programmed solution is required it was sensible to concentrate on python as there is a huge amount of software available, it is easy to implement and generally easier to understand the programs.
Looking for a python solution
Initially I looked for linux-based python audio players to familiarise myself with considerations for playing files and streams using python. There wasn't as much choice as I expected, pygame, pyaudio and pyglet looked more complicated than they needed to be but playsound was a good simple solution to play files.
Looking for a program to play on dlna devices didn't have many candidates but nanodlna seemed promising. It is a command line utility written by Gabriel Magno which lists DLNA devices and plays a video. It was installed using pip but didn't work initially. I don't have a solid python programming environment so there wasn't much thought put into installation. The source programs were simple and I adjusted them so I could run nanodlna using the python3 command. For the first test I just specified a music track and no destination which causes nanodlna to pick a device at random. I was surprised and excited that it worked perfectly and played good quality music via a jongo.
nanodlna investigation - request content
nanodlna comprises 4 source files:
cli.py the main program which parses the command and calls other sources.
devices.py identify dlna devices attached to local network
streaming.py reads a music file so it can be played
dlna.py sends instructions to DLNA device for playing music.
I hacked the code extensively to see how it works. In particular I printed variables and replaced variables by their values in the program to see exactly what information was used. dlna.py sends 2 http instructions to a Jongo. The first includes the URI of the the file to be played and the second is an instruction to start playing it. The standard python http request urllib.request is used to send the information to the jongo. It comprises an http header and an xml body formatted for SOAP.
headers = {
'Connection': 'close',
'Content-Type': 'text/xml; charset="utf-8"',
'Content-Length': '465',
'SOAPACTION': '"urn:schemas-upnp-org:service:AVTransport:1#SetAVTransportURI"'}
action_data = b'<?xml version=\'1.0\' encoding=\'utf-8\'?>\n \
<s:Envelope s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" \
xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">\n \
<s:Body>\n \
<u:SetAVTransportURI xmlns:u="urn:schemas-upnp-org:service:AVTransport:1">\n \
<InstanceID>0</InstanceID>\n \
<CurrentURI>http://192.168.0.33:9000/file_video/kylie.mp3</CurrentURI>\n \
<CurrentURIMetaData></CurrentURIMetaData>\n \
</u:SetAVTransportURI>\n \
</s:Body>\n</s:Envelope>\n'
headers = {
'Connection': 'close',
'Content-Type': 'text/xml; charset="utf-8"',
'SOAPACTION': '"urn:schemas-upnp-org:service:AVTransport:1#Play"',
'Content-Length': '337'}
action_data = b'<?xml version=\'1.0\' encoding=\'utf-8\'?>\n \
<s:Envelope s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" \
xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">\n \
<s:Body>\n \
<u:Play xmlns:u="urn:schemas-upnp-org:service:AVTransport:1">\n \
<InstanceID>0</InstanceID>\n \
<Speed>1</Speed>\n \
</u:Play>\n \
</s:Body>\n \
</s:Envelope>\n'
This is extremely helpful. I could have found the same information by looking at TCP traffic using tcpdump or wireshark but this is much easier to understand. All requests to Jongos to play a track are identical, the only information which changes in these requests is the name of the track.
Previously I supposed that the python program streamed all chunks of a track to the Jongo. Clearly what actually happens is that the python program only has to send the name of the track to be played in a SetAVTransportURI request and then send a Play request. It doesn't seem necessary to know any more about the requests but Gabriel Magno provides a link to the official spec. The DLNA control device (remote/phone/RPI etc) doesn't need to communicate or even be switched on whilst the Media Renderer (Jongo) is playing music. The controller is only required to change instructions.
The program streaming.py uses a framework called Twisted Web to load a file into an internal web server so that it can be streamed. In practice I don't need that I have a lot of files already hosted on RPI lighttpd webservers which I can use. I was thus able to dispense with al the Twisted Web functionality and choose a URL directly.
nanodlna investigation - addressing
These dlna requests need to be sent to one or more Jongos. The address information discovered by nanodlna in devices.py is:
# print("video_data");print(video_data);
# deviceJon = {'st': 'urn:schemas-upnp-org:service:AVTransport:1', \
# 'friendly_name': 'Jon', 'hostname': '192.168.0.102', \
# 'action_url': 'http://192.168.0.102:48567/Control/org.mpris.MediaPlayer2'+ \
# '.mansion/RygelAVTransport', \
# 'location': 'http://192.168.0.102:48567/93b2abac-cb6a-4857-b891-0019f5844dd8.xml'}
# deviceJoe = {"st": "urn:schemas-upnp-org:service:AVTransport:1",
# "action_url": "http://192.168.0.122:55746/Control/org.mpris.MediaPlayer2.mansion/RygelAVTransport",
# "friendly_name": "Joe",
# "hostname": "192.168.0.122",
# "location": "http://192.168.0.122:55746/93b2abac-cb6a-4857-b891-0019f584c8f8.xml"
# }
From this we can see that a "random" port is used on the Jongo as the destination and there is some sort of file structure within the jongo. The port number changes from time to time, perhaps when a Jongo is rebooted. This information which is discovered using SSDP protocol. It was a simple job to change the program to send requests to two Jongos and I ascertained that they synchronise quite well. Listening to them in the same room gives a slight echo but was not irritating.
Streaming - curl
At this stage I stumbled across an excellent article which shows how to submit requests to uPnP devices using the Linux command line utility curl. The example provided closely resembles what I have found with python. I was quickly able to setup simple scripts / files assisted by this blog to allow me to send various requests to Jongos much more quickly than amending a python program each time.
In particular it seemed reasonable that the jongo should accept stream URLs. I tried a radio station and it worked fine. I tried an httpd stream from MPD and I had exactly the same problem as previously with node.js - the stream started for a moment then stopped. This time around, I realised that MPD was the problem and I looked more closely at setting up MPD shout cast streaming. I found a good RPI centric article and realised I needed to install icecast before MPD shout output would work. Once I had done this jongos played my mpd out perfectly. The solution doesn't require pulseaudio at all so it is simpler. I did note that changing tracks or streams within the muse application requires Jongo streams to be restarted.
uPnP discovery - udp multicast investigation
upnp hacks provides a good explanation of how uPnP devices find each other on a LAN. A new device uses the UDP protocol to send a multicast request which all others must reply with a UDP unicast message. Ideally we would like a linux command line tool to do this for us. netcat can easily send requests but I couldn't find an easy way to view the responses.
The most common solution appears to use Python UDP capabiilities to send UDP multicast and receive replies just like devices.py in nanodlna. The electric monk provides a lot more detail.
The best way to carry out a command line discovery is gssdp-discover from gupnp-tools:
gssdp-discover -i wlan0 --timeout=3 --target=urn:schemas-upnp-org:device:MediaRenderer:1
This will show media render devices on the LAN :
Using network interface wlan0
Scanning for resources matching urn:schemas-upnp-org:device:MediaRenderer:1
Showing "available" messages
resource available
USN: uuid:93b2abac-cb6a-4857-b891-0019f5844dd8::urn:schemas-upnp-org:device:MediaRenderer:1
Location: http://192.168.0.102:48567/93b2abac-cb6a-4857-b891-0019f5844dd8.xml
resource available
USN: uuid:93b2abac-cb6a-4857-b891-0019f584c8f8::urn:schemas-upnp-org:device:MediaRenderer:1
Location: http://192.168.0.122:55746/93b2abac-cb6a-4857-b891-0019f584c8f8.xml
resource available
USN: uuid:93b2abac-cb6a-4857-b891-0019f584dcf0::urn:schemas-upnp-org:device:MediaRenderer:1
Location: http://192.168.0.118:56405/93b2abac-cb6a-4857-b891-0019f584dcf0.xml
Subscribe to:
Posts (Atom)