ESP32 SSH

breezy_ssh

My breezybox project provided me with a terminal shell for ESP32 through a PC USB COM port complete with unix-style commands.  This was a very good start but I really want remote access to ESP32 which would give me freedom to connect from various devices and wouldn't require the ESP32 device to be attached to my PC.

Breezybox has an additional component called breezy_ssh which implements an SSH server alongside breezybox to meet this requirement.  Building on knowledge gained from the breezybox install breezy_ssh installation was very straightforward (it was a breeze ;)).

I downloaded an example provided by Valentin Danylchuk which included an sdkconfig and partition table.  After a little tweak the build was successfully flashed to ESP32-S3 and started up. Using the serial console I could see that breezybox was working and I could use the file system as previously.  In addition there was a prompt to start the ssh server.

The first action is to connect to wifi ("wifi connect") and bring up the ssh server ("sshd start").  I could then start an ssh client session on the PC😊😊.

Using breezy_ssh

The SSH client session works much better than a serial console.  I/O redirection works so I can echo text and redirect (">") it to a file.  wget is now working so I can download files using http.  vi works so I can edit files locally.  This gives me everything I need to set up a simple web site, just containing an index.html file.  I then start a web server ("httpd web") and access it from my browser.

The "big picture" below shows COM ports, serial console, ssh session and web access.

One downside with breezy_ssh is that I need to use the serial console to start wifi and ssh.  Generously, Valentin does provide the ability to run shell scripts and a script "init.sh" is executed when you start up.  Unfortunately the script didn't work for me when I put "wifi connect" in init.sh, so I have to start it manually.  With further effort I could probably arrange for wifi and sshd to start within the program or diagnose how to make init.sh work.

libssh

I felt breezybox + breezy_ssh looked a bit complicated for my needs and wanted to look at alternatives.

libssh, written by David Cermak, is the ssh library for ESP32 IDF and is used within breezybox. I decided to test the example he provides (esp_ssh).  Initially I tried to use IDFv6 both VSC and CLI versions.  Surprisingly it wouldn't compile.  I found this is due to "breaking changes" introduced between ESPv5 and ESPv6.  David does provide a "Shim" to enable libssh to work with IDFv6 but I didn't have much success with it.
Instead I used IDFv5 to and the libssh example builds and flashes without problems. I can start ESP32S3, wifi and ssh server are started automatically.  I can ssh in from WSL or Putty, which works better.  Any text typed into SSH client is echoed on the serial console. There are only three command reset, hello (hello world message) and exit.

The example is much simpler than breezybox which is good.  Naturally it has less functionality and I would need to write my own terminal shell.  

Libssh is the base library used for ssh on ESP32 with IDF and therefore the simplest possible implementation.  Looking at the source code I see that there is much complexity to deal with ssh authentication, creating and checking keys etc.  

For my purposes I don't actually need any security as I only intend to use CLI access on the local network.  This thought leads me to try telnet access in the next stage of my investigation.

 

Breezybox Terminal

 Rationale

The idea of running a terminal shell on an ESP32 appeals to me.  Normally you download a program to ESP which then does what it has been instructed in the source code.  Input and output are either carried out through peripherals (buttons, keypad, OLED screen, LEDs, etc.) or a web page which is built to display results or accept instructions (buttons, text boxes etc).

A terminal shell allows typing inputs on a keyboard and showing outputs on a screen.  You can run a command and display results using a Command Line Interface (CLI).  I have used CLI for over 40 years and it makes me feel I have a real computer under my control rather than pre-programmed hardware.

Previously I have tried out Shellminator on an ESP32-C3.  However, I am interested in using Breezybox which has many more capabilities including a file system, together with associated commands, and SSH access.

ESPShell

I purchased an ESP32-S3 N16R8 for the new project.  Although most current ESP32 models are RISC-V, ESP32-S3 runs more powerful Xtensa cores.  This model comes with an extra 8MB PSRAM (pseudo-static RAM, memory contents are preserved when powered off) and 16MB flash memory.  It has no SD card or screen.

As a first test, before working on Breezybox I tried ESPShell.  Installation is as simple as installing the library and adding the header ("#include espshell.h") to your Arduino sketch.  There is no need to amend setup() or loop() functions!  Results are very impressive, a set of commands to tell you about your system.  Arduino serial monitor doesn't display well so a TeraTerm session is preferred. It gives a lot more information than Shellminator.

Breezybox Headaches

Although ESPShell is very good I would like to try the Unix like interface, SSH access and other features available on Breezybox.

In contrast to ESPShell, breezybox required plenty of time and patience.  It hasn't been developed for Arduino meaning I had to use the more complex/advanced/complete ESP-IDF environment.

At first I had no success, I could not build an image either using ESP-IDF CLI or VSC (Visual Studio Code).   Firstly there were errors mainly relating to "json" not found which I circumvented.  Then came "usb_serial_jtag not found" errors.

Eventually it occurred to me that I am using a recent major new version ESP-IDF v6.0.1.  Perhaps  Breezybox had been developed using IDF v5.  It turns out there are major "breaking" changes in the new IDF version.  I installed IDF v5.5.5 which was the last one before IDF v6 and the breezybox build worked.

Breezybox Build

In fact, with the correct environment, installation and build are incredibly easy using the ESP-IDF framework.

Although the build was successful, I had more work to do using "IDF menuconfig" to configure appropriate options.

The image was bigger than the default 0x100000.  So we need to add a partitions.csv file which gives us 0x2000000 in the "factory partition".

Next I had to configure 16MB RAM

Finally I had to configure the USB COM port to be used as the system console.

At last I have a working image.

 

Using Breezybox

Breezybox has its own virtual filesystem in Flash memory.  Its initial size is 1.5M.  The only file existing initially is "init.sh" which echoes a message "Welcome to Breezybox".  A set of commands  is provided to manage files.  

Other useful shell commands are also provided for system status, networking.

More interestingly there is a command "httpd" to start a web server and "eget" to download executable apps to the system.

Adding memory

The first thing we want to do is use eget to download some apps to provide us with extra functionality.  Unfortunately the system is severly memory constrained.  We only have 300k SRAM (static RAM) available.
The solution is to configure to configure our 8MB PSRAM for our use.  In menuconfig we enable PSIRAM.  It happens that our PSRAM is "Octal" which also needs to specified.

Now when the image is rebuilt we have 8MB PSRAM available.

Now we can download apps from valdanylchuk/breezybox.   The ones I want to use are wget (to download files via http) and vi (to edit files locally).  Unfortunately neither appear to work on the serial console.  In addition I/O redirection wasn't working for me, so I had no way of creating webpages so that I could bring up the web server (httpd).

Instead I tried adding a custom command to breezybox source code.  This was a very helpful example as  it worked straightaway and showed me how to register an ESP console command so that it can be used on the command line.

Outro

At this stage I have working breezybox environment although it is not fully functional through the serial console.

A lot of progress has been made, with Google's help, in understanding my ESP32S3 N16R8 system memory/flash and how to use IDF menuconfig.

It is rewarding to understand the complexity of ESP-IDF a little bit more.  It also helps me appreciate the magic which Arduino does to hide/disguise most of this to give casual users the ability to build ESP32 images.

I don't really want to use a serial console to control ESP32, I would like a remote console.  Breezybox provides a component breezy_ssh which provides an ssh server and allows standard access over wifi.  This is the next area to test.