Sunday, September 20, 2015

Shaking the bugs down with ad-hoc communications via RFD900

Last week I was up at Arkaroola again working on getting the Mesh Extenders to communicate via the RFD900 radios running my new CSMA firmware using our new spectrum license that allows us to avoid frequency hopping at (relatively) high power levels.

The CSMA firmware itself had very few problems to be solved.  The main change I made was introduce a couple of escape sequences to allow setting the power level of each packet to either low (requiring no spectrum license) or high (requiring a spectrum license).

This was part of a general effort to make the power level easily controllable so that I could set the Mesh Extenders to either "normal" or "Arkaroola" mode, without having to reflash them.

In the end, I implemented two tests that the Mesh Extenders perform before sending a packet at high power: First, the file "HIPOWER.EN" must exist on the USB memory stick on the FAT32 partition, and second, the slider switch on the MR3020 PCB must be set to the middle position. If either of those are not set correctly, then packets will be transmitted at low power.  Thus I can either replace the contents of the USB memory stick, or slide the switch to disable hi-power mode when I return from Arkaroola.  Ideally I should modify the lid design of the 3D-printed case so that I can access the slider switch without unscrewing it, but that's really just a matter of convenience.

So after fixing that, and having a frustrating day while I realised I was using the wrong binary for the RFD900 radios, I managed to send a MeshMS message between a pair of Mesh Extenders operating in this way.  There are still some scheduling bugs in LBARD that prevented the confirmation reply from getting back to the sending phone. I also have a list of other relatively minor issues to work through to get it all operating stably, all of which I hope to work through this week ready for the next trip to Arkaroola, where I hope to be able to operate 8 or more Mesh Extenders together using the RFD900 in this ad-hoc mode, and to be able to send and receive MeshMS messages over a fairly large area.  This will be very satisfying and a significant step forward if I can achieve it.

Monday, August 31, 2015

Simple CSMA firmware for RFD900 radios

I have been continuing to work on getting multiple Mesh Extenders communicating nicely via the RFD900 radio.  

As I have previously mentioned, the challenge with this is that the frequency hopping firmware that they normally run cannot work with multiple radios without a defined network controller.  This just isn't possible with Mesh Extenders, where we expect nodes to wander in and out of coverage on a regular basis.

Fortunately, it is allowable to use the ISM915 band in Australia, the USA, New Zealand and a few others without frequency hopping.  To do this, you must however use very wide channels, or limit your transmit power to not more than 3mW EIRP.  Fortunately, as described in my last post, the RFD900 can still transmit a long way on a couple of milli-Watts EIRP.  While at Arkaroola, we can of course transmit at upto 1W EIRP under our scientific spectrum license.  In the longer term, the next model of radio from RFDesign will allow the use of very wide channels, so the scientific license is really just an interim solution. Anyway, enough of the spectrum regulation side of things ...

The normal RFD900 firmware's frequency hopper uses a Time Division Multiplexing (TDM) scheme, which is great for a pair of radios, or a small group with a defined controller, who can control who has what slice of time.  But as mentioned, that doesn't make sense for Mesh Extenders.

Instead, we need a simple CSMA (Carrier Sense Multiple Access) or similar scheme that allows disorganised radios to communicate with each other on a transient and dynamic basis. The downside to this is that the effective bandwidth will be probably less than 1/3 of what can be achieved using TDM (which is part of why the original firmware uses TDM).

Because we can now also have potentially a lot of radios talking, we need a clear way to mark packet boundaries in the serial stream -- for both transmit and receive -- so that we can reliably send and receive whole packets.

To this end, I have culled all the TDM code out of the RFD900 firmware, and implemented some simple packet framing.

The framing on the receive side we had already implemented for the Mesh Extenders. This just puts a simple prefix on each received packet in the serial buffer. That prefix contains a magic value, as well as some useful statistics, mostly signal strength.

For the transmit side, I have changed the firmware so that it holds onto bytes in the TX buffer until it sees "!!".  To include a ! in the packet it must be escaped as "!.".

After a bit of fiddling around, I was able to get all this working fairly nicely.

The only other piece was to add a simple Listen Before Talk (LBT) function that holds off sending a packet until the ambient noise drops below a threshold.  This is to stop us talking over the top of other stations.  This is vital now that we no longer have TDM to keep the radios from talking over the top of each other.  For now I have a hard-wired threshold, but I will make this configurable soon, and if I can think of a good approach, I will make it dynamic based on the background noise on the channel, so that it doesn't stop transmission in busy urban areas, but still stops talking over the top of distant stations when in low-noise environments.

(In fact, this whole effort was triggered by problems using the existing LBT code in version 1.6 of the RFD900 firmware, on which our Mesh Extender version is based. Basically if you set the LBT register on that firmware, the radio stops responding.  This may well have been fixed in later versions of the firmware.)

Anyway, while there is more work to be done, you can see the work so far at

Friday, August 28, 2015

Testing RFD900 radio at Arkaroola

Last week I was up at Arkaroola for Science Week, and took the opportunity to test the RFD900 radios up there in preparation for some upcoming Mesh Extender testing there.

The idea of the tests was to try the RFD900s out in single-channel (non-hopping) mode, now that we have an experimental spectrum license issued for Arkaroola.  This means that we can run the RFD900 at full power, without frequency hopping enabled, as a precursor to the new radio the RFDesign folks are working on, that can be operated in an equivalent mode without the special spectrum license.

The benefit of all this is that we can completely avoid the synchronisation issue between Mesh Extenders, and thus allow more than just pairs of Mesh Extenders to communicate. This is a big step forward, and one that has been a long time coming!

So, I configured the radios down here in Adelaide, but set to 1mW, so that we would not be in breach of the LIPD Class License under which the RFD900 is typically operated.  This let me confirm that I had them setup properly.

Then, up at Arkaroola, I put one radio in the window of one of the buildings in the Arkaroola Village, and drove up towards Coulthard's Lookout, about 4 - 5km away, to confirm that we could get that kind of range.  We were not expecting any problems, but just wanted to make sure that they worked in the rocky terrain that is full of iron and odd shaped rocks on the ground that can mess with radio propagation.

A little way before Coulthard's Lookout there is a ridge that has excellent line of sight back to the village, and is almost 4km from the village.  So we stopped there to test performance, which confirmed that we were losing only a small percentage of packets.  We did also discover that horizontal polarisation was pretty much useless, and that it was the vertical polarisation that was doing all the work.

Here I am in my "other" office. The village is just visible in the distance.

We then went up to Coulthard's Lookout, and were receiving only perhaps 50% of packets, which rather surprised us, as we had expected better performance, however, it was adequate.  It was only the next morning driving home that I realised that I had left the radio set to 1mW !  So the RFD900 was still getting 50% of packets through at 1/500th the TX power that we could use.

Given that every 4x increase should double the range, this means that we should expect 50% packet throughput at a range more like 85km -- and this is with both radios on the ground. Previous tests indicating ~80km range of the RFD900 has involved one radio being in the air on a UAV.

Part of the reason for the great performance (apart from the great design and implementation of the RFD900) is the REALLY low level of background noise at Arkaroola.  Based on the RSSI indications of the RFD900 it is probably 10 - 20dB lower than in Adelaide.  But this isn't news to us, as we know from previous work that the noise floor at 2.4GHz - 5GHz is around -100dBm, compared with around -80dBm here in Adelaide.

Apparently it was family day in the office.

Thursday, August 13, 2015

Working on the Mesh Extender UHF Radios and other things

It's been a while since the last post, but we have been busy in the background on some exciting things:

With the help of the NLnet Foundation, we are making the first steps towards an iOS port of the Serval Mesh.

With the help of USAID, we are working on "slow-media transports" for Android, in particular using Bluetooth device names, and hopefully, Wi-Fi Direct service discovery information, to carry mesh traffic. This is really significant, because it allows carriage of ad-hoc communications on non-rooted devices.  The trade-off is that it is much slower than ad-hoc Wi-Fi.

The third thing we are working on at the moment is making Mesh Extenders work in groups of more than two units.

Historically we have been stuck with pairs because the radios we are using are required to frequency hop when using the ISM 915MHz band available here in Australia, New Zealand and the USA.

We are now actively working to remove this limitation, using the new radios that RFDesign are creating that allow broader spectrum occupancy that will allow them to qualify as spread spectrum devices under the relevant FCC and ACMA rules (LIPD class 45 for the ACMA rules).  This means that we can drop the frequency hopping that causes all the synchronisation issues for ad-hoc and transient groups of Mesh Extenders that currently exist.

In other words, this will allow people to carry a Mesh Extender, and for it to automatically mesh with any other Mesh Extenders that come into range from time to time.  The practical limit will probably be 5 - 25 Mesh Extenders in range at any one time -- something I hope to quantify better in the near future.

Now, the only trouble is that those radios are not yet finished and available.

So we have embarked on an interim solution, by obtaining a spectrum license from ACMA that will allow us to operate the RFD900 radios without frequency hopping at upto 1W EIRP in our favourite piece of Outback, the Arkaroola Wilderness Sanctuary.

Our scientific spectrum license which was granted earlier in the week allows us use of 915MHz - 928MHz at upto 1W anywhere within 130km of 29'52"35S, 138'52"25E, which is great because this affords us great flexibility for testing in a variety of settings.  

The license stands for the next 12 months, which will hopefully be time enough for us to transition to the new RFDesign radios.

We are also open to making use of this license open to other people who would like to experiment in similar ways in that area (but note that it is really quite remote, 400km from the nearest large town, and centred about 130km from the nearest settlements with any facilities).  We would also need to make sure that ACMA and the University are comfortable with such an arrangement, but I am happy to explore this if anyone is interested.

Meanwhile, I hope to modify the configuration of some radios, and put the latest servald on a couple of Mesh Extenders and, all things going well, to perform some rudimentary testing at Arkaroola this coming week.

Update: This map shows the approximate area of the license.

Monday, March 16, 2015

Mapping assistance for Vanuatu Cyclone Response

We have had a request from the head of the Disaster Research Centre here at Flinders University asking for volunteers to help re-map Vanuatu following the category 5 cyclone that hit there in the last few days.

Here is the call for assistance:

Dear MicroMappers,

The United Nations has just activated MicroMappers to support their rapid damage assessments in Vanatu following the Category 5 Cyclone. You can accelerate the UN's efforts by donating a few minutes of your time. Simply click here: 

No prior experience necessary. If you can click "Like" on a Facebook picture or video, you can be a Digital Humanitarian.

Thank you!"

Monday, March 2, 2015

OSX Yosemite Printer keeps pausing, and gets stuck on "Ready to print"

If you are printing to a PaperCut server, apparently printing can have trouble from a mac running Yosemite, due to the improved security in CUPS 2.0, which is part of Yosemite.

The interim solution is to relax the sandboxing policy in CUPS 2.0, so that it behaves like it used to.  Of course, it would be better if PaperCut fixed the underlying problem in their software.

The short version of what to do is:

sudo sh -c 'echo "Sandboxing Relaxed" >> /etc/cups/cups-files.conf'
sudo launchctl stop org.cups.cupsd

The long version can be found at:

Sunday, March 1, 2015

MDP, Rhizome, MeshMS and VoMP over Bluetooth

Under a grant from US-AID we are in the process of adding some low-bandwidth transports to the Serval Mesh.  Ultimately, this will hopefully include Wi-Fi Direct service directory, Bluetooth Device Names, as well as using those transports directly.

What Jeremy has achieved so far is to use Bluetooth with an automated peer discovery.  That is, if two phones running Serval Mesh have bluetooth turned on, and at least one of them is marked discoverable, then Serval Mesh on the phones will automatically create a bluetooth serial connection, and start transferring data as though it were another wireless interface, which, after all, it is.

The following photos show Andrew and Jeremy making an authenticated, encrypted VoMP call over Bluetooth:

The in-call latency is probably around 800ms, so quite usable.

And here we have an upside-down MeshMS conversation.  MeshMS messages deliver in just a couple of seconds if there is no other Rhizome traffic.

More posts as we make further progress.