Monday, August 31, 2015

Simple CSMA firmware for RFD900 radios

I have been continuing to work on getting multiple Mesh Extenders communicating nicely via the RFD900 radio.  

As I have previously mentioned, the challenge with this is that the frequency hopping firmware that they normally run cannot work with multiple radios without a defined network controller.  This just isn't possible with Mesh Extenders, where we expect nodes to wander in and out of coverage on a regular basis.

Fortunately, it is allowable to use the ISM915 band in Australia, the USA, New Zealand and a few others without frequency hopping.  To do this, you must however use very wide channels, or limit your transmit power to not more than 3mW EIRP.  Fortunately, as described in my last post, the RFD900 can still transmit a long way on a couple of milli-Watts EIRP.  While at Arkaroola, we can of course transmit at upto 1W EIRP under our scientific spectrum license.  In the longer term, the next model of radio from RFDesign will allow the use of very wide channels, so the scientific license is really just an interim solution. Anyway, enough of the spectrum regulation side of things ...

The normal RFD900 firmware's frequency hopper uses a Time Division Multiplexing (TDM) scheme, which is great for a pair of radios, or a small group with a defined controller, who can control who has what slice of time.  But as mentioned, that doesn't make sense for Mesh Extenders.

Instead, we need a simple CSMA (Carrier Sense Multiple Access) or similar scheme that allows disorganised radios to communicate with each other on a transient and dynamic basis. The downside to this is that the effective bandwidth will be probably less than 1/3 of what can be achieved using TDM (which is part of why the original firmware uses TDM).

Because we can now also have potentially a lot of radios talking, we need a clear way to mark packet boundaries in the serial stream -- for both transmit and receive -- so that we can reliably send and receive whole packets.

To this end, I have culled all the TDM code out of the RFD900 firmware, and implemented some simple packet framing.

The framing on the receive side we had already implemented for the Mesh Extenders. This just puts a simple prefix on each received packet in the serial buffer. That prefix contains a magic value, as well as some useful statistics, mostly signal strength.

For the transmit side, I have changed the firmware so that it holds onto bytes in the TX buffer until it sees "!!".  To include a ! in the packet it must be escaped as "!.".

After a bit of fiddling around, I was able to get all this working fairly nicely.

The only other piece was to add a simple Listen Before Talk (LBT) function that holds off sending a packet until the ambient noise drops below a threshold.  This is to stop us talking over the top of other stations.  This is vital now that we no longer have TDM to keep the radios from talking over the top of each other.  For now I have a hard-wired threshold, but I will make this configurable soon, and if I can think of a good approach, I will make it dynamic based on the background noise on the channel, so that it doesn't stop transmission in busy urban areas, but still stops talking over the top of distant stations when in low-noise environments.

(In fact, this whole effort was triggered by problems using the existing LBT code in version 1.6 of the RFD900 firmware, on which our Mesh Extender version is based. Basically if you set the LBT register on that firmware, the radio stops responding.  This may well have been fixed in later versions of the firmware.)

Anyway, while there is more work to be done, you can see the work so far at https://github.com/gardners/SiK/tree/csma-packetised.

Friday, August 28, 2015

Testing RFD900 radio at Arkaroola

Last week I was up at Arkaroola for Science Week, and took the opportunity to test the RFD900 radios up there in preparation for some upcoming Mesh Extender testing there.

The idea of the tests was to try the RFD900s out in single-channel (non-hopping) mode, now that we have an experimental spectrum license issued for Arkaroola.  This means that we can run the RFD900 at full power, without frequency hopping enabled, as a precursor to the new radio the RFDesign folks are working on, that can be operated in an equivalent mode without the special spectrum license.

The benefit of all this is that we can completely avoid the synchronisation issue between Mesh Extenders, and thus allow more than just pairs of Mesh Extenders to communicate. This is a big step forward, and one that has been a long time coming!

So, I configured the radios down here in Adelaide, but set to 1mW, so that we would not be in breach of the LIPD Class License under which the RFD900 is typically operated.  This let me confirm that I had them setup properly.

Then, up at Arkaroola, I put one radio in the window of one of the buildings in the Arkaroola Village, and drove up towards Coulthard's Lookout, about 4 - 5km away, to confirm that we could get that kind of range.  We were not expecting any problems, but just wanted to make sure that they worked in the rocky terrain that is full of iron and odd shaped rocks on the ground that can mess with radio propagation.

A little way before Coulthard's Lookout there is a ridge that has excellent line of sight back to the village, and is almost 4km from the village.  So we stopped there to test performance, which confirmed that we were losing only a small percentage of packets.  We did also discover that horizontal polarisation was pretty much useless, and that it was the vertical polarisation that was doing all the work.

Here I am in my "other" office. The village is just visible in the distance.

We then went up to Coulthard's Lookout, and were receiving only perhaps 50% of packets, which rather surprised us, as we had expected better performance, however, it was adequate.  It was only the next morning driving home that I realised that I had left the radio set to 1mW !  So the RFD900 was still getting 50% of packets through at 1/500th the TX power that we could use.

Given that every 4x increase should double the range, this means that we should expect 50% packet throughput at a range more like 85km -- and this is with both radios on the ground. Previous tests indicating ~80km range of the RFD900 has involved one radio being in the air on a UAV.

Part of the reason for the great performance (apart from the great design and implementation of the RFD900) is the REALLY low level of background noise at Arkaroola.  Based on the RSSI indications of the RFD900 it is probably 10 - 20dB lower than in Adelaide.  But this isn't news to us, as we know from previous work that the noise floor at 2.4GHz - 5GHz is around -100dBm, compared with around -80dBm here in Adelaide.

Apparently it was family day in the office.

Thursday, August 13, 2015

Working on the Mesh Extender UHF Radios and other things

It's been a while since the last post, but we have been busy in the background on some exciting things:

With the help of the NLnet Foundation, we are making the first steps towards an iOS port of the Serval Mesh.

With the help of USAID, we are working on "slow-media transports" for Android, in particular using Bluetooth device names, and hopefully, Wi-Fi Direct service discovery information, to carry mesh traffic. This is really significant, because it allows carriage of ad-hoc communications on non-rooted devices.  The trade-off is that it is much slower than ad-hoc Wi-Fi.

The third thing we are working on at the moment is making Mesh Extenders work in groups of more than two units.

Historically we have been stuck with pairs because the radios we are using are required to frequency hop when using the ISM 915MHz band available here in Australia, New Zealand and the USA.

We are now actively working to remove this limitation, using the new radios that RFDesign are creating that allow broader spectrum occupancy that will allow them to qualify as spread spectrum devices under the relevant FCC and ACMA rules (LIPD class 45 for the ACMA rules).  This means that we can drop the frequency hopping that causes all the synchronisation issues for ad-hoc and transient groups of Mesh Extenders that currently exist.

In other words, this will allow people to carry a Mesh Extender, and for it to automatically mesh with any other Mesh Extenders that come into range from time to time.  The practical limit will probably be 5 - 25 Mesh Extenders in range at any one time -- something I hope to quantify better in the near future.

Now, the only trouble is that those radios are not yet finished and available.

So we have embarked on an interim solution, by obtaining a spectrum license from ACMA that will allow us to operate the RFD900 radios without frequency hopping at upto 1W EIRP in our favourite piece of Outback, the Arkaroola Wilderness Sanctuary.

Our scientific spectrum license which was granted earlier in the week allows us use of 915MHz - 928MHz at upto 1W anywhere within 130km of 29'52"35S, 138'52"25E, which is great because this affords us great flexibility for testing in a variety of settings.  

The license stands for the next 12 months, which will hopefully be time enough for us to transition to the new RFDesign radios.

We are also open to making use of this license open to other people who would like to experiment in similar ways in that area (but note that it is really quite remote, 400km from the nearest large town, and centred about 130km from the nearest settlements with any facilities).  We would also need to make sure that ACMA and the University are comfortable with such an arrangement, but I am happy to explore this if anyone is interested.

Meanwhile, I hope to modify the configuration of some radios, and put the latest servald on a couple of Mesh Extenders and, all things going well, to perform some rudimentary testing at Arkaroola this coming week.

Update: This map shows the approximate area of the license.

Monday, March 16, 2015

Mapping assistance for Vanuatu Cyclone Response

We have had a request from the head of the Disaster Research Centre here at Flinders University asking for volunteers to help re-map Vanuatu following the category 5 cyclone that hit there in the last few days.

Here is the call for assistance:

Dear MicroMappers,

The United Nations has just activated MicroMappers to support their rapid damage assessments in Vanatu following the Category 5 Cyclone. You can accelerate the UN's efforts by donating a few minutes of your time. Simply click here: 


No prior experience necessary. If you can click "Like" on a Facebook picture or video, you can be a Digital Humanitarian.

Thank you!"


Monday, March 2, 2015

OSX Yosemite Printer keeps pausing, and gets stuck on "Ready to print"

If you are printing to a PaperCut server, apparently printing can have trouble from a mac running Yosemite, due to the improved security in CUPS 2.0, which is part of Yosemite.

The interim solution is to relax the sandboxing policy in CUPS 2.0, so that it behaves like it used to.  Of course, it would be better if PaperCut fixed the underlying problem in their software.

The short version of what to do is:

sudo sh -c 'echo "Sandboxing Relaxed" >> /etc/cups/cups-files.conf'
sudo launchctl stop org.cups.cupsd

The long version can be found at:

http://www.papercut.com/kb/Main/MacOS1010YosemiteKnownIssues

Sunday, March 1, 2015

MDP, Rhizome, MeshMS and VoMP over Bluetooth

Under a grant from US-AID we are in the process of adding some low-bandwidth transports to the Serval Mesh.  Ultimately, this will hopefully include Wi-Fi Direct service directory, Bluetooth Device Names, as well as using those transports directly.

What Jeremy has achieved so far is to use Bluetooth with an automated peer discovery.  That is, if two phones running Serval Mesh have bluetooth turned on, and at least one of them is marked discoverable, then Serval Mesh on the phones will automatically create a bluetooth serial connection, and start transferring data as though it were another wireless interface, which, after all, it is.

The following photos show Andrew and Jeremy making an authenticated, encrypted VoMP call over Bluetooth:


The in-call latency is probably around 800ms, so quite usable.

And here we have an upside-down MeshMS conversation.  MeshMS messages deliver in just a couple of seconds if there is no other Rhizome traffic.



More posts as we make further progress.

Thursday, January 22, 2015

Mac OSX "Hold for authentication" when trying to print

Today I tried to get printing to work from my mac (running Yosemite) to the printers here at work.  The printers are accessed using SMB.

I could easily add the printers, but when I tried to print, it would accept my username and password for the printer, but then revert straight back to saying "Hold for authentication".

Deleting the printers and recreating them didn't fix it.
Nor did deleting the associated entries from the OSX keyring help.

Thus the usually recommended steps failed to fix the problem.

It took a couple of hours of research and experimentation to discover that this page had most of the information needed to solve the problem.

The problem in my case was caused by the print system on the mac from failing to properly negotiate authentication in some way. But here is how I fixed it:

Step 1: Try to print something to this printer.  Then open the print queue for the printer, and click on the refresh (curly-arrow) icon to the right of the job name.  It should prompt you for your username and password for the print server, and then revert to "Hold for authentication".

This is really important. If you don't try to send a print job and have it stuck in "Hold for authentication" first, strange things can still happen after following the procedure below.  For me, I found that the print job would try to print, but the print queue would then immediately become paused, with the print job marked "Ready to print".  Very frustrating.

Step 2: Open a terminal window.  Everything we do from here will be done from in the terminal window.

Step 3: Find out the name of the printer on the command line by typing the following and pressing return:

lpstat -s

Step 4: Find the printer in question in the output. You will see that the names of the print queues have any fancy characters replaced with an underscore.  So your print queue name might have lots of underscores in it.  For example, my printer called "Tonsley : mono" appears in the output of lpstat -s below as "Tonsley___mono" (highlighted below):

$ lpstat -s
system default destination: Tonsley___colour__A3_in_tray_2_
device for GenericScanner: usb://00000000-0000-0000-FA13-000000000000
device for LaserJet_6MP__Jet_Direct_: socket://192.168.1.200/
device for Lexmark_6200_Series: usb://Lexmark/6200%20Series?serial=21B1280030008E5
device for Loopback: socket://127.0.0.1/
device for Samsung_CLP_310_Series: usb://Samsung/CLP-310%20Series?serial=149RBAFZ400869Z
device for Samsung_CLP_315_Series___alfred: dnssd://Samsung%20CLP-315%20Series%20%40%20alfred._ipp._tcp.local.
device for Samsung_ML_1640_Series: usb://Samsung/ML-1640%20Series?serial=3511BAFS501610N.
device for Tonsley___colour__A3_in_tray_2_: smb://tonsprint.isd.ad.flinders.edu.au/mdf-colour

device for Tonsley___mono: smb://tonsprint.isd.ad.flinders.edu.au/mfd-bw
$

Step 5: Force the mac to use username,password authentication for this printer by typing the following command (but don't forget to change the printer name from Tonsley___mono to the name of your print queue that you obtained in the previous step).  

sudo lpadmin -p Tonlsey___mono -o auth-info-required=username,password

When you hit return after typing this command you will be asked for your mac password.  Type it in.

Step 6: Attempt to print using the printer.  It should now work.  If it doesn't try deleting the printer and following the sequence again.  If the print queue keeps pausing itself and saying the print job is "ready to print", read the important note on step 1.

If this information helps you, please consider donating to http://servalproject.org.