Tuesday, November 22, 2016

Exploring the legal situation of disaster communications networks

I'm currently based in Darmstadt, Germany for a few months with the nice folks in the Secure Mobile Communications laboratory (SEEMOO) at TU-Darmstadt.

One of the main things that we are looking at while I am here, is to better understand the current legal situation facing networks like Serval.

This is important, because the telecommunications laws in most countries were made before such networks were beginning to be widely considered.  As a result, the particular characteristics and requirements of these networks are typically not accommodated in the telecommunications regulations of most countries (or at least the ones we have looked at so far).

So we are hoping over the next month or so to compare the situation here in Germany with Australia, and perhaps a few other countries, to see what road-block might be there, and perhaps to propose how such legislation could be amended to better facilitate them.

As a result I have spent much of the last week reading through the EU Directive 2014/53/EU (in both English and German), as well as draft legislation to implement it in Germany and Austria.

There are some points of concern that we have, particularly that open-source development using wireless routers and SDRs would effectively be outlawed, if the legislation is not carefully worded.  This would be a very unfortunate outcome, not just for the open-source communities who volunteer their efforts and donate the results of their efforts to the common good, but to society as a whole, who would be denied the benefits of these activities (most wireless routers are based on a version one of the open-source projects, very often OpenWRT) because modifying the software on a router to patch a security fix would also be illegal, and router vendors could be required to lock down firmware on the routers to prevent it being patched, updated or replaced in the first place.

This all echoes some of the same problems that are coming up in the USA, with the FCC's proposed rules to require firmware-lockdown on routers.

As we have seen with the confusion there, including TP-LINK being fined for locking down router firmware by the FCC, the matter is not a simple one, and certainly it is one that there still seems to be an understanding gap that we need to help regulators bridge, so that they can be enabled to enact regulations that protect these important rights, while addressing other legitimate social and political concerns.

Wednesday, August 31, 2016

Bridging Rhizome over HF

I'm currently up at Pacific Endeavour in Brisbane, which is a multi-national defence and civilian humanitarian and disaster response interoperability exercise, attended by, among others, the military of 22 nations around the Pacific basin.

We have been talking with Codan for a while about integrating Serval with their HF radios to allow Rhizome-based services over HF distances.  The goal is to allow MeshMS and our (still being written) Mesh-based social media applications to be carried over HF-distances (kilometres to hundreds of kilometres) between more localised concentrations of Mesh Extenders equiped with Wi-Fi and UHF packet radios.

This is of direct interest for our AusAID/DFAT grant to pilot Serval in the Pacific, where the beneficiary nations typically have small populations on islands separated by tens to hundreds of kilometres.   If we can link distant islands, then we can provide national and regional level communications, subject to the limitations of HF radio bandwidth (typically in the bytes to tens of bytes per second, although hundreds of bytes per second is possible in principle).

As part of the Pacific Endeavour, exercise the folks from Codan, Barrett and the other HF radio vendors are here.  The folks from Codan and Barrett in particular have been super helpful and enthusiastic as I have been explaining to them what we are trying to do, and how, to me at least, it seems to create the potential for new markets for HF radios.

They have kindly provided me with access to radios and documentation that they have with them, so that I can attempt a code sprint to establish two-way packet communications between their radios. That is, my goal is not just to prove that Serval Rhizome can run over HF radio, but that we can do it between what I understand are the two largest vendors in the humanitarian and disaster HF radio space.  Here is my setup with the Barrett radio (the black one on the desk) and the Codan radio (on the floor next to me):



After some false starts with having the correct firmware on one of the radios, I was pleasantly surprised to find that I was able to establish basic packet communications after only about ten hours of work.  Here you can see a packet being received, and a peer being detected by LBARD (the Serval Mesh program for handling slow radio links):




As the messages above show, LBARD knows what brand the radios are.  

This is partly out of necessity, because the serial control interface is quite different on the two radios, but also because we know if we are linking radios produced by the same vendor, that we have the potential to use higher-speed communications options.  

But between vendors, we currently have to use only inter-operable standards, which when it comes to HF radio means the older version of the Automatic Link Establishment protocol.  This is a bit unfortunate, because the interoperable ALE 2G standard is really slow.  The native data rate of ALE 2G is only 375 bits per second to begin with.  There is some protocol overhead, which reduces that somewhat.  Then we discovered that some radios don't support the full ASCII-64 restricted character set specified by the ALE 2G standard, and as their documentation claims. In particular, most of the punctuation characters in the ASCII-64 character set actually cause the message to be truncated on some of the radios.  This means that we can only use hex encoding, which means that we are consuming 2x 6-bit characters to encode each byte, i.e., 12 bits per byte. This reduced bandwidth by a further third.  Then there are a number of other losses relating to slow turn-around from TX to RX and back again, the need to fragment messages into 43 byte pieces to fit into the short ALE 2G AMD message limitations, all of which together reduce the effective bandwidth to perhaps 10 bytes per second.  Fortunately LBARD was designed to handle such slow links.

(We also found that some versions of firmware on some of the radios wouldn't receive ALE AMD messages sent by the opposite vendor, although we think that particular problem has been corrected in the latest version of firmware, so the problem is manageable.)

Naturally, we would much prefer to have higher speed connections, including the ALE 3G standard, that allows for up to 1KB/sec under ideal conditions, and without the need to fragment packets, which further saves on the TX/RX/TX turnaround cycle.  Unfortunately not all HF radios support ALE 3G.  That said, we will add support for radios that have it, and thus need to know if the remote radio also supports it -- thus another reason to know the vendor and model of the remote radio.

Higher-speed HF modems are also available, but are rather expensive.  Thus we are contemplating designing a low-cost HF radio using a cheap Spartan 3 or 6 FPGA for DSP, and attempt to implement something compatible with one of the existing HF modem standards (since they have done the hard work of knowing which modulation schemes are actually sensible on HF), that can be part of a Serval Mesh Extender HF adapter, to allow similar practical performance to the way that we are using the RFD900 UHF packet radios.    The RFD900 is actually much faster natively, but we are using it in an ad-hoc manner to allow many radios to be in range of each other, thus limiting each transmitter to ~0.25 - 1KB/sec.  For HF links, we are using 1:1 direct links, so that the best channel and modulation can be used, and thus we can use a simple TDM or equivalent scheme, that allows us to utilise a channel fully.

After getting everything working on ALE 2G, here is our Codan / Rhizome gateway (running from my laptop), and using this Codan "manpack" radio:


And the Barrett / Rhizome gateway, which I have yet to setup for the demo, will use this radio, a Barrett 2050:


So now I just need to update the version of OSX on the 2nd-hand Mac I just bought down the street, so that I can run this updated version of LBARD on it, and show it all running during the demo session this afternoon.  

But in the end, we have reached a significant milestone in that we have a proof-of-concept of near-real-time Serval communications over hundreds of kilometres, by using HF radios.

Sunday, July 24, 2016

Prototype replica bush-telegraph insulators

A little side project we have been looking at is using Serval to support tourism activities in regional and remote Australia.

The idea is that we want to create infrastructure-independent communications nodes at points of interest around Australia, that together we would call "The New Bush Telegraph" or "The New Overland Telegraph."  

Each station would consist of a replica bush telegraph pole, preferably manufactured in regional Australia (more on that in a moment), that has a variant of the Mesh Extender that can serve tourist information via Wi-Fi, and be itself updated via the Serval Mesh, so that the maintenance of the system is as simple as possible.

We had a student work on part of this earlier this year, working out what dimensions we would need for the replica insulator, so that it could fit the necessary electronics (Mesh Extender + batteries) inside, to keep them out of the weather.

We then contracted the Pilliga Pottery to make the prototype earthenware insulators.  The Pilliga Pottery are themselves a regional/remote Australian business, and are also the largest pottery in Australia -- so they were a natural choice for us to work with.  I had also seen some of their work during a family holiday last year, so I knew that they could do the kind of work that we were looking for.

Anyway, while I was away in Samoa and Arkaroola on work trips, the prototype insulators have arrived!

Here are our two nice boxes direct from the Pilliga Scrub (fortunately the dingrel didn't get them):


You can tell it has come from regional Australia, because it is tied with bailing twine:


And padded with egg cartons from Coonabarabran:


Finally we got down to where our new insulators were hiding:


We had told the pottery to choose whatever colours and painting they wished, so it was a moment of surprise to see what colours they had chosen:





Well, it looks like they have given us three quite different and interesting colours.  This is really nice, because we want each station to be distinctive, to add a further bit of interest to the stations.  Here they all are lined up on our bench:


So the next step is to get some poles, hopefully made by Arrium Steelworks in Whyalla, solar panels and electronics, so that we can make complete prototype units. This will be much easier once we have the new Mesh Extender prototypes, that should include solar regulators and LiFePO4 battery charger circuits (more on that in another blog post), and might need to wait until we have a student to continue to drive the project forward.

Wednesday, July 6, 2016

Mesh Extender Mark-II design progress

Again a rather short post, just to say that we are working with our contractors on the PCB and case design for the new generation of Mesh Extender prototypes that we will use in the Pacific next year, and likely in a variety of other places as well.

Together with everything else, this has me rather busy at the moment, so apologies for the lack of pictures and detailed information just yet.

Hopefully in a couple of weeks I will be able to share some of our design notes, to give you an idea of what we are hoping to build.

One big challenge is to make something that is both cheap, and sufficiently rugged to endure tropical maritime conditions in the pacific, and the boiling hot / freezing cold and very dusty conditions of the Australian Outback.

Another challenge is working around the short-comings of the USB mass storage capability of the Atheros 9311 SoC, for devices like the Mesh Extender, where power could be cut at any moment without notice.  This rather upsets some brands of USB memory sticks, potentially causing them to go read-only forever.

MeshMS over 4km via UHF

Another short post as I try to get on top of everything at the moment to say that on our last trip to Arkaroola we successfully tested the UHF packet radio functionality over a distance of about 4km.  This was from the observatory ridge to Coulthard's Lookout.

With the setup we were using, we had about 50% packet loss, and jet LBARD kept persevering, and delivering MeshMS messages within a minute or so.

We did see a recurrence of a bug affecting a corner case of LBARD operation, where sometimes you have to send a 2nd message to flush out a stuck first message.  However, this should not be too hard to fix.

This weekend I fly out to Samoa to meet some folks there to talk about our AusAID / Pacific Humanitarian Challenge grant to run a pilot using Mesh Extenders there, and it will be nice to show off the UHF functionality while there.

Wednesday, June 15, 2016

Rhizome over ad-hoc UHF is FINALLY working


 As many of you will already know, we have been working away in the background on getting the RFD900 radios working in something that approximates ad-hoc mode on Wi-Fi, but using the ISM 915 MHz band, so that we can have Mesh Extenders communicate over long distance, without any limit on the number of Mesh Extenders, and without them having to be in specially paired groups.

FINALLY at 4am this morning I fixed the remaining bugs that were stopping this from working in most cases.  There are still a few little niggly issues of course, but it is now working.

Together with some work-experience students, we have tested the units in and around the lab today.

First, we set one unit up on the bench in our lab on the fourth floor, together with an Android tablet running Serval Mesh that could receive MeshMS messages, and therefore cause automatic delivery acknowledgements to be pushed back to the message sender.


We then went for a wander with another Mesh Extender and a phone, and sent messages to the tablet, and waited for delivery confirmation to come through.  This typically took a couple of minutes, although in many cases the MeshMS text message itself is delivered in about 15 seconds.

Here we are one floor down from the lab:


And then two floors down.  Note that our building has 1/2 metre thick concrete floor decks.


And yet another floor down:

 And a bit closer in on the same floor:


And then down on the first floor -- with three thick floor decks between us and the Mesh Extender.


We also went outside on the ground floor under the metal main assembly building roof, and saw radio packets, and the units attempting to transfer messages, but no messages came through.  We think that the bundle synchronisation process needs a bit of help when faced with >50% packet loss.  This should be quite possible to achieve.

Then we went to a local supermarket to buy lunch, and took the Mesh Extenders with us.  One stayed in the car in the underground car park, and the other we took with us into the shopping centre.  Here is our Mesh Extender in its special shopping centre disguise vehicle:



This was quite nice, with one of our work experience students sending text messages down to the car park, and receiving delivery confirmations in the supermarket.

So, in other words, it lives!  Hopefully we will have more updates soon, as we start testing the resulting capabilities more thoroughly and in different contexts, particularly outdoors.

Wednesday, June 1, 2016

Mesh Extenders for the Pacific

Recently we were named a winner in the Pacific Humanitarian Challenge, receiving a grant from the Australian Department of Foreign Affairs to undertake a pilot of the Serval Mesh in one or more Pacific nations.

We're now starting to get underway in our planning for this, in particular, how we can make the Mesh Extender easily manufacturable in reasonable quantities (we need about 100 for the pilot), and as robust and reliable as possible, without making it too expensive.  Of course, we know that at these small quantities, the cost will still be much higher than we would like.

We have begun talking with our good friends at RFDesign about making an all-in-one Mesh Extender PCB, including both RFD900X + Atheros 9k based embedded Linux computer.  I think we have some good ideas about how we can go about that within the budget constraints of this project.  In many ways, this is the most important part, because it simply isn't realistic to hand-build 100+ Mesh Extenders the way that we have been prototyping them so far.

A picture of an injection moulding machine (Cjp24 CC-BY-SA 3.0), because this post was too boring with just text.

One of the really interesting ideas we are exploring is including a solar and battery controller in the unit, so that you can just plug in a solar panel, car battery or other supply to run the unit, in addition to the normal 5V USB supply.  You would then also be able to connect two LiFePO4 cells, that the unit would charge from the supply, and use to operate when there is no power supply available.  We have yet to work out what impact this will have on the cost, although at this stage, the RFD900 radio is by far the single most expensive component, so we are hopeful that we can make it fit.

I have also been introduced to the world of injection moulding through a helpful colleague here in the department.  Together with a local injection moulding company, we think we have found a way to get full-custom polycarbonate injection-moulded cases designed and manufactured at a relatively affordable price.  We are currently aiming for IP65 or IP66 rating for the case, so that it can be safely used in dusty outback conditions, as well as in tropical maritime climates.

However, injection moulding is not cheap, so "relatively affordable" is still code for "will stretch our budget to the very limit".  Our current estimate is that it will take at least AUD$30,000 for us to get the tools designed, built, and then squeeze out several hundred complete cases.  Given that it was previously looking like $50,000 to just get the tools made, and then only being able to do large production runs, this is a huge step forward. Also, with the injection moulding company being within easy riding distance, and being able to do very small production runs, and pick them up in beloved my cargo bike, this all spells good news for the Mesh Extender being able to be turned into a real, purchasable product, with some very nice characteristics and capabilities.

There are some other hurdles to jump through before we would be able to offer it as a complete product, including regulatory certification from the FCC and our Australian equivalent, and ideally, also EC approval from Europe.  Given the current shenanigans with the FCC, TP-LINK and locking down router firmware to prevent undesired 3rd-party access, we are thinking very carefully about how we can avoid that whole problem, but still have common hardware between the totally incompatible US/Australia/New Zealand/Canada versus Europe radio bands that we need to use for the UHF radio.

One approach that we are thinking about is having the power/data cable that feeds the Mesh Extender be wired differently for each regulatory domain.  That way, the firmware and everything can be completely common, and the radio firmware can simply probe the cable to work out which bands it should be operating on.  Given that owners of Mesh Extenders are likely in many cases to be internationally mobile, especially when trying to respond to disaster events, we think that this is a sensible approach.

We would also likely have another cable wiring that would tell the unit to accept unsigned firmware, so that people who wish to replace the firmware and experiment can do so, but there would be no risk of someone accidentally flashing the unit with firmware that would make it operate outside of authorised bands.

We understand that this whole area is a rather sensitive one for all of us in the open-source community, so we invite your feedback on whether what we are proposing in this regard, and whether there might be any better ways of doing it, without risking the FCC of EC folks refusing certification.

Also, I'll be picking the brains of some other folks who have created open-source consumer products to try to find out about and avoid any likely hazards along the way.  If you have any wisdom or experience to offer here, it would also be very welcome.

Paul.