Developing Challenges of Multi-Hop Mesh Telephony on Android

If you are reading this blog, you probably know that the Serval Project is all about making mesh telephony practical, so that people can communicate anywhere, any time.

For regular Android software development, it is often practical to use the Android emulator to reduce the time between updating source code and seeing the effect of the change.

For Serval, this isn't really an option, because we work below the Android API layer, and much of the software development challenge is related to making the WiFi chipsets do what we want.  We also have a fair amount of native code to do the mesh routing and telephony functions.  In fact, we currently have an entire distribution of Asterisk in the Serval Mesh software, which must be decompressed and installed when the Serval Mesh APK is installed or updated.

Together, these contribute to an update process that can take several minutes to between modifying a line of code and seeing it on an Android device, especially on the low-cost devices that are our primary targets, and when using the debug option in eclipse.

Those minutes add up over time, and so I decided to minimise the install time as much as possible, and also address another problem with developing our software, that is detecting when you have an actual multi-hop network.

To reduce the install time, I started looking at the Asterisk installation to see what we could safely throw out.  I managed to get rid of about half of the loadable modules, several utilities and all of the speech files that Asterisk comes bundled with.  This reduced the size of the APK by a little over 1MB, down to just under 5MB, and cut down the install time by about 15%.  Further savings came from removing unused audio codecs in our SIPDroid-derived SIP client, saving another 0.4MB, and getting the whole thing down to less than 4.5MB and installing about 25% faster than previously.

The bulk of the remaining time is about evenly split between installing Asterisk and testing that the WiFi control works properly.  For development, it probably makes sense to separate out the Asterisk installation from the other files we need to deploy on installation, as we really don't modify the Asterisk files between builds.  This shouldn't be too hard to do, but I have run out of time tonight.

Also, for our target handsets, we really don't need to test the WiFi control every time we update, so we should be able to set a flag to tell the software that we are in development mode, and just assume that the WiFi control works.  This now works by checking if a file called /sdcard/serval/developer-mode/fast-wifi exists, and if so, bypassing those tests, and thus saving about 15 seconds.

So over all the minimum time from hitting "run" in Eclipse to getting the software ready has been reduced from 85 seconds down to 48 seconds (without debugging).

The over significant development convenience task I have worked on in the last few days is one that is probably of more general interest.  The peer list now shows the number of hops to each of the nodes that your phone knows about, as shown in the image below.  This was produced by having two phones at opposite ends of my house, and having the phone shown here with me out on the street.

Each row of the peer list now shows the telephone number, e.g., 5550344, the batman link score, e.g., 243, the ping time to the peer, e.g., 19 milli-seconds, and either "direct" for nodes we can reach directly, or the number of hops to reach the node, e.g., 2 hop.

Importantly for debugging, this displays doesn't ask BATMAN how far to the node in question, but actually enquires of the time-to-live (TTL) field in the IP header, and thus returns the true distance from your phone to the phone in question (the return path might be a different length, though).

Actually, because of limitations with the Java UDP programming interface, the TTL field is read by the Serval DNA daemon (which is written in C), and included in the returned UDP packet to the requesting client.

This means that we can just look at the interface on the screen (and so can all of you who have been asking us for this feature) and see what is happening to routes, instead of having to connect to the phone with a USB cable and enter all sorts of commands to ask BATMAN and the operating system what might be happening.

So all in all we now have a slightly faster and easier development process, that I am sure will more than save the couple of person days of effort that they took to develop, and thus enable us to work faster instead of harder.

Demonstrating the Serval Rhizome Store-and-Forward MeshMS SMS Service

(Please take a look at our crowd-funding campaign at igg.me/at/speakfreely.)

Update: We trialled Rhizome on a training exercise in New Zealand recently.  Follow the link to read the blog post about it.

SMS messages on a cellular network get delivered via the cell towers and message centre infrastructure.

On a mesh, it is possible for us to deliver messages to other phones that are reachable on the mesh at the same time.  This works great, and we have had this capability in the Serval BatPhone software for a while now.

However, it does have some limitations, in particular, if there is no link all the way from the sender to the receiver on the mesh at the instant the message is sent.  This is rather unfortunate, as we usually think of SMS as being the most reliable fall-back on a mobile communications network.

So we set about replicating this resilience by creating a store-and-forward SMS-like service on top of the Serval Rhizome mesh file distribution framework.  We call this MeshMS, for Mesh Message Service.

This service uses direct mesh links if they are available by making use of the existing SMS capability in the Serval BatPhone software.  However, if there is no direct link, then it uses a store-and-forward scheme, that asks any passing phones to copy the message and distribute it to other phones on the mesh, until it (hopefully) eventually reaches its intended destination.  The cartoon below shows how this works.

The man wishes to send the message "Prepare the Zeppelin" to his hench-person, but there is no direct mesh link at that time due to the challenging topology.  The message gets picked up (stored) by the compatible phone in the pram, without any action on the part of the woman.  The woman then keeps walking along, and eventually the phone in the pram is able to automatically deliver (forward) the message to the ultimate destination, whereupon the minion knows to ready his master's zeppelin, to do the weekly grocery shopping, one trusts. Of course, there could be many intermediate steps, instead of just the one shown here.

The great thing about this approach is that it doesn't require a complete path to the destination at the time of sending, but can propagate progressively across the mesh as it is able, and can make use of nodes that move between otherwise isolated mesh networks, creating an asynchronous link between them where it would not otherwise be possible.

While the delay in such a service is huge, the bandwidth is also great, as potentially gigabytes of data can be transferred between nodes.  One side use of this protocol that we intend to exploit is to provide an efficient means of distributing updates to the Serval software suite, so that field updates can occur without dependence on infrastructure, and with much greater aggregate bandwidth than any single-cast cellular or wireless approach.  In fact, this will even allow the update of software, maps and other resources during a disaster.

We have even demonstrated it to deliver an SMS message between South Africa and Australia, using nothing but compatible phones to carry the message more than 10,000km:

This feature has profound utility in allowing the exchange of messages, files, and all manner of data in what are otherwise very difficult settings.

It also provides the capabilities required to enable disadvantaged communities, perhaps in remote locations, war zones or informal settlements, to create their own infrastructure, carrier and cost-free SMS networks using compatible handsets, so that they can enjoy the benefits of digital communications that many of us take for granted.

Serval is in finals for the Ashoka Foundation Change Makers Citizen Media Competition!

It seems that this last month has been one of the Serval Project gaining international recognition as leading the mesh communications space.  First some thoughts on that, and then some of the exciting things that have been happening.

This is incredibly gratifying and humbling, and I think also says something about the value of the open development and free software model, and also about the value of pragmatism and choosing the right applications and approach.  Ultimately it must spur us onto continuing to advance not only the technology, but also its use.

We are quite aware that there are other more experienced players in this space, who have in some cases invested many millions of dollars into mesh networking.  However, the closed-handed commercial approach that many such group have taken means that many innovations in the mesh networking space do not get well known, and that people feel isolated from what is happening in this space -- which is quite counterproductive, since mesh is by it's nature a word-of-mouth and peer-to-peer technology, and that character bleeds over into how it is best used, valued, evangelised and adopted.

We are also quite aware that there have been significant efforts put into making robust mesh routing protocols and standards, for example 802.11s, and yet in a short space of time we seem to be accelerating our traction at a much faster rate.  I think that there are a few interrelated causes for this.  First, we are extremely pragmatic in our approach, and this is partly enabled by us driving a vision forward, rather than seeking consensus to generate a moderated output.

The unfortunate reality for mesh networking is that it is still in such infancy that the various components have to be well matched to perform well, and thus the traditional compromise approach to standards generation, without an working example to inform the standard, is prone to produce a sub-optimal result, and take a very long time to produce.

This is also related to the lack of clear commercial application (or more specifically, of commercial return) of mesh networking in the eyes of many companies.  This is because when viewed as a traditional network, mesh networks are very poor cousins; they have poor reliability, poor bandwidth, poor quality of service, limited range and so on. All these make it very difficult to layer existing commercial models over a mesh network.  For example, adapting Google's advertising model onto a mesh network has trouble getting the advertisements over the mesh to each device, let alone the search data.

It takes a fresh approach to mesh networking to understand where the substantial commercial opportunities are hiding.  This requires gaining an understanding of the strengths of mesh networks, rather than their weaknesses.  The natural broadcast nature of wireless networks actually works well for broad dissemination of data.  The ability of these networks to operate without supporting infrastructure allows the provision of services when not ordinarily possible.

These services do not need to be data-poor services, either.  Serval has already created a prototype mapping application that could easily support navigation, geotagged advertising by businesses, crowd sourcing of points of interest and a variety of other features -- and would continue to be available when infrastructure is not available.

In this kind of setting, the limitations of mesh networking become strengths. The limited range of mesh networks means that locally relevant data is available where it is relevant, and doesn't flood the rest of the world or distant parts of the mesh.  The broadcast nature of wireless communications allows multiple receivers to receive these locally relevant data, whether it be advertisements, maps, communications between nearby friends or otherwise.  Also, by focussing on hop-by-hop dissemination of data, the traditional bandwidth degradation of mesh networks can be significantly avoided -- at the extreme it is single-hop communications which can be performed at the order of 100mbit between nearby WiFi enabled devices, which is way faster than my home ADSL internet connection.

But back to some of the recent successes we have had in our work and leadership being recognised, including the headline of this post:

The Serval Project is one of 11 finalists from a field of around 500 entries in the Ashoka Foundation's Change Maker Citizen Media competition.  This is tremendous recognition of the value of our work in the citizen empowerment space, and we are delighted to engage with this community, and with the other amazing finalists.  We'd also love you to vote for us if you think we should be one of the winners.  You can see the other finalists and vote using the links below:

We have also made it to the finals of the World Embedded Software Competition in Korea, for the second year running.

Meanwhile, we have begun to engage with the IEEE 802.11 standards process, with Romana in Atlanta at their meeting as I type.  It continues to amaze me the generally positive reception and voice that we are granted at events such as this, and we are hopeful that we can effect positive change in the standards to better support mesh networking.

We are also beginning to form partnerships with Universities, companies, not-for-profit and other entities, not just in Australia, but around the world.  More on some of those as they mature, but it is exciting for me to see our efforts multiplied as we gain partners.  As I have previously mentioned, we are keen to not only create resilient software, but produce it in a resilient manner, and that means with multiple teams in multiple countries around the world.  While this process has begun, we would love to expand this even further, so if you are interested, drop me a line.

Closer to home, but from my perspective very satisfying is that two of our students won their sections of the departmental end of term student expo.  Congratulations to both Swapna Palaniswamy and Corey Wallis for your great work. Swapna's work has also resulted in an upcoming publication at the International Conference on Wireless Technology for Humanitarian Relief later this year, which is an unusual achievement for a student in her course.  Corey has similarly had a presentation based on his work accepted  at Linux Conference Australia in January (we also have another talk at LCA on our Rhizome mesh file distribution protocol).

So it is a tremendously exciting time, and we are incredibly grateful to the Shuttleworth Foundation, Flinders University, NLnet and the Awesome Foundation for their enabling support of our activities.

So now it is back to driving the technology side of our activities so that we can deliver on the promise.

We have two primary areas of activity on this front at present.  First, we are working to consolidate the existing software to make it generally more usable and stable.  Second, we are working on incorporating the Rhizome file and messaging system so that we can provide what we think will be a very powerful mesh messaging service, that will even make possible the delivery of messages when there is no path immediately available through the mesh, and for the sender to receive confirmation when the message has been received.  But more on that when we have something concrete to show.

Oh the fun of the Android linker

So this week I decided that I would finally do the initial integration of the excellent NaCl crypto library into DNA, but with the view of making it available from Android so that it can be used as part of our forthcoming Rhizome browser (this is for browsing Rhizome offerings, not the web -- more on this thing in a future post).

My main objective was to have only one copy of the NaCl library in our Android application so that the APK stays as small as possible, and so that we don't waste any more memory than we need to.

The first challenge is that we want to access NaCl from Java, as well as the command-line dna utility.

The second challenge is that the crypto code should be inseperable from the dna code, so that it is impossible, for example, to use linker tricks to switch the NaCl library for something insecure.

We can't do this perfectly because it needs to be accessible from Java, which can only load native shared libraries, but it seems that we should do what we can.

So I put all of dna and NaCl into a single shared library, libdnalib.so, using the Android NDK to build it, so that we can load it into Java and use the NaCl (and possibly dna) functions directly from in Java.
This even means that dna's main() function is in there, and so we can run dna queries and actions from in Java as though we were using the command line tool.

Great.

Then I tried a bit of good old fashion linker skullduggery, by making an empty C file, and compiling it linked against libdnalib.so, so that it gets dna's main() from that library, by doing (in effect):

echo >null.c
gcc -o dna null.c -ldnalib

However, the linker on Android doesn't look in the host applications lib directory when linking an executable.  The Android linker also doesn't support the -Wl,-rpath= option for specifying where to find a library.

This is a real pain, and probably qualifies as a bug.  I should probably file the bug, but I wanted an immediate solution, since my past attempts at getting the Android folks to fix things have not met with much success.  Besides, I needed a solution that works with existing handsets, not future ones. So in the words of Marvin the Martian, it was back to the old drawing board.

So my solution was to create a wrapper that just dlopen()'s libdnalib.so, and then finds and runs the main() function from in there:


#include <dlfcn.h>
#include <stdio.h>


int main(int argc,char **argv)
{
 void *h = dlopen("/data/data/org.servalproject/lib/libdnalib.so",RTLD_LAZY);
 int (*dnamain)(int,char **) = dlsym(h,"main");
 if (!dnamain) return fprintf(stderr,"Could not load libdnalib.so\n");
 return (*dnamain)(argc,argv);


}

The downside is that it assumes that /data/data/org.servalproject is where the application will get installed, which the Android people tell us we should not assume.

Thus I will probably be about as popular with them as I am when I tell people to root their phones to get ad-hoc mode WiFi instead of waiting for an API to turn up in some future release of Android.

But, when necessity beckons, one must do what one must do.

And so I now have a solution that works and I can move onto more pressing problems.

Deploying a Phone Tower, Serval Style (The Day In Pictures)

So yesterday we went to beautiful Normanville with the goal of testing the maximum range between phones running our mesh telephony software.

We didn't get particularly far on that front, but we did discover some very interesting things about the unique requirements of mesh routing between mobile telephones, which will likely be the subject of a upcoming blog post.

But what we did do was deploy a mobile phone tower of a different kind.  Romana has wanted to make her wheel chair into a mobile phone tower for some time, and yesterday was her chance, as the pictures show.

Romana in her chair with Lyn beginning to attach the 7 metre long squid pole we use as the mast.

Further attaching of the mast.

Beginning to raise the mast.  The blob on the top is a Huawei IDEOS U8150 mobile phone running our software and acting as the "repeater".

 The raising of the mast:

Victory! Romana's wheel chair is now a mobile phone tower.

 But it wasn't all hard work.  The weather was glorious, and the views most excellent, and others were out and about enjoying the day ...

Or hoping for a chip.

Meanwhile, we moved to the jetty so that we could place our mobile phone tower (including blanket for the cool sea breeze).

The pole really does stand up quite high. It gets the phone about 6m above the deck of the jetty:

... which is a little higher than even the lamp post

Then we started walking along the beach towards Carrickallinga, slowly deploying a 6-hop mesh, and making multiple simultaneous voice calls amongst the nodes.

Corey and Romana did a great job standing on the jetty and talking to us via our impromptu phone network, while locals and visitors enjoyed the day:

This was also the first time that my son Caleb came along on a Serval expedition.  Little did he know that his pram was soon to become a phone tower as well.

Here I am with Lyn deploying a node some distance from the jetty:

Then it was Jeremy's turn:

Then the pram:

And all the while we were able to make calls among the various nodes.

We did encounter some serious mesh routing issues, which as I mentioned I will talk about soon, but for now I just wanted to let you all enjoy the pictures.

Kicking The Infrastructure Habit

Last weekend I delivered the inaugural Jim Bettison Memorial Oration at the biannual Adelaide Festival of Ideas. This was a great honour, and my presentation was well received by the audience, as was evidenced by the poignancy of their questions.

Here is the abstract of my oration:

Modern communications systems use extensive and expensive infrastructure to deliver services we could only dream of a few decades ago. This works for those who enjoy peace and sufficient wealth, but fails to reach the last billion people in poorer countries, as well as those in remote, emergency or disaster situations. Now modern mobile phones have the potential to communicate directly, to form networks without reliance on any infrastructure. The Serval Project based at Flinders University is turning this dream into a reality. It is working to make communications available to everyone, anywhere, any time especially to those who need it most.

You can listen to the oration by downloading the MP3 from the Radio Adelaide website.

You can also listen to an ABC Bush Telegraph interview about the oration that I recorded a few days prior to the oration.

More than we expected at the IEEE

If you have been reading this blog you will know that we sent Romana over to Okinawa to present our desire to see ad-hoc mode wifi improved, and also to consider how to use the baseband radio in mobile phones to support wifi on very low cost phones, and also to provide much longer range meshing using the ISM915/868 bands that offer very useful link budget advantages over the 2.4GHz band used by regular WiFi.

Well, we have been absolutely gobsmacked by the interest and positive response that Romana experienced at the meeting, and have the opportunity of taking up the baton to champion proposing these enhancements through the formal IEEE 802 standards process.

This is a great privilege, and also an opportunity that cannot be accepted without serious and solemn consideration, because it will basically require someone's effort, full time, for perhaps three years or more.

We are convinced that we want to, and that we should engage with this standards process, but just need to work out whether we have the resources to do it, without drawing all of our available resources away from progressing the development of the Serval mesh telephony software.

More news as we have it.

Why Serval is getting involved in the 802.11ah standards process

Serval is getting involved in the IEEE 802.11ah (WiFi on ISM bands below 1GHz) to try to make sure that it is well suited to infrastructure-independent ad-hoc and mesh networking at the IEEE meeting in Okinawa next week.

Here is our submission:

https://mentor.ieee.org/802.11/dcn/11/11-11-1138-00-00ah-packet-radio-mode-for-802-11ah-a-b-g-n.ppt

Basically we are asking for two things:

1. Improvements to ad-hoc mode (or provision of a new "packet radio" mode) that remove some of the problems currently faced when creating wifi-based ad-hoc mesh networks.

and

2. That the 802.11ah standard consider speeds below 1mbit and using cell phone baseband radios as a supported transport so that even cheap cell phones can form relatively long range mesh networks without any supporting hardware.

This second point is really important, because compared with 2.4GHz WiFi a mesh running in the ISM 915MHz band gains about +9db just from the change in frequency, which alone improves range by almost 3x.  If it supports lower bit rates, then further significant gains are possible, e.g., allowing 100kbit communications gives another +10db, for a total of 8x range versus WiFi.

There are some significant protocol challenges to be addressed, but if the standard doesn't support the use-case, then there will be no hardware and no chance.

These measures have the potential to push the indoor range up from WiFi's "about one house wide" to "about a block wide" and clear line-of-sight range up to a few km, which suddenly makes the formation of suburban ad-hoc mesh networks possible, which has profound impact for creating resilient infrastructure-independent communications solutions, for example for sustaining communications during disaster or enabling communications for rural and remote or developing populations.

We admit that we are very green to this process which we frankly find to be daunting.

But we feel compelled to try, regardless of what we perceive to be the odds of success.

Why User-Land Mesh Networks Are Necessary For Heterogeneous WiFi Mesh Networks

The normal approach to creating a mesh network is to write a program that does some tricky stuff to work out how to get from node A to node Z via whatever nodes might be in the middle, and then tell this to the operating system so that packets get routed according to these instructions.

We are doing something different for Serval, in that we are having the program that does all that tricky stuff carry the messages between nodes, instead of asking the operating system to help out.

This has some disadvantages and some advantages.

First, the disadvantages, and why we think that they aren't such a problem for our application.

1. Such "user-land" overlay networks suffer from an inefficiency known as "double-copy", where every packet sent or received ends up getting copied not once by the kernel, but twice, once by the kernel, and once by the overlay program.  This is bad if you are processing lots of data.  But on a mesh network the actual throughput of data tends to be fairly low (it is one of the well known problems with mesh networks).  So this problem is kept under control.

2. Programs that want to know about the mesh network need to know about the overlay and have to use special calls to talk to the mesh.  This is bad if your primary goal is allowing people to run BitTorrent, SSH, for example, because you need to modify each of those programs to support the new network, and without operating system cooperation that can get a little messier than you might like.

However, for a mesh telephony platform, and where we know that bandwidth starvation is one of the greatest potential problems, there is actually merit in requiring people to go to some effort before they enable their application to consume the limited resources of the mesh.  We will be adding the support to the initial core applications we intend to run over the mesh, so this isn't really a big deal.

Now for some advantages:

1. We don't need operating system cooperation.  This makes it easier to port the mesh to all operating systems, both desktop and mobile.  It is especially important because the need to fiddle with routing tables requires root/administrative access on most platforms.

For mobile platforms, this is certainly true on Android, and I believe iOS, with together constitute the majority of smart phones.  This alone means that we have to take this path.

But it also gives us some additional upside here, in that it means we could make a USB key loaded with Serval mesh software that could be run on any desktop computer, and without administrative access or being "installed" could enable that computer to participate in the mesh.  In fact, it could probably even be an HTML5 application running in a web browser.

2. It separates us from the underlying network addressing. Many devices and networks only support IPv4, but there aren't enough IPv4 addresses to go around. But by pushing our mesh network addressing into a dedicated addressing layer we can solve this problem.

We use 256bit Elliptic Curve cryptography keys as the basis for our address

In fact, using this approach it is possible for every single Serval enabled device on a mesh to all have the same IPv4 address, provided we use only broadcast traffic, which we do for a very important reason:

Broadcast traffic is the only traffic which WiFi ad-hoc can reliably carry between devices with incompatible or buggy ad-hoc implementations.  For example, we cannot get unicast traffic to flow reliably between Mesh Potatoes and Huawei IDEOS U8150/U8180 phones.  However, we can get unicast traffic to flow between Mesh Potatoes and the original Android G1 developer phones, and between the G1 and the U8150/U8180 phones.

Unfortunately mesh routing protocols all tend to use broadcast traffic to discover available routes, and thus the routes choose the impossible Mesh Potato-U8150/U8180 links, which is very bad indeed.  But by making the mesh traffic itself broadcast we ensure that what the mesh routing protocol and what we are trying to do is the same, instead of merely similar.

Taking this a little further, by transporting the mesh topology management information in the same packets as the data we can directly measure the performance of the mesh delivering the traffic we really care about, rather than using different packets as an indirect measure -- especially since the distribution of packet sizes can otherwise be very different, and WiFi packet losses are not insensitive to packet length or differentiation between broadcast and unicast traffic.

Implementing this is of course a significant undertaking, and I have already spent a few weeks programming away.  But the effort is beginning to yield results.  Today for the first time I have the new overlay mesh detecting nodes and building the in-memory route table. You can see a screen grab here:

So here you can see some debug output from a Serval DNA node running in overlay mode using a file called dummy0 as a dummy network interface.

I realised early in the development of the overlay network that I would need some convenient simulation tools to make it easier to debug the behaviour of the mesh.  I decided the best way to do this was to make the software itself also a network simulator.  So I added support to pretend an ordinary file is a network interface.  Any instances of the overlay pointed to the same file would all append their "packets" to the end of the file, and remember where they had got up to reading from the file, and keep an eye out for any packets deposited by other nodes.

Each packet also has a header that has space for location information, transmission power, time, node velocity and heading and a whole pile of other information that can be used to drop packets that are deemed to have been sent from too far away, or suffering collision during reception and so on.

The file then also represent a log of all activity on the mesh which has already proven very helpful for debugging.  I will also create a tool that allows for the creation of visualisations of the mesh in action to help understand why things go wrong, and to explain the operation of the mesh.  I will post such animations here once I get that far.

Back to the trace, the error messages are mostly information for me for development (although there is on link-list bug that I can see I will need to fix to avoid a memory leak).

There are a few 256bit addresses that can be seen.  The all-f's is the mesh broadcast address, while the one that starts with 9d5f is the address of a node on the mesh.

These addresses are really big, and so I have already implemented a variety of mechanisms for abbreviating the addresses.

For example, the broadcast address is used so often, that it has a one-byte abbreviation (0x0f).  For other addresses, we only need to transmit as much of the address is required for the receiver to discriminate it from all other nodes.  The birthday paradox comes into play here, and means that we typically need about twice the number of bits as the base-2 log of the number of nodes on the network.  For a small network of a few hundred nodes 24 bits (3 bytes) of address is a pleasant over-kill.  Even at a global scale, we need to accomodate around 10 billion addresses = ~2^34 addresses, and so about 70 bits (7 bytes) should be enough.

We also have abbreviation strategies where neighbouring nodes that talk with each other a lot can allocate a one-byte short code for each other, however, as you might be able to guess from the trace that abbreviation method is temporarily disabled in the code to keep my life a little simpler at present -- which is a good move because of the density of new features in the code.

At the end of the trace we see a summary of the overlay routing table which shows 7-digit address prefixes for all know nodes on this network, and the link-scores to each via the various routes known to each.

The scores have two components. The first being a BATMAN-like score derived from beacon counting, although we take into account that the remote node might have different interfaces running at different beacon rates.  The second part is the number of Serval gateways that are on the path.

This gateway count is there to prevent routing loops and other nasties in meshes where the overlay has managed to infer some hierarchy, or where inter-mesh links have been purposely provisioned.

I'll talk about inferred hierarchy in another post, but it is one of several measures we are taking to ensure that a Serval mesh can scale beyond the normal bandwidth-starvation limit imposed by all nodes trying to know about all other nodes all of the time.

Our premise is that you need frequent information for nearby nodes because their "bearing" can change wildly over short time frames, but that for more distant nodes their "bearing" on the mesh should change much more slowly, and so we should be able to cope with less frequent updates about the position of such distant nodes.

But meanwhile, as you can see we are making good progress, and the Serval DNA overlay mesh is beginning to come together.

Day 1 of Shuttleworth Foundation Fellowship

So today I have commenced a telecommunications fellowship with the Shuttleworth Foundation, which will be undertaken here at Flinders University where with the University's generous support we have based the Serval Project.

This is an amazing support for our work on the Serval Project, as not only does it free up my salary so that we can employ a project manager, but it also provides access to a very helpful operational budget, and a great team of dedicated open-technologists working on a variety of projects.  Check out their list of past and current fellows to get a feel for what they are supporting.

This support builds on that already provided by The Awesome Foundation, Flinders University and NLnet, as well as that of many students, volunteers and my family, which has enabled us to reach this point.

The particular mandate of my work under the fellowship is to:

0. 
   

- Build on existing trials, which include remote, in-motion, underground and
indoor tests by linking with partner organisations to deploy the
technology in an escalating series of situations;– Increasing the breadth of applications for the technology, and the usability
of the technology in disaster, by governments, and in remote locations;
and– Improving the maximum range between handsets beyond what is
possible using WiFi, but without altering the hardware in existing mobile telephones (the “Focus Area Objectives”). 


What this means is that we will be pursuing a simultaneous software improvement and testing program that will, hopefully, get us to the point of a quality public release within a year.


The greatest challenge comes from the range extension, which we know is possible using the baseband radio in mobile phones. However, it is extremely difficult to get the programming information to enable modification of the firmware loaded into the radio's baseband processor. 


We will be exploring multiple angles to achieve this, ranging from engaging with the IEEE 802.11 standardisation process, to seeking to open channels with cell phone chipset manufacturers, through to, building custom cell phones that contain friendlier baseband or other radio chipsets that we can more easily reprogram, and where legal and appropriate, reverse-engineering the information we require.


The potential gains over using WiFi to build the mesh (perhaps 300m between phones indoors and 10km or more between phones outdoors in open country) are so great that we must engage with this process, regardless of the difficulty.