Romain one of our team has been working hard to get the Rhizome Retriever together as part of our work in supporting citizen journalism in a community in Nigeria, that I have blogged about previously.
They are already using our software in that community to make free mesh telephone calls, but from the outset the plan has been to enable them to be citizen journalists, and to share news and information in their community.
To enable this, we are making our Rhizome Retriever (Serval RR), which basically lets you pick files on your phone, and put them up on the mesh, from where they will be automatically distributed to other phones on the mesh.
Mass file distribution on a mesh is usually fraught with difficulties, not the least of which is that the available bandwidth on a mesh halves (or worse) for each extra hop that you try to traverse.
So you can often get decent bandwidth to nearby nodes, but poor or intermittent connection to more distant nodes.
Also, the more people trying to transfer things many hops across the network reduces the available bandwidth for everyone.
So we decided to make the Serval RR only ever send data one hop to the nearest neighbours. They can then in turn send data to their neighbours later on at their convenience.
As a result, Serval RR can already distribute even relatively large files, certainly into the several mega-byte range, without great difficulty, and without making the network unusable for everyone else.
Here is a quick demo of the initial prototype of the Serval RR software in action:
Tuesday, October 25, 2011
Sunday, October 23, 2011
The importance of communications during emergencies
Early 2011 saw the worst floods in Queensland, Australia in 40 years.
Not surprisingly, this placed significant stresses on response efforts, and for a variety of reasons, the Queensland Government has issued an interim report on the event and what responses should be made.
You can read the interim report here.
I have been reading through this from a telecommunications perspective, and found the following items:
2.24 Seqwater should give consideration to posting information about current and future releases on its website during flood events as one method of ensuring accurate and timely information is available to the public.
This may seem like a small matter, but by making the information available somewhere where someone in the community could access it, and possibly disseminate it further, perhaps by word of mouth, or perhaps by feeding it into a crowd-source incident reporting system such as Ushahidi or the Serval Rhizome/Mapping combination, where it could be made available to others who need it. Indeed, this process could be automated, providing a very valuable community service. However, if the information is not shared in the first instance, then such broad and means of distribution will be troublesome to implement, and potentially disadvantage many.
In short, putting information out where the public can get to it, and make further use of it supports resilience.
5.21 The Queensland Fire and Rescue Service should ensure that rescue technicians on deployment are provided with individual radios, rather than sharing a communications pack
Presumably the sharing of communications packs is related to their cost and bulk.
Using off-the-shelf meshing mobile phones would allow all rescue technicians to have a personal means of communications that makes use of the shared communications pack, relaying traffic back through that resource.
5.29 The Queensland Fire and Rescue Service should consider isolating repeaters during a large scale emergency response. If this solution is found to be feasible, it should be implemented as protocol as soon as possible. If it is not, the Queensland Fire and Rescue Service should explore other solutions to the issue of the fire communications network being overloaded and firefighters resorting to localised networks during large scale emergency response situation.
This seems to be responding to the issue of excessive traffic being relayed across the region, congesting the system and preventing it being of effective use. The Serval Mesh system has the potential to localise message delivery to the local teams that need it (whether or not they are locally based), and still retaining the ability to broadcast messages over the whole region when suitably addressed.
This is not a particular mode we had considered greatly in the past, so this is a helpful thing for us to discover.
7.2 Lockyer Valley Regional Council should investigate the feasibility of installing alarm-activating gauges in the creeks at Spring Bluff, Murphys Creek and other communities where communication systems are poor and there is a risk of rapid and unexpected water rise.
This is interesting for us to read, as we are already contemplating creating a very similar system for use in the Mekong River Basin on conjunction with a Lao doctoral student here in the department. The characteristics of the system are having a local mesh broadcast an alarm to all mesh nodes when one or more mesh-equipped sensors indicates a high water level.
It is possible that those sensors could be low-cost mobile telephones connected to a device such as the IOIO, creating a very simple and low-cost solution, provided the device can be powered, perhaps using a solar panel and modest storage battery, such as an old car battery no longer able to supply the peak load required to start a car, but more than sufficient for the couple of watts required to run a phone and sensor.
"SMS alerts are also not a reliable method of providing flood warnings in parts of Queensland which experience problems with telephone coverage. That difficulty is compounded during a flood, when telephone reception can be affected by flood related power outages and congested telecommunications networks."
This is precisely the kind of short coming with cellular networks that Serval has sought to find and create solutions for, that complement the existing infrastructure. In particular, the ability of the Serval Rhizome mesh file/message distribution system to work asynchronously, and automatically share information carried in phones as they more from region to region has particular value in these sorts of situations.
Also, the ability place local calls on the mesh, and where possible, setup connections from the mesh to the outside world offers a significant capability that is otherwise a considerable liability in these situations.
The ability for Serval technology to allow the rapid setup of a local mobile telephone network, and then connect it to the global telecommunications network using the nearest functional mobile phone tower was demonstrated in Brisbane in late January 2011, just a couple of weeks after the flooding had subsided.
We did this by placing a phone running the Serval software at a high vantage point where it could communicate with the nearest functional mobile phone tower (we used a helium balloon, but a roof top, or even a long fishing pole gaffer taped to a car would suffice). This then allowed the phones on the ground that were also running our software to relay calls out to the world. Here is the video we took of us doing exactly this, and watching people use it:
Not surprisingly, this placed significant stresses on response efforts, and for a variety of reasons, the Queensland Government has issued an interim report on the event and what responses should be made.
You can read the interim report here.
I have been reading through this from a telecommunications perspective, and found the following items:
2.24 Seqwater should give consideration to posting information about current and future releases on its website during flood events as one method of ensuring accurate and timely information is available to the public.
This may seem like a small matter, but by making the information available somewhere where someone in the community could access it, and possibly disseminate it further, perhaps by word of mouth, or perhaps by feeding it into a crowd-source incident reporting system such as Ushahidi or the Serval Rhizome/Mapping combination, where it could be made available to others who need it. Indeed, this process could be automated, providing a very valuable community service. However, if the information is not shared in the first instance, then such broad and means of distribution will be troublesome to implement, and potentially disadvantage many.
In short, putting information out where the public can get to it, and make further use of it supports resilience.
5.21 The Queensland Fire and Rescue Service should ensure that rescue technicians on deployment are provided with individual radios, rather than sharing a communications pack
Presumably the sharing of communications packs is related to their cost and bulk.
Using off-the-shelf meshing mobile phones would allow all rescue technicians to have a personal means of communications that makes use of the shared communications pack, relaying traffic back through that resource.
5.29 The Queensland Fire and Rescue Service should consider isolating repeaters during a large scale emergency response. If this solution is found to be feasible, it should be implemented as protocol as soon as possible. If it is not, the Queensland Fire and Rescue Service should explore other solutions to the issue of the fire communications network being overloaded and firefighters resorting to localised networks during large scale emergency response situation.
This seems to be responding to the issue of excessive traffic being relayed across the region, congesting the system and preventing it being of effective use. The Serval Mesh system has the potential to localise message delivery to the local teams that need it (whether or not they are locally based), and still retaining the ability to broadcast messages over the whole region when suitably addressed.
This is not a particular mode we had considered greatly in the past, so this is a helpful thing for us to discover.
7.2 Lockyer Valley Regional Council should investigate the feasibility of installing alarm-activating gauges in the creeks at Spring Bluff, Murphys Creek and other communities where communication systems are poor and there is a risk of rapid and unexpected water rise.
This is interesting for us to read, as we are already contemplating creating a very similar system for use in the Mekong River Basin on conjunction with a Lao doctoral student here in the department. The characteristics of the system are having a local mesh broadcast an alarm to all mesh nodes when one or more mesh-equipped sensors indicates a high water level.
It is possible that those sensors could be low-cost mobile telephones connected to a device such as the IOIO, creating a very simple and low-cost solution, provided the device can be powered, perhaps using a solar panel and modest storage battery, such as an old car battery no longer able to supply the peak load required to start a car, but more than sufficient for the couple of watts required to run a phone and sensor.
"SMS alerts are also not a reliable method of providing flood warnings in parts of Queensland which experience problems with telephone coverage. That difficulty is compounded during a flood, when telephone reception can be affected by flood related power outages and congested telecommunications networks."
This is precisely the kind of short coming with cellular networks that Serval has sought to find and create solutions for, that complement the existing infrastructure. In particular, the ability of the Serval Rhizome mesh file/message distribution system to work asynchronously, and automatically share information carried in phones as they more from region to region has particular value in these sorts of situations.
Also, the ability place local calls on the mesh, and where possible, setup connections from the mesh to the outside world offers a significant capability that is otherwise a considerable liability in these situations.
The ability for Serval technology to allow the rapid setup of a local mobile telephone network, and then connect it to the global telecommunications network using the nearest functional mobile phone tower was demonstrated in Brisbane in late January 2011, just a couple of weeks after the flooding had subsided.
We did this by placing a phone running the Serval software at a high vantage point where it could communicate with the nearest functional mobile phone tower (we used a helium balloon, but a roof top, or even a long fishing pole gaffer taped to a car would suffice). This then allowed the phones on the ground that were also running our software to relay calls out to the world. Here is the video we took of us doing exactly this, and watching people use it:
Saturday, October 22, 2011
Serval is going to Atlanta
Well, Romana (co-founder of Serval Project) is. And that is because she will be representing the Serval Project's interests at the IEEE 802 plenary meeting there in just a couple of weeks (November 2011).
Romana will be putting forward use cases that reveal deficiencies in the current 802.11 family of WiFi standards for mesh and ad-hoc communications.
If all goes well, we may have the opportunity to input into a process of looking to address these issues, which is tremendously exciting for us.
It could mean much longer range mesh hops using ordinary mobile phones, by using the cellular radio to implement some kind of WiFi in the ISM 915MHz band.
It could also spell the end of frustrating incompatibilities in the 802.11 ad-hoc WiFi standard, including the bizarre lack of specification of ad-hoc in 802.11n, preventing standards conforming devices from using the higher speeds and other benefits that 802.11n brings.
Either or both of these have the potential to make citizen generated infrastructure-free communications systems mainstream, thus our excitement.
So Atlanta will be the official first turning of clod in a process that could extend up to four years, and represents a huge personal investment on the part of Romana, as well as a substantial financial investment on the part of the Serval Project, and thus the Shuttleworth Foundation whose support makes this possible.
We would invite any other open efforts that have concerns with aspects or omissions of the IEEE 802.11 WiFi standards to get in touch with us, so that we can do our best to represent the needs of more than just ourselves.
Romana will be putting forward use cases that reveal deficiencies in the current 802.11 family of WiFi standards for mesh and ad-hoc communications.
If all goes well, we may have the opportunity to input into a process of looking to address these issues, which is tremendously exciting for us.
It could mean much longer range mesh hops using ordinary mobile phones, by using the cellular radio to implement some kind of WiFi in the ISM 915MHz band.
It could also spell the end of frustrating incompatibilities in the 802.11 ad-hoc WiFi standard, including the bizarre lack of specification of ad-hoc in 802.11n, preventing standards conforming devices from using the higher speeds and other benefits that 802.11n brings.
Either or both of these have the potential to make citizen generated infrastructure-free communications systems mainstream, thus our excitement.
So Atlanta will be the official first turning of clod in a process that could extend up to four years, and represents a huge personal investment on the part of Romana, as well as a substantial financial investment on the part of the Serval Project, and thus the Shuttleworth Foundation whose support makes this possible.
We would invite any other open efforts that have concerns with aspects or omissions of the IEEE 802.11 WiFi standards to get in touch with us, so that we can do our best to represent the needs of more than just ourselves.
Tuesday, October 18, 2011
Oh the fun of the Android linker
So this week I decided that I would finally do the initial integration of the excellent NaCl crypto library into DNA, but with the view of making it available from Android so that it can be used as part of our forthcoming Rhizome browser (this is for browsing Rhizome offerings, not the web -- more on this thing in a future post).
My main objective was to have only one copy of the NaCl library in our Android application so that the APK stays as small as possible, and so that we don't waste any more memory than we need to.
The first challenge is that we want to access NaCl from Java, as well as the command-line dna utility.
The second challenge is that the crypto code should be inseperable from the dna code, so that it is impossible, for example, to use linker tricks to switch the NaCl library for something insecure.
We can't do this perfectly because it needs to be accessible from Java, which can only load native shared libraries, but it seems that we should do what we can.
So I put all of dna and NaCl into a single shared library, libdnalib.so, using the Android NDK to build it, so that we can load it into Java and use the NaCl (and possibly dna) functions directly from in Java.
This even means that dna's main() function is in there, and so we can run dna queries and actions from in Java as though we were using the command line tool.
Great.
Then I tried a bit of good old fashion linker skullduggery, by making an empty C file, and compiling it linked against libdnalib.so, so that it gets dna's main() from that library, by doing (in effect):
echo >null.c
gcc -o dna null.c -ldnalib
However, the linker on Android doesn't look in the host applications lib directory when linking an executable. The Android linker also doesn't support the -Wl,-rpath= option for specifying where to find a library.
This is a real pain, and probably qualifies as a bug. I should probably file the bug, but I wanted an immediate solution, since my past attempts at getting the Android folks to fix things have not met with much success. Besides, I needed a solution that works with existing handsets, not future ones. So in the words of Marvin the Martian, it was back to the old drawing board.
So my solution was to create a wrapper that just dlopen()'s libdnalib.so, and then finds and runs the main() function from in there:
#include <dlfcn.h>
#include <stdio.h>
int main(int argc,char **argv)
{
void *h = dlopen("/data/data/org.servalproject/lib/libdnalib.so",RTLD_LAZY);
int (*dnamain)(int,char **) = dlsym(h,"main");
if (!dnamain) return fprintf(stderr,"Could not load libdnalib.so\n");
return (*dnamain)(argc,argv);
}
The downside is that it assumes that /data/data/org.servalproject is where the application will get installed, which the Android people tell us we should not assume.
Thus I will probably be about as popular with them as I am when I tell people to root their phones to get ad-hoc mode WiFi instead of waiting for an API to turn up in some future release of Android.
But, when necessity beckons, one must do what one must do.
And so I now have a solution that works and I can move onto more pressing problems.
My main objective was to have only one copy of the NaCl library in our Android application so that the APK stays as small as possible, and so that we don't waste any more memory than we need to.
The first challenge is that we want to access NaCl from Java, as well as the command-line dna utility.
The second challenge is that the crypto code should be inseperable from the dna code, so that it is impossible, for example, to use linker tricks to switch the NaCl library for something insecure.
We can't do this perfectly because it needs to be accessible from Java, which can only load native shared libraries, but it seems that we should do what we can.
So I put all of dna and NaCl into a single shared library, libdnalib.so, using the Android NDK to build it, so that we can load it into Java and use the NaCl (and possibly dna) functions directly from in Java.
This even means that dna's main() function is in there, and so we can run dna queries and actions from in Java as though we were using the command line tool.
Great.
Then I tried a bit of good old fashion linker skullduggery, by making an empty C file, and compiling it linked against libdnalib.so, so that it gets dna's main() from that library, by doing (in effect):
echo >null.c
gcc -o dna null.c -ldnalib
However, the linker on Android doesn't look in the host applications lib directory when linking an executable. The Android linker also doesn't support the -Wl,-rpath= option for specifying where to find a library.
This is a real pain, and probably qualifies as a bug. I should probably file the bug, but I wanted an immediate solution, since my past attempts at getting the Android folks to fix things have not met with much success. Besides, I needed a solution that works with existing handsets, not future ones. So in the words of Marvin the Martian, it was back to the old drawing board.
So my solution was to create a wrapper that just dlopen()'s libdnalib.so, and then finds and runs the main() function from in there:
#include <dlfcn.h>
#include <stdio.h>
int main(int argc,char **argv)
{
void *h = dlopen("/data/data/org.servalproject/lib/libdnalib.so",RTLD_LAZY);
int (*dnamain)(int,char **) = dlsym(h,"main");
if (!dnamain) return fprintf(stderr,"Could not load libdnalib.so\n");
return (*dnamain)(argc,argv);
}
The downside is that it assumes that /data/data/org.servalproject is where the application will get installed, which the Android people tell us we should not assume.
Thus I will probably be about as popular with them as I am when I tell people to root their phones to get ad-hoc mode WiFi instead of waiting for an API to turn up in some future release of Android.
But, when necessity beckons, one must do what one must do.
And so I now have a solution that works and I can move onto more pressing problems.
Thursday, October 13, 2011
Deploying a Phone Tower, Serval Style (The Day In Pictures)
So yesterday we went to beautiful Normanville with the goal of testing the maximum range between phones running our mesh telephony software.
We didn't get particularly far on that front, but we did discover some very interesting things about the unique requirements of mesh routing between mobile telephones, which will likely be the subject of a upcoming blog post.
But what we did do was deploy a mobile phone tower of a different kind. Romana has wanted to make her wheel chair into a mobile phone tower for some time, and yesterday was her chance, as the pictures show.
The raising of the mast:
But it wasn't all hard work. The weather was glorious, and the views most excellent, and others were out and about enjoying the day ...
Or hoping for a chip.
Meanwhile, we moved to the jetty so that we could place our mobile phone tower (including blanket for the cool sea breeze).
The pole really does stand up quite high. It gets the phone about 6m above the deck of the jetty:
... which is a little higher than even the lamp post
Then we started walking along the beach towards Carrickallinga, slowly deploying a 6-hop mesh, and making multiple simultaneous voice calls amongst the nodes.
Corey and Romana did a great job standing on the jetty and talking to us via our impromptu phone network, while locals and visitors enjoyed the day:
This was also the first time that my son Caleb came along on a Serval expedition. Little did he know that his pram was soon to become a phone tower as well.
Here I am with Lyn deploying a node some distance from the jetty:
Then it was Jeremy's turn:
Then the pram:
And all the while we were able to make calls among the various nodes.
We did encounter some serious mesh routing issues, which as I mentioned I will talk about soon, but for now I just wanted to let you all enjoy the pictures.
We didn't get particularly far on that front, but we did discover some very interesting things about the unique requirements of mesh routing between mobile telephones, which will likely be the subject of a upcoming blog post.
But what we did do was deploy a mobile phone tower of a different kind. Romana has wanted to make her wheel chair into a mobile phone tower for some time, and yesterday was her chance, as the pictures show.
 |
| Romana in her chair with Lyn beginning to attach the 7 metre long squid pole we use as the mast. |
 |
| Further attaching of the mast. |
 |
| Beginning to raise the mast. The blob on the top is a Huawei IDEOS U8150 mobile phone running our software and acting as the "repeater". |
 |
| Victory! Romana's wheel chair is now a mobile phone tower. |
Or hoping for a chip.
Meanwhile, we moved to the jetty so that we could place our mobile phone tower (including blanket for the cool sea breeze).
The pole really does stand up quite high. It gets the phone about 6m above the deck of the jetty:
... which is a little higher than even the lamp post
Then we started walking along the beach towards Carrickallinga, slowly deploying a 6-hop mesh, and making multiple simultaneous voice calls amongst the nodes.
Corey and Romana did a great job standing on the jetty and talking to us via our impromptu phone network, while locals and visitors enjoyed the day:
This was also the first time that my son Caleb came along on a Serval expedition. Little did he know that his pram was soon to become a phone tower as well.
Here I am with Lyn deploying a node some distance from the jetty:
Then it was Jeremy's turn:
Then the pram:
And all the while we were able to make calls among the various nodes.
We did encounter some serious mesh routing issues, which as I mentioned I will talk about soon, but for now I just wanted to let you all enjoy the pictures.
Wednesday, October 12, 2011
Kicking The Infrastructure Habit
Last weekend I delivered the inaugural Jim Bettison Memorial Oration at the biannual Adelaide Festival of Ideas. This was a great honour, and my presentation was well received by the audience, as was evidenced by the poignancy of their questions.
Here is the abstract of my oration:
Here is the abstract of my oration:
Modern communications systems use extensive and expensive infrastructure to deliver services we could only dream of a few decades ago. This works for those who enjoy peace and sufficient wealth, but fails to reach the last billion people in poorer countries, as well as those in remote, emergency or disaster situations. Now modern mobile phones have the potential to communicate directly, to form networks without reliance on any infrastructure. The Serval Project based at Flinders University is turning this dream into a reality. It is working to make communications available to everyone, anywhere, any time especially to those who need it most.
You can listen to the oration by downloading the MP3 from the Radio Adelaide website.
You can also listen to an ABC Bush Telegraph interview about the oration that I recorded a few days prior to the oration.
Wednesday, October 5, 2011
Photos and Update from Nigeria - and introduction to Rhizome
We have a partner organisation working in Nigeria looking at citizen-sourced journalism and using our mesh network as the connectivity among a number of camera-equipped cell phones, so that citizen journalists there can make a short video or news piece and then have it distribute automatically over the mesh using our Rhizome protocol.
Rhizome is our mesh file distribution protocol that allows user-created content and software updates to spread hop-by-hop over the mesh. This hop-by-hop approach elegantly solves many of the problems with mesh-wide file transfer, in particular bandwidth starvation and instability of long links.
We have called it rhizome because it is like the vegetable reproductive process of the same name, because we think it is a good analogy for the way that Rhizome allows data to spread by small hops, but such that over time it can spread over great distances.
We are busy working on making Rhizome do all that the team in Nigeria needs, but in the meantime, the team in Nigeria have sent us some photos showing the mesh in action.
First, they sent photographic evidence of 10 IDEOS U8150 phones forming a mesh with our software:
They also sent us this picture which I really love, because it shows a community getting started with out technology thousands of kilometres from where we are writing the software here in Australia. This is what it is all about, people using and benefiting from the software that we are creating and releasing.
One of the challenges we are looking at solving with this group is auto-update of the mesh software, over the mesh. That is, we want to be able to load a signed software update onto any one of the phones on the mesh, and have the phones on the mesh share it automatically with one another.
This is a great idea, and the Rhizome protocol will allow us to do this, however it introduces some security challenges.
In particular, we don't want someone or some party seizing or cracking our signing key, and then being able to spread a trojan update that destroys the Serval mesh.
One approach we are looking at is making the update require that the new version be signed by a majority of keys from a pool, where each key would be held by a different mesh-friendly organisation. The pool of trusted keys could then be similarly updated by a majority vote of existing trusted keys.
This will allow us to spread the keys among countries and jurisdictions and provide the kind of distributed resilience that the Serval Project is predicated on, and even allows the software to continue to be refined and distributed if the Serval Project as an organisation ceases to exist.
If you think that your organisation might like to be one of our key custodians, let me know.
Rhizome is our mesh file distribution protocol that allows user-created content and software updates to spread hop-by-hop over the mesh. This hop-by-hop approach elegantly solves many of the problems with mesh-wide file transfer, in particular bandwidth starvation and instability of long links.
We have called it rhizome because it is like the vegetable reproductive process of the same name, because we think it is a good analogy for the way that Rhizome allows data to spread by small hops, but such that over time it can spread over great distances.
We are busy working on making Rhizome do all that the team in Nigeria needs, but in the meantime, the team in Nigeria have sent us some photos showing the mesh in action.
First, they sent photographic evidence of 10 IDEOS U8150 phones forming a mesh with our software:
 |
| 10 Serval BatPhones on the mesh (and a call in progress!) |
Finally, just a week after they got 10 phones on the mesh, they sent this shot showing 15 phones on the mesh. This was just a case of them getting 15 phones within range, as the underlying BATMAN mesh protocol should be able to handle a couple of hundred devices without any difficulty:
One of the challenges we are looking at solving with this group is auto-update of the mesh software, over the mesh. That is, we want to be able to load a signed software update onto any one of the phones on the mesh, and have the phones on the mesh share it automatically with one another.
This is a great idea, and the Rhizome protocol will allow us to do this, however it introduces some security challenges.
In particular, we don't want someone or some party seizing or cracking our signing key, and then being able to spread a trojan update that destroys the Serval mesh.
One approach we are looking at is making the update require that the new version be signed by a majority of keys from a pool, where each key would be held by a different mesh-friendly organisation. The pool of trusted keys could then be similarly updated by a majority vote of existing trusted keys.
This will allow us to spread the keys among countries and jurisdictions and provide the kind of distributed resilience that the Serval Project is predicated on, and even allows the software to continue to be refined and distributed if the Serval Project as an organisation ceases to exist.
If you think that your organisation might like to be one of our key custodians, let me know.
Subscribe to:
Posts (Atom)