Thursday, October 25, 2012

Open Writing of Serval Documentation

As the Serval Project is based in a University, we have a need to write academic papers.  However, such papers take a while to write, get peer reviewed and published. This is a problem since Serval is also an open-source project, and we need to get information out to the community in a timely manner.

So I am trying an experiment.  I am progressively moving all paper preparation activity into our github repository (http://github.com/servalproject/serval-docs/).

Some legacy documents will be in LibreOffice /OpenOffice format.

For new documents I will be aiming to use LyX (a LaTeX frontend), because commits that change documents will result in meaningful diffs between versions.  Also figures exist as separate files, and so are easier for people to access, update and/or reuse.

One of the nice advantages is that people can submit corrections, view older revisions, or submit "bugs" or "feature enhancements" for improvements to the document.  Overall, it seems like it should result in better papers.

Anyway, the first document there is a document describing Serval Rhizome.

Monday, October 15, 2012

Secure Communications Is Valuable

Well, we already knew this, but it is pleasing to see when others create things that are similar to what we are doing.  In this case, it is Phil Zimmermann (of PGP fame).  He has a startup called Secure Circle offering secure telephony and messaging:

http://www.fastcompany.com/3001938/phil-zimmermanns-silent-circle-builds-secure-seductive-fortress-around-your-smartphone

The actual company website is here.

Similarities between Silent Circle and Serval include the use of ECC for the crypto (we are using Curve25519 (effectively 251 bits) while they are using the NIST 384-bit curve), and some of the telephony services being offered.   There is no reason why we couldn't move to 384-bit or stronger crypto, but we do not see the need at this time.  It would be quite possible for us (or someone else) to switch out the crypto system for a different one should people desire to do so. The most practical reason why we are are not yet on a longer ECC system is that the NaCl library that we use is yet to support one, which reflects, in part, the fact that a (well implemented) 251-bit ECC system offers ample security for the time being.

Differences include that Serval is free (as in beer and as in liberty) and open-source, and rather importantly, that Serval technologies are designed for resilience, and do not require a cellular network or internet access to facilitate communications.


Monday, October 8, 2012

Using Serval as a Basis for Environmental Monitoring

One of our developers, Corey Wallis, is in the midst of a project with some of the other researchers in the Flinders University Disaster Research Centre (DRC) where they are creating a system to acquire temperature and humidity data from mass gathering events, such as concerts and sports events.

This involves both fixed and mobile devices with sensors continuously collecting data for analysis both at the event and on-line.

One of the tricks is that we can't rely on cellular service being available at these events.  Also, we want all of the field teams who are gathering data to also be able to visualise and act on the data, so we need a many-to-many distribution scheme.

These features make it an good fit for Serval Rhizome, our store-and-forward/Delay Tolerant Networking (DTN) system, that is able to handle both of these requirements, and will result in a very versatile system that can be used anywhere in the world.

Corey has made some good progress on the hardware and software for the data gathering, as can be seen in this blog post:

http://magdaaproject.org/2012/10/08/mobile-environment-monitoring-software-update/

Among the next steps are to actually integrate it with the Serval Mesh software so that the data can be shared among the field teams and analysis staff.

Saturday, October 6, 2012

The Serval Technology Stack

Back when we started with the Serval Project we were focussed on creating a proof-of-concept that showed the potential of the concept of enabling mobile telephones to communicate directly, and allow people to communicate in difficult situations.

As a result of that approach, we took largely existing VoIP and mesh routing technologies and fitting them into a mobile telephone.

We already had ideas at the time that there was a better approach. This was reinforced by our early experience where we found that the combination of SIP was very sensitive to packet loss during call setup.

To cut a long story short, we decided to implement our own suite of mobile mesh oriented protocols overlay network, and mesh oriented protocols and technologies to replicate the core functionality, not just of SIP/VoIP, but also UDP-style datagram services, file distribution, SMS-like services and more.

Today I presented at #IS4CWN describing this new technology stack and explained a lot of our reasoning behind it.  I have been meaning to document some of this for a while, and now it has finally happened.  Now that I have some diagrams and slides, my intention is to write a series of posts explaining the various parts of the new Serval technology stack.

For this post I want to start with an overview of the new technology stack, which I will then expand on each part in future posts.


Starting at the bottom, we have the various base layers that the Serval Mesh protocols all might be encapsulated in.  While the current implementation is an overlay encapsulated in IPv4 UDP packets, the overlay was designed to allow the kind of flexibility reflected here, with digital packet radio and other transports being anticipated from the outset.

The trained squirrels is somewhat tongue-in-cheek, but serves to represent that we are thinking broadly about the form of possible transports.  It is hard to write kernel drivers that can expose a fleet of trained squirrels as a first-class network interface on a variety of operating systems, and the same goes for packet radio and other interfaces.

This is a good reason for us to pursue an overlay network approach, because it avoids the need for kernel cooperation, even if using strange transports.  It also allows us to step back from some of the overheads and anachronisms of IP networking that are not appropriate for low-bandwidth or high-latency links, such as packet radio.

This naturally leads into explaining the nature of the Serval Overlay Mesh protocols itself.  As mentioned, one reason for pursuing the an overlay model is to allow the use of non-conventional packet transports.  Another reason is to avoid the pain of IP address allocations.  IPv4 doesn't have enough address space to allow random self-allocation of addresses at a global scale.  It also turns out that IPv6 doesn't either, because there are only 64 host address bits, and there are more than 2^(64/2) people and devices in the world, which is the practical limit for random address allocation.  Structured address allocation is not feasible for mobile mesh networks that are intended to provide resilient communications.  Further, not all mobile phone platforms support IPv6.

So our view is that IPv6 doesn't really offer any compelling advantage for our use-case.  Our approach was to instead use 256-bit Elliptic Curve Cryptography public keys as network addresses.  This also makes end-to-end encryption of communications between nodes trivial, because a Diffie-Hellman shared secret agreement process can be used to facilitate the use of a stream cipher.  And this is exactly what we have implemented for our Mesh Datagram Protocol (MDP), which is otherwise very much like UDP.

The transparent encryption offered by MDP is used to host our Voice over Mesh Protocol, which then benefits from this.  Of course security isn't just encryption, and there are authentication and man-in-the-middle attack issues to be considered.  But I will leave that discussion for a future post.

So that covers the right hand side of our protocol stack, which is all the real-time protocols.

The left hand side covers the Rhizome store-and-forward or Delay Tolerant Networking (DTN) protocol and applications layered on top of that.  Full explanation of that will also be the subject of a future post.

Friday, August 3, 2012

It's competition season

I have been named a finalist in the South Australian Science Excellence Awards this year, which is really nice.

You can check it out at: http://www.scienceawards.sa.gov.au/science-excellence-awards-fina/

In a rather surprising turn of events there are four people from the University department where I am based, not bad considering that our department is no where near 1/22 of the science output of the state.

This year they have announced a "people's choice" award that has a basket of fine South Australian chocolates as the lure to get people to "like" the science awards facebook page.

If you are not adverse to facebook, feel free to vote for me so that we can win the chocolate hamper and share it with our supporters.  You can vote every day until 17th August 2012 to maximise our chances of winning the hamper. The link to vote for me is: http://bit.ly/SWLk02

Sunday, July 15, 2012

The Serval Project - Reflections Two Years In

It is now almost exactly two years since we made the first public demonstration of the Serval Project's vision for keeping mobile phones working without cellular networks or infrastructure, and it seems right to recap where we have come in the past year.


A year ago, the software was still more or less the same as was demonstrated in the video linked above in mid-2010.

Support

Support for the project consisted of initial seed funding from the Awesome Foundation, and a generous research fellowship from Flinders University, that freed up the majority of my time to focus on the project.  The project then received a further significant boost with my being granted a Shuttleworth Foundation fellowship.
Dr. Paul Gardner-Stephen in South Africa, while meeting with the Shuttleworth Foundation.
This combination of University and foundation support has proved to be amazingly effective.  The University has provided facilities to accommodate the project, and access to a variety of expertise and relevant research interests, especially in the disaster research space, where Flinders University is particularly active and well respected internationally.  The Shuttleworth Foundation has provided the financial means to free up all of my time to focus on the project, as well as employ developers and other resources to allow the project to drive forward much more quickly than would otherwise have been possible, and to enable me to keep some focus on the big picture, instead of having to do all the technical work myself.  Both organisations have the public interest at heart, and have been particularly accommodating of the unusual nature of the project, and so the relationship has worked much better than I had even hoped.  This is something of which I am continually grateful.

Technology

Further resources were added due to support by the Open Technology Initiative's Commotion Project, which is focussed around resilient communications systems to protect against human rights violations and what might be generally called "politically induced disasters" as compared to natural disasters.


Using these resources, the Serval Project has set about transforming the proof-of-concept demonstration software of 2010 into a production quality system.  This has resulted in near complete replacement of every component.

Instead of using Asterisk and SIPdroid for voice calls we have implemented our own mesh-oriented voice protocol and programs, resulting in a much smaller, faster program.

The underlying mesh network that allows the phones to communicate directly with one another, and to relay calls and data for one another has been redesigned and reimplemented from the ground up, with high-grade security baked right in.

The Rhizome store-and-forward data service has been created, and then rewritten to take advantage of the new mesh implementation.  MeshMS (Mesh-based SMS) has in turn be reimplemented to make use of Rhizome, and has been demonstrated delivering a message between Africa and Australia, using nothing more than three mobile telephones.

The user interface has been completely redesigned and reimplemented, to give a much more integrated user experience.
Screen-shot from a nightly build of the next planned release of the Serval Mesh, featuring redesigned user interface.

Increasing Recognition

This year has also seen a continuing increase in recognition of what we are doing, both locally and internationally.  Either myself or the Serval Project has been short-listed or awarded in the Rolex Awards for Enterprise (short-listed), South Australian Tall Poppy Awards (short-listed, announcement of winners pending), South Australian Science Excellence Awards (short-listed, announcement of winners pending), Flinders University Early Career Researcher Awards (awarded), Anthill Smart 100 (awarded), NexExplo Top 100 (awarded), and Ashoka Change Maker's Competition (short-listed).
The Serval Project was one of 11 finalists in the Ashoka Change Makers Citizen Media competition.
The Serval Project was one of the top-100 innovations named in the Netexplo list of 2012.



The Serval Mesh software has now been downloaded more than 13,000 from the Android Market.  More than half of the downloads have been from France, we presume the result of our appearing in high-profile news service La Monde, and the involvement of exchange students from INSA Lyon in the Serval Project.

Increasing Collaboration & Impact

But perhaps most exciting has been the increase in collaboration and impact as we begin to reach sufficient technology maturity for our software to be used in trials, and hopefully in the next 12 months in operational deployments.  

Collaborations with researchers in South Africa and Israel have already resulted in one journal paper being published, and another two are in preparation.  Perhaps most importantly, these collaborations are a clear marker of the Serval Project becoming international in its endeavour and undertaking, so that in the long term it will not fall upon our shoulders alone to develop and maintain the technology.

But most exciting of all has been the two trial deployments, one in Nigeria as part of a human rights/citizen journalism project, and the other in New Zealand with the New Zealand Red Cross IT&T Emergency Response Unit by trialling our technology at their week-long KiwiEx'12 training exercise.
Members of a community in Nigeria familiarise themselves with the Serval Mesh software.

The relationship we have developed with the New Zealand Red Cross ERU has been tremendously helpful in a number of ways.  It has given us the opportunity to understand, first hand, what the needs of an operational deployment in a disaster zone are, and this has directly shaped the development of the technology, and continues to do so.  It has also opened the doors to other NGOs and even UN agencies to discuss our technology and how it might be able to be used by those organisations.  
Emergency Operations Centre at KiwiEx'12, where Serval Mesh technology was trialled.

A recent outcome of this is that along with the NZ Red Cross and other partners who trialled technology at KiwiEx'12, we have been invited by the UN WHO to put together a proposal for a standardised resilient field data collection capability for their consideration.

Summary

So overall it has been a year of getting ready and making tentative first steps of engagement with the international community.  Together with the software development progress that has been made the feeling is that this year has been one of preparation for broader engagement and increased impact in the year to come.  By continuing to focus on the humanitarian/disaster response use-case for the Serval Mesh technology, and the relationships that we have already begun to create in that space, we are confident that the coming year will be an exciting one, and one where it is our goal that by mid-2013 at least one emergency service or disaster response organisation will have some aspect of Serval technology ready for deployment to support them in their future operational exercises.

Thursday, July 5, 2012

Sneak-Peak of Encrypted Calls, Even on Slow Android Phones

Just a quick post to show the latest development code of the Serval Mesh running an encrypted phone call on some low-end handsets, and keeping comfortably below 50% CPU:


As well as encryption, this call is being carried by the Serval Overlay Mesh, and is all native VoMP and MDP, and contains no SIP, RTP or anything IP dependent.  So this demonstrates the feasibility of our Overlay Mesh architecture as it might be used for phones with a built-in Arduino or similar with ISM-band UHF radio modules to support longer-range mesh links.

Some of the output from top while in the call is below.



User 27%, System 25%, IOW 0%, IRQ 2%
User 46 + Nice 43 + Sys 85 + Idle 147 + IOW 0 + IRQ 0 + SIRQ 7 = 328

  PID CPU% S  #THR     VSS     RSS PCY UID      Name
 2627  21% R     1  10336K   4544K  bg app_72
/data/data/org.servalproject/bin/servald
 2537  11% S    21 140628K  23560K  fg app_72   org.servalproject
  135   8% S    60 197008K  28672K  fg system   system_server
 2600   7% R     1    904K    420K  fg shell    top
 2611   6% S     1      0K      0K unk root     dhd_dpc
   54   0% S     1      0K      0K  fg root     gs_wq
    4   0% S     1      0K      0K  fg root     events/0
 1702   0% S    12 129616K  22368K  bg app_19
com.google.android.apps.maps:NetworkLocationService
   76   0% S    12  17188K   1340K  fg root     /system/bin/rild
   11   0% S     1      0K      0K  fg root     kseriod



User 31%, System 21%, IOW 0%, IRQ 2%
User 63 + Nice 43 + Sys 73 + Idle 146 + IOW 0 + IRQ 0 + SIRQ 8 = 333

  PID CPU% S  #THR     VSS     RSS PCY UID      Name
 2627  19% R     1  10532K   4740K  bg app_72
/data/data/org.servalproject/bin/servald
 2537  18% S    21 137556K  23600K  fg app_72   org.servalproject
 2600   7% R     1    904K    420K  fg shell    top
 2611   5% S     1      0K      0K unk root     dhd_dpc
  135   1% S    60 197008K  28672K  fg system   system_server
 1824   0% S    10 123752K  21508K  bg app_33   android.process.acore
   54   0% S     1      0K      0K  fg root     gs_wq
    8   0% S     1      0K      0K  fg root     sync_supers
    9   0% S     1      0K      0K  fg root     bdi-default
   10   0% S     1      0K      0K  fg root     kblockd/0



User 31%, System 24%, IOW 0%, IRQ 0%
User 64 + Nice 39 + Sys 79 + Idle 145 + IOW 0 + IRQ 0 + SIRQ 1 = 328

  PID CPU% S  #THR     VSS     RSS PCY UID      Name
 2627  18% R     1  10728K   4936K  bg app_72
/data/data/org.servalproject/bin/servald
 2537  14% S    21 137524K  23416K  fg app_72   org.servalproject
 2600   7% R     1    904K    420K  fg shell    top
 2611   5% S     1      0K      0K unk root     dhd_dpc
 1859   4% S    12 131092K  25252K  bg app_18   com.huawei.launcher2
  135   3% S    60 197008K  28672K  fg system   system_server
   54   1% S     1      0K      0K  fg root     gs_wq
  227   0% S     8 112696K  12084K  fg app_44   com.nuance.nmc.sihome
    9   0% S     1      0K      0K  fg root     bdi-default
   10   0% S     1      0K      0K  fg root     kblockd/0



User 26%, System 23%, IOW 0%, IRQ 0%
User 43 + Nice 45 + Sys 76 + Idle 161 + IOW 0 + IRQ 0 + SIRQ 2 = 327

  PID CPU% S  #THR     VSS     RSS PCY UID      Name
 2627  22% R     1  10916K   5124K  bg app_72
/data/data/org.servalproject/bin/servald
 2537  14% S    21 137524K  23484K  fg app_72   org.servalproject
 2600   6% R     1    904K    420K  fg shell    top
 2611   4% S     1      0K      0K unk root     dhd_dpc
  135   1% S    60 197008K  28672K  fg system   system_server
   54   1% S     1      0K      0K  fg root     gs_wq
    7   0% S     1      0K      0K  fg root     suspend
    8   0% S     1      0K      0K  fg root     sync_supers
    9   0% S     1      0K      0K  fg root     bdi-default
   10   0% S     1      0K      0K  fg root     kblockd/0



User 29%, System 23%, IOW 0%, IRQ 0%
User 55 + Nice 41 + Sys 78 + Idle 150 + IOW 0 + IRQ 0 + SIRQ 3 = 327

  PID CPU% S  #THR     VSS     RSS PCY UID      Name
 2627  19% R     1  11112K   5320K  bg app_72
/data/data/org.servalproject/bin/servald
 2537  18% S    21 137524K  23528K  fg app_72   org.servalproject
 2611   6% S     1      0K      0K unk root     dhd_dpc
 2600   6% R     1    904K    420K  fg shell    top
  135   1% S    60 197008K  28672K  fg system   system_server
   54   0% S     1      0K      0K  fg root     gs_wq
 1859   0% S    12 131092K  25236K  bg app_18   com.huawei.launcher2
    4   0% S     1      0K      0K  fg root     events/0
   10   0% S     1      0K      0K  fg root     kblockd/0
   11   0% S     1      0K      0K  fg root     kseriod



User 27%, System 22%, IOW 0%, IRQ 1%
User 41 + Nice 49 + Sys 73 + Idle 160 + IOW 0 + IRQ 0 + SIRQ 5 = 328

  PID CPU% S  #THR     VSS     RSS PCY UID      Name
 2627  20% R     1  11308K   5516K  bg app_72
/data/data/org.servalproject/bin/servald
 2537  16% S    21 137524K  23396K  fg app_72   org.servalproject
 2600   6% R     1    904K    420K  fg shell    top
 2611   6% S     1      0K      0K unk root     dhd_dpc
  135   0% S    60 197008K  28672K  fg system   system_server
    6   0% S     1      0K      0K  fg root     async/mgr
    7   0% S     1      0K      0K  fg root     suspend
    8   0% S     1      0K      0K  fg root     sync_supers
    9   0% S     1      0K      0K  fg root     bdi-default
   10   0% S     1      0K      0K  fg root     kblockd/0