tag:blogger.com,1999:blog-90686389948941029682024-03-18T00:46:03.362-07:00Enabling Communications, Anywhere, AnytimePaul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.comBlogger201125tag:blogger.com,1999:blog-9068638994894102968.post-3459583173793763532022-09-30T04:40:00.003-07:002022-09-30T04:40:21.128-07:00Distributed Search & Discovery - Milestone 1<p>With the support of the NLnet Foundation, we have been looking at extending the Serval Mesh to support search and service discovery functionality on isolated mesh networks, i.e., without relying on internet access. This is what we talked about doing for this project:</p><p>
</p><p class="western">
<span style="font-family: Liberation Sans;">Search and discovery have become
invaluable elements of the modern internet. However, for communities
experiencing transient or chronic loss of internet access, these
facilities are not available. This can prevent these communities from
being able to promote and find and utilise local services – which
together can impair economic capacity and recovery. This project
seeks to add a simple fully-distributed search and discovery
mechanism to the Serval Mesh.</span></p>
<p class="western"><span style="font-family: Liberation Sans;">The COVID19, Cyclone
Harold and the complete lock-down of Vanuatu has made it problematic
to pilot the result with our South Pacific partners there. Testing
will thus occur with IRNAS in Europe and also in Australia, where we
have access to the Arkaroola Wilderness Sanctuary as a geographically
large test site. The capstone output being a Ubuntu package for the
Serval Mesh, including the new search and discovery functionality.</span></p>
<p><style type="text/css">p { margin-bottom: 0.1cm; background: transparent }p.western { font-family: "Liberation Sans", sans-serif }a:link { color: #95be00; background: transparent; text-decoration: underline }a.western:link { font-size: 10pt; so-language: en-AU }</style></p><p>This blog post describes the work undertaken over the part months on this, to satisfy the first milestone. <br /></p><p>
</p>We have come to take the ability to search for information and
services on the internet for granted, precisely because they have
become so useful and universally available. The trouble is that they
require an extensive international infrastructure in order to
operate. This means that they are not available in areas lacking
communications infrastructure, or where communications infrastructure
has been destroyed or denied.
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;"><br />
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">It is
not merely as simple as saying that we should create a distributed
internet that supports all of the services that we currently expect
from the internet, because global services require infrastructure to
operate. Otherwise the provider’s of these services would have
implemented them themselves in this way, because they would have been
very happy to save all of the money that they otherwise have to spend
on creating and maintaining that infrastructure.</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;"><br />
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">However,
if we narrow our focus down to how we can provide <i>local</i> search
and discovery services, then the situation changes. Just as Serval’s
SMS-like MeshMS service and local social-media system, MeshMB, are
able to provide local services without requiring infrastructure, the
challenge of this current project is how we can define and provide
local search and discovery services built on the Serval Mesh’s
distributed communications primitives. And that is exactly what this
milestone sets out to do.</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;"><br />
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">So
let’s break this down into the three steps that we need to pursue
to achieve this:</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;"><br />
</p>
<h2 class="western" lang="en-AU">1. Identify the precise search and
discovery services to be provided, including from both data provider
and consumer perspectives.</h2>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">So what
do we think are the kinds of search and discovery services that we
can implement over the Serval Mesh’s distributed networking
primitives? Serval’s Rhizome distributed data system effectively
works like a magic file share: Various users put files (or file-like
objects we call bundles) into it, and those then get distributed
throughout the network in an opportunistic manner. The result is an
approach that provides eventual consistency, provided that there is
sufficient storage on each node, and provided that there is
sufficient opportunities for network communications.
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;"><br />
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">The
storage limitation is addressed by having a prioritisation system, so
that the highest priority material is retained, if there is not space
for everything. To date, the priority system has been relatively
simple, with smaller files/bundles taking priority over larger ones.
This could be easily extended to prioritize local content over remote
content in a variety ways. This could include either geographic
bounding boxes for content, that indicate that it is only of
relevance in a given geographical area, a hop counter, or
alternatively that content is tagged as belonging to a given
community. Community tagging has the advantage that material could
follow a community who were on the move. Neither is perfect, nor are
they bullet-proof against various forms of denial of service
measures, as is common for most mesh networking systems. Essentially
any mesh network is very difficult to defend against a well-resourced
aggressor. We will thus just focus on how we can add the ability to
search and discover content on a mesh network.</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;"><br />
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">The
Serval Rhizome system supports both encrypted and non-encrypted
content. Each bundle also has meta-data. At the most basic level, we
can simply allow users to search this content, without describing any
additional services or semantics. We should do this, but also
consider services that can be created that intentionally support
discovery and search. Also, from another perspective we should think
about the types of discovery and search services that are desirable
for infrastructure-deprived contexts. Ideally, by considering what is
possible, and what is desirable find those capabilities that are
going to be successful in practice.</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">From
the possibility perspective, we need to formulate search and
discovery systems that can work using only cached data. This leads to
a positive characteristic, in that the time to perform a search can
be very rapid in almost all cases, because no network latency will be
involved. There is of course a negative aspect, in that if the data
has not reached the device performing the search, than it cannot be
searched. This is the fundamental limitation of the the Serval
Rhizome data model, which is the trade-off of the ability to operate
completely independent of supporting infrastructure. As with every
other network model, the nature of the network influences the nature
and architecture of the network services that can be delivered –
especially if they are to perform well.</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;"><br />
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">To help
us identify services that are of interest to consumers, it’s
probably a good idea for us to look at statistics relating to search
engine use, such as from somewhere like these data from 2020:</p>
<p lang="en-US" style="line-height: 100%; margin-bottom: 0cm;"><a href="https://www.statista.com/statistics/271599/most-common-online-search-topics-of-internet-users/"><span lang="en-AU">https://www.statista.com/statistics/271599/most-common-online-search-topics-of-internet-users/</span></a></p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">This
lists the top 20 reasons for search globally, which has some
limitations for our purposes, which we will discuss shortly, but
let’s look at the top 10:</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;"><br />
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">13% -
Translate</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">11% -
Social Network</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">10% -
Entertainment</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">10% -
Shopping</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">10% -
Weather</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">9% -
News</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">8% -
Web Services</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">5% -
Gambling</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">4% -
Mail</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">4% -
Traveling</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;"><br />
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">Let’s
start by saying that I don’t really have a strong opinion as to how
reliable I believe these percentages to be, but the overall
categories look reasonably sensible, although I am dubious when
pornography doesn’t make it into the top-20 list at all. Perhaps
that comes under “entertainment”, “shopping” and/or “Web
Services” to some degree. But that’s rather academic for us,
because we are not interested in creating an infrastructure-free
pornography distribution service. In fact, history has shown that
you can tell when a communications medium begins to be useful,
precisely because it starts being used to carry pornography.
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;"><br />
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">Rather,
we are actually actively wanting to create a platform that <i>doesn’t</i>
get overwhelmed by porn. This can be difficult, precisely because of
the propensity for communications media to get used for this purpose.
The services we are creating need to be crafted as intrinsically
safe places, to the maximum extent possible. This is not always easy,
and can never be done perfectly, but there are sensible measures that
can be employed to help.
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;"><br />
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">The
corollary to this is that if you don’t take steps to make
communications spaces safe, then they rapidly end up as very unsafe
places. This has shown itself repeatedly, especially in recent
history: CB radio with its near total lack of accountability consists
in most places in large part of a stream of profanity and abuse,
making it unsafe for families and young peoples – in spite of often
quite strong laws that ban even the use of swear words on the medium.
Then we have more similar examples to our area of interest, where
online chat systems such as FireChat. I recall around the time it
was released trying it out in the mundane environment of an
Australian city, and even here, the lists of nearby users were
eye-scorchingly offensive to read. This isn’t a complaint about
FireChat per se (and they might well have great tools for dealing
with this now), but really just an example of what I consider to be
an axiom of human communications: If people are unaccountable for
what they say, they will say just about anything, and a lot of it
will be designed to shock, and a lot of it will be sexually explicit
and abusive. If we want to create services that are going to be wide
spread by civil society, rather than just “uncivil” users, then
we have to take active measures to help nudge the use of the system
in the right directions, and to reduce the potential for abusive use
to be easy or attractive.</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;"><br />
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">For
example, the text messaging and neighbour discovery mechanisms
already in the Serval Mesh are quite conservative about showing you
random participants who are a long distance away on the network, to
reduce the potential for unaccountable abusive activity, such as
using offensive account names, and allowing random people to send you
messages that appear together in your personally curated inbox.
Instead, your inbox only shows content from people you have added as
contacts, or in the case of the Twitter-like micro-blogging facility,
only of people you have chosen to follow their feed. This creates a
safe working place, where you can carry on your day to day activity,
without fear of random anonymous abuse.
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;"><br />
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">The
trade-off is that you have to actively choose to go through your
contact requests, when you want to add new people to your network.
This makes it harder for social groups to self-organise, and
necessarily means that you may have to make excursions from the safe
space into the wild exterior. One way to improve on this, is to have
a filter for truly “nearby” users, so that you can find people on
the network when you meet in person. This could require direct
network connection, i.e., that user’s devices are physically
directly peered with one another, or use some other mechanism, such
as a more relaxed hop-count limit, or perhaps better, to sort the
list of potential contacts by hop-count and time since last
reception, so that fresh local requests appear at the top of the
list. Of course, this current work on supporting search and discovery
mechanisms helps to directly address this issue as well, and we will
keep solving this problem in mind as we design them.</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;"><br />
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">Another
approach that could help would be to have a mechanism to create
closed communities, where a common passphrase or similar is used to
tag network content as belonging to a particular community. There are
also clever cryptographic mechanism to achieve this effect that don’t
allow for a random unaccountable user to obtain and use the
passphrase. Such approaches help not only to filter results to a
manageable level in a large network, but also directly addresses some
of the unaccountability that leads to what we might generally call
anti-social behaviour on these platforms.</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;"><br />
</p>
<p lang="en-AU" style="line-height: 100%; margin-bottom: 0cm;">Hybrid
approaches are also possible, where for example, only users in your
verified social group are able to have image content displayed, so
that the benefits of images can be used in a service or facility, but
with reduced probability of this filling up with pornographic or
other anti-social content. There is considerable potential solution
space for such measures, well beyond the scope of this current
project. But what is relevant to this project and should be
included, is at least some exemplar mechanism for discouraging
anti-social behaviour, and where necessary, excluding anti-social
users and their content. That is, it must like the existing Serval
Chat (MeshMS) and Serval Micro-Blogging (MeshMB) services follow a
“safe place by default” approach.</p>
<h2 class="western" lang="en-AU">2. Create several user-stories for
how the search and discovery systems might be used in practice.</h2>
<p lang="en-AU">To create our user-stories, we will follow the Serval
Project’s long focus on post-disaster response and recovery among
the general public. We will thus imagine a situation where a
community has had infrastructure damaged, and is without functioning
communications infrastructure to reliably connect them to the outside
world. Of course this scenario has strong parallels to what we see
when people are living in regions of unrest as well, so it’s a good
starting point for us. The following user-stories will walk us
through an extended scenario in such a post-disaster situation. All
names and locations are fictitious.</p>
<h2 class="western" lang="en-AU">2.0 Scenario Background</h2>
<p lang="en-AU">The communities of Ulia, Luania, Saua and Kiatu live
on, Tija, a long and thin island of 800 square kilometres at the
northern end of the archipeligo, some 400km from the national capital
of Ari Ula, and 90km from the next nearest populated island. The
3,500 people living on Tija enjoy life on their tropical island,
located in the South Pacific. Kiatu, the regional capital, is the
largest population centre on the island, with a population of just
over 2,000 living in the town and surrounding villages. There is 4G
cellular coverage in the town centre, with 2G coverage extending out
to the surrounding villages. The cellular service, including internet
access, is connected to the rest of the country via a VSAT satellite
link. Luania is the nearest of the other communities to Kiatu,
although it is still some 15km away. Depending on weather conditions,
can sometimes use mobile phones from on the top of a nearby hill,
that has line of sight to Kiatu. Saua is 7km on the other side of
Luania from Kiatu, and has no cellular network access. Ulia is yet
another 12km further on from Saua, and also lacks cellular coverage.
There are NGOs or small government presences in Saua and Ulia, who
have HF radios, that are used to provide a basic level of
communications back to Kiatu and the national capital.</p>
<p lang="en-AU">Tija’s location in the southern tropical zone of
the Pacific exposes it to considerable geophysical hazards, including
cyclone, tsunami and earthquake, like many other Pacific Island
Countries. GDP per capita is relatively low, yet as has become common
around the world, almost all families and most adults have access to
a low-end smart-phone. The relative expense and difficulty accessing
mobile internet means that the population is quite familiar with
“side loading” of applications on their Android phones, as is the
case in many Pacific Island Countries. The Tijans have a strong
community structure based around traditional chiefdoms in each of the
four communities, and work constructively with one another whenever
disasters strike.</p>
<p lang="en-AU">Because of their vulnerability, and isolation from
conventional communications, a number of residents in each of the
four communities on Tija already use the Serval Mesh peer-to-peer
software on their Android smart-phones.</p>
<p lang="en-AU">Our scenario begins with a 7.2 magnitude earthquake
that causes substantial damage, especially in Kiatu, which is nearest
to the epicentre. Fortunately the earthquake triggers only a
relatively small tsunami, and there is no immediate loss of life.
However the combination of the earthquake and tsunami have destroyed
the cellular infrastructure and its satellite earth-station. It will
likely take the mobile network operator a couple of weeks to restore
service, as they are also having to proritise restoration of service
to larger communities nearer the national capital.</p>
<h2 class="western" lang="en-AU">2.1 Forming a community</h2>
<p lang="en-AU">With conventional communications to the outside world
down, the Tijan communities need to find their own ways to
communicate, so that they can coordinate their own response to the
disaster, and generally keep their communities functioning as
effectively as possible.</p>
<p lang="en-AU">The larger community in Kiatu is particularly keen to
enable local businesses to continue trading, and to ensure that the
limited local resources can be readily used where required, e.g.,
building materials required to repair infrastructure, as well as the
people required to undertake such work. They also have a need to
identify where infrastructure is damaged, so that they have the
situational awareness that will enable them to be effective in their
response. Knowing where damaged infrastructure is will also be
helpful for re-routing traffic and ensuring public safety.</p>
<p lang="en-AU">The smaller out-lying communities are also keen to
ensure that local communications is possible, so that they can ensure
the well-being of their communities, and collate information about
any damaged infrastructure so that this can be communicated to the
regional and national capitals, so that remedial works can be carried
out when the expected foreign aid support starts to flow into the
country.</p>
<p lang="en-AU">To support this, in each of the communities someone
opens the Serval Mesh app, and uses the <b>Create Community</b>
function to create a new local community on the mesh network. As well
as a community name, a shared secret is set up that is required by
participants who wish to contribute material into the community, but
reflecting the friendly socio-political environment on Tija, the
communities are set up to allow read and search access without
needing the shared secret. Also in the smaller communities, the
shared secret is set to something quite simple, such as 1234, as the
communities prioritise ease of access over security.</p>
<p lang="en-AU">In Kiatu, the larger population means that several
different people have attempted to create the community, resulting in
some initial confusion. Later in the day, the regional
administration select one of those communities to be the official
Kiatu earthquake response community on the Serval Mesh. Nonetheless,
it takes a few days for almost all members of the dupliate
communities to switch to the official community. These problems do
not emerge in the smaller communities, as their populations are small
enough that the normal village meeting mechanisms are able to lead a
unified approach in each community.</p>
<p lang="en-AU">At this point, there are four primary communities
that have been created: <i>Official Kiatu Earthquake Response</i>,
<i>Ulia Earthquake</i>, <i>Luania Together</i>, and <i><span style="font-weight: normal;">Saua
Saua</span></i>. This is inaddition to the short-lived duplicate
communities in Kiatu: <i>Kiatu Earthquake Help</i>, <i>Earthquake
Help</i>, and <i>Kiatu Together</i>. However, there are relatively
few users in each. Initially, the duplicate Kiatu communities have
more users than the official community, reflecting the ability of
grass-roots organisation to be faster than government processes,
especially following a disaster.</p>
<h2 class="western" lang="en-AU">2.2 Building a community</h2>
<p lang="en-AU">After creating the communities, individuals within
those communities approach others and ask if they have joined the new
communities, and explain to them how the communities can help them,
by allowing the collection and sharing of information of relevance to
them. For those who do not yet have the Serval Mesh on their phone,
they side-load the application using the bluetooth tools for doing so
that they are already familiar with, or they use the wifi-based
sharing mechanism in the Serval Mesh app.</p>
<p lang="en-AU">This process continues until all community members
who wish to join a community have joined.</p>
<h2 class="western" lang="en-AU">2.3 Searching for communities</h2>
<p lang="en-AU">Some community members don’t wait until someone
invites them, but use the <b>Search for communities</b> function in
the Serval Mesh app to find communities. Most users use the <b>Find
communities that my contacts are in</b> function to do this in a safe
way, that avoids the risk of seeing offensively named communities.
Some users choose to search through all available communities, after
clicking on a confirmation that unprotected searching may result in
them seeing offensive material. Either way, when they find the
community that they are looking for, they use <b>Join Community</b>,
and if they make an error, or wish to otherwise leave a community,
they use the <b>Leave community</b> function.</p>
<p lang="en-AU">As members of the communities have family and friends
in the other villages on Tija, they use the <b>Search for communities</b>
function to discover and join the other communities that have been
setup on Tija. Among those is Neire, who lives in Ulia.</p>
<h2 class="western" lang="en-AU">2.4 Searching for people: User
discovery</h2>
<p lang="en-AU">Neire helps to look after her grandmother, Iu. Neire
helps her grandmother setup the Serval Mesh on her phone, and helps
her to join the <i>Ulia Earthquake</i> community. She then uses the
<b>find users</b> function, with the search scope set to <i>only look
for people in my community</i><span style="font-style: normal;">, to
find herself on Iu’s phone, and to find Iu on her phone. They
confirm that they can send each other text messages and voice
messages. Neire then helps Iu to find other relatives who have
joined the community, and adds them all to Iu’s contact list, so
that she can easily communicate with them.</span></p>
<h2 class="western" lang="en-AU">2.5 Searching for services: Service
& Data discovery</h2>
<p lang="en-AU">Neire is helping with the recovery effort. As she can
drive, the village chief allows her to use the village‘s truck to
help move people and goods, as required. Neire uses the <b>offer
service</b> function to advertise to each of the communities the
availability of freight and transport services for the next three
days. She offers this for free.
</p>
<p lang="en-AU">The truck has only a ¼ of a tank of Diesel, so Neire
uses the <b>service search</b> feature to search for available
sources of diesel fuel. She also uses the <b>publish request</b>
feature to announce that she is looking for diesel fuel. Using this,
after a short time, she receives a <b>resource match alert</b>, that
contains a list of current offers for diesel fuel that have been made
in the communities that she is participating in. She thus discovers
that there is some diesel available in Saua, and prepares to head to
Saua. Before departing, she <b>searches the list of published
requests</b> from the Saua community.
</p>
<p lang="en-AU">By this time, more people have had time to make
requests, and she discovers that Saua is running short on fresh
drinking water, because the earthquake has damaged some of their
water tanks, and the tsunami has caused contamination of several
other water tanks. Neire thus arranges with others from her village
to cart 900L of drinking water in a variety of containers. One of
the other community members uses the <b>offer resources</b> function
to announce the availability of ample fresh drinking water in Ulia.
Finally, Neire and several of her extended family set off to Saua
with the water, and to collect the diesel.</p>
<h2 class="western" lang="en-AU">2.6 Searching for data/information:
Data discovery</h2>
<p lang="en-AU">Other members of the communities use the <b>search
published requests</b> and <b>search services</b> functions, both to
find resources that they need, as well as to understand what
resources and services they might be able to provide to meet unmet
needs.
</p>
<p lang="en-AU">Boli from Luania notices that there are unmet
requests for diesel fuel, because the only offers for diesel fuel are
from a few community members who have diesel in portable containers.
Boli assumes that this means that the fuel distribution centre in
Kiatu was damaged in the earthquake or tsunami. Boli and several
others in Luania have coconut plantations, and have hand-presses for
extracting the oil from coconuts, which can be used in adapted
diesel engines, provided that the weather is warm enough that it
doesn’t solidfy. Fortunately, the tropical climate on Tija means
that this is a very rare event. Boli thus uses the <b>publish service</b>
and <b>offer resources</b> functions to offer diesel fuel and fueling
services. Boli and the other coconut plantantion managers arrange for
the production of several hundred litres of oil over the next 24
hours, and keep an eye on demand through customers and through the
<b>search published requests</b> function.</p>
<h2 class="western" lang="en-AU">2.7 Searching for places: Mapping
and geoinformatic discovery</h2>
<p lang="en-AU">Meanwhile, Neire and her team are making their way
down towards Saua. Along the way, they discover several places where
the road has been damaged due to rock slides or erosion from the
tsunami. Neire has one of her team use the <b>crowd-sourced mapping</b>
function to mark these locations on the map, so that others will know
about the damage, allowing them to avoid it, or to begin work on
repairing them. Some of the damage is quite near to Saua, and after
Neire arrives, she shows some of the people in Saua how to use the
<b>crowd-sourced mapping</b> to <b>search for damaged infrastructure</b>,
and <b>search for resource offers</b> to find the materials that they
need to start repairing the worst of the damage.</p>
<p lang="en-AU">Neire uses the crowd-sourced mapping to navigate to
the locations where people had requested fresh drinking water and
helps distribute this. She is also able to efficiently plan her route
to include collecting the diesel fuel. The recipients of the water
use the <b>cancel resource request</b> after they receive the
drinking water. They have in the meantime logged their damaged water
tanks on the <b>crowd-sourced mapping</b> function. After Neire has
collected the diesel fuel, that offer is also removed, using the
<b>update resource offer</b> function.</p>
<p lang="en-AU">Neire now has enough diesel fuel to respond to
several other requests for moving materials around Saua, including
helping the road repair volunteers get their tools and materials to
the nearest damaged location. She knows that she will still need
more fuel, and has been keeping an eye on the <b>resource match alert</b>
function, and discovers the offer of coconut oil for use as diesel
substitute. She uses the Serval Mesh‘s <b>MeshMS text messaging
</b>function to make arrangements to collect 300 Litres of coconut
oil, enough to fuel her truck for the next couple of days. Because
she is able to communicate with Boli, she knows when the coconut oil
will be ready, and is able to help with other recovery activities
beforehand. The <b>crowd-source mapping</b> function also lets her
discover ahead of time the condition of the road, and allow
sufficient time to get there, and to bring some repair materials and
volunteers to stabilise the worst of the damage.</p>
<p lang="en-AU">Neire arrives at Luania in the early evening, soon
after Boli has her fuel ready. Neire uses the <b>search for services</b>
function to find a place where she and her team can spend the night,
before they continue helping their communities the next morning.</p>
<h2 class="western" lang="en-AU">2.8 Searching by low-literacy user</h2>
<p lang="en-AU">Not everyone on Taji has a high level of literacy.
Yeco is an older man in Luania who lives on his own when his family
travel to Kiatu for work. This is the case following the earthquake.
His family have not been able to return from Kiatu yet, as a bridge
has been damaged by the earthquake, making the road impassible.
However, Yeco is able to exchange voice messages with his family
sporadically, as people walk from Luania and Kiatu to opposite sides
of the damaged bridge, allowing messages to be automatically
exchanged via the Serval Mesh. Thus Yeco and his family know that
one another are safe. However, Yeco’s kitchen has been damaged,
and so he will require assistance finding nutritious food. Based on
instructions from his family, he opens the <b>crowd-sourced mapping</b>
function, and uses emojis attached to the <b>service offers</b> and
<b>resource offers</b> to find someone nearby who is offering food.
He sends them a voice message asking if they can bring him some food.
They ask him in a voice message to use the <b>share location</b>
function, which he does, thus enabling them to deliver him food until
his family are able to return from Kiatu.</p>
<h2 class="western" lang="en-AU">2.9 Searching by visually impaired
user</h2>
<p lang="en-AU">Ado is another elderly person living in Luania, and
while she is literate, her eyesight has deteriorated over the years,
making it very difficult for her to read. She is, however, able to
use the <b>vision impaired interface option</b> to access basic
functions of the Serval Mesh. This provides a text-to-speech
interface, which allows her to access and send voice messages to her
contacts. She also requires assistance with obtaining food while her
family are busy with the recovery efforts. She is able to <b>send
them voice messages</b> to let them know how she is going, and set
their minds at ease, and that she has used the <b>vision impared
interface to listen to the various resource offers</b>, and thus find
someone nearby who is able to provide her with nutritious food, and
to communicate with them using <b>voice messages</b>.</p>
<h2 class="western" lang="en-AU">2.10 Summary</h2>
<p lang="en-AU">The above scenario and its embedded user stories show
how communications in these sorts of situations can be very effective
at helping vulnerable communities. The combination of the existing
Serval Mesh functions, with the ability to share, discover and
request services and resources makes it much easier for isolated
communities to respond efficiently and with effective situational
awareness in ways that have not previously been possible. Now our
challenge is to collate the functions required to enable these
improvements, and map them to the Serval Mesh‘s delay-tolerant
networking primitives.</p>
<h2 class="western" lang="en-AU">3. Map these services to the Serval
Mesh Rhizome Delay Tolerant Networking Primitives, and work though
several user-stories to confirm that the model is fundamental
workable.</h2>
<h2 class="western" lang="en-AU">3.1 Mapping to Rhizome Primitives</h2>
<p lang="en-AU">Let’s start by summarising the functions that are
referenced in the above user-stories. Then we can look at which ones
are related, and how we can map these to Serval Mesh Rhizome
Delay-Tolerant Networking primitives, excluding those already in the
Serval Mesh, such as <b>find user</b>, or <b>Send MeshMS text
message</b><span style="font-weight: normal;">. These collect into
several fairly obvious groupings:</span></p>
<p lang="en-AU">Community Management:</p>
<p lang="en-AU"><b>Create Community <br />
</b><b>Search for
communities<br />
Find communities that my contacts are in<br />
</b><b>Join
Community<br />
Leave Community</b></p>
<p lang="en-AU" style="font-weight: normal;">Services & Resources:</p>
<p lang="en-AU"><b>Offer Service<br />
Offer Resource<br />
Search for
Service [Offer]<br />
Search for Resource [Offer]<br />
P</b><b>ublish
request </b><b>for service<br />
Publish request for resource<br />
Cancel
request for service<br />
Cancel request for resource<br />
Update
request for service<br />
Update request for resource<br />
Update
offer of service<br />
Update offer of resource<br />
Search service
requests<br />
Search resource requests<br />
A</b><b>lert </b><b>on
Resource Match</b></p>
<p lang="en-AU" style="font-weight: normal;">Crowd-Sourced Mapping:</p>
<p lang="en-AU"><b>Share to Crowd-Sourced Map<br />
Show Crowd-Sourced
Map</b></p>
<p lang="en-AU" style="font-weight: normal;">Transcending these are
the need to support accessible content, such as emojis for searching
visually, and text-to-speech and emoji-to-speech (including in
multiple languages). The mapping functions can be supported by
including a geographic tag, which might represent a point, a locality
or a geographic bounding box, depending on the circumstances. For
simplicity, it probably makes sense to use Google’s Plus Codes as a
succinct way to describe a location or locality.
</p>
<p lang="en-AU" style="font-weight: normal;">At a lower level, it
looks like we can fairly easily map these to Rhizome primitives. For
each user, we can have a bundle that contains the current list of
services and resources that they are offering and seeking. Each of
these can be associated with a Plus Code, that locates the request in
space.</p>
<p lang="en-AU" style="font-weight: normal;">One of the challenges
that we will need to face, is the cluttering of the system,
especially when people make offers or requests, and either forget to
remove them, or perhaps move out of the community. In these cases, we
don’t want these stale offers and requests to hang around, making
the system difficult to use. To solve this, in addition to the Plus
Code to locate the request in space, a time maker can also be added,
that indicates when the request or offer is valid, with fairly tight
maximum bounds. The complete bundle should include a claimed time of
publishing, and the time markers should be relative to this, and
limited to a maximum of, say, 3 days from that time of publishing.
This is designed to be long enough for requests and offers to spread
through the network, including in fairly difficult situations, but
not so long that they can clog up the system and cause more trouble
than help. Based on experience with working with humanitarian
response organisations, 3 days is a very long time in a disaster.
</p>
<p lang="en-AU" style="font-weight: normal;">Then we also have the
crowd-sourced mapping elements, that are separate from the resource
and service requests and offers. These can, however, be included in
the same bundle, and simply encode points of interest. Again, a
similar expiry system should apply, but probably with a longer
expiry, perhaps allowing up to 14 days. After 2 weeks, the
information is likely to be quite out of date. But the user can of
course choose to re-publish the information.</p>
<p lang="en-AU"><span style="font-weight: normal;">This approach
requires only one bundle per user, which will help the system be more
efficient. This bundle should use a unique service field in the
Rhizome Manifest that contains the bundle meta-data. </span><span style="font-weight: normal;">The
various functions then simply either modify a user’s own bundle,
and re-publish it into Rhizome, or search the bundles in Rhizome that
have this service field. No changes are required to the low-level
functionality of the Serval Mesh. </span>
</p>
<p lang="en-AU" style="font-weight: normal;">The existing
prioritisation scheme that prioritises small over large bundles will
prioritise those bundles that contain only a single request or offer,
over those that contain many offers. This is probably reasonable,
and is certainly easy to explain to users. The software could guide
the user to understand that if they want a “high priority”
request, then they need to make only that request, and not others,
and not also make offers. While imperfect, it is a reasonable
approach where priority and user focus are interconnected in a manner
that reflects real-life.</p>
<p lang="en-AU" style="font-weight: normal;">One challenge that has to
be addressed is to ensure that each user can have only one such
bundle. One approach to this is to have the Bunde ID (BID) of the
service discovery bundle be cryptographically tied to the user’s
ServalID (SID). This can be done in a hard 1:1 way, however, only
one such bundle can exist for a user for all purposes, and that
bundle is already used for sharing identity information on the mesh.
Thus we need to use a looser coupling. This can be done, by requiring
the User’s to sign the BID of the service discovery bundle to
attest its authenticity. If multiple bundles for the same user are
found, the search mechanism (and Rhizome storage system) can discard
those bundles that are not currently valid in time (which the
expiration scheme neatly enables), and if multiple bundles are valid
simultaneously, then the bundle with the later publication time
should be used in preference to any older ones. In this way, exactly
only the most recent bundle will be valid for the search function.</p>
<p lang="en-AU" style="font-weight: normal;">This leaves only the
community management functions. The communities could be managed in
separate bundles, however, we can handle this in the same bundle, by
allowing the inclusion of a number of communities that a user claims
to be a member of. Each community name would be followed by a
cryptographic hash that is the hash of the user’s SID, the
community name, and the shared secret that attests membership to the
community. The search functions then need only to filter the search
results according to whether the service discovery bundle has any
communities in common with the searcher.
</p>
<p lang="en-AU" style="font-weight: normal;">This information can also
be used to filter search results that are outside of the service
discovery bundle. For example, to allow searching through all
microblogging messages posted by members of a given community, or
other content that they might have uploaded into Rhizome.</p>
<p lang="en-AU" style="font-weight: normal;">Together, this all means
we need only one Rhizome bundle per user to support distributed
search and service discovery. Circling back to our need to enable
safe search spaces, this is also extended in a pleasingly simple way:
You can search your contacts, or extend that to searching in one or
more communities in which you participate, or you can search without
filters. This allows a graduated transition from very safe to unsafe
searching.</p>
<p lang="en-AU"><span style="font-weight: normal;">From an individual
security perspective, it is acknowledged that this approach discloses
to other users which communities a user is in. This could be
partially concealed, by not including the community names in the
service discovery bundle, but rather only the hash that can be
interrogated to verify if a user is a member of a given community.
However, this would only provide limited protection, because if
anyone knew the name of a community, they would be able to test if a
user was a member of that community. It would also require a
separate community advertisement mechanism, if communities are to be
able to be found. Those advertisements would then effectively provide
the missing information required to determine the communities in
which a user resides. </span>
</p>
<p lang="en-AU" style="font-weight: normal;">For more sensitive
environments, it would be possible to make “silent communities”,
where the community name is neither advertised nor included in the
bundles. To provide security for communities, it would then be
necessary to encrypt the payload of the service discovery bundle
using a key derived from the shared secret of the community. This
could be done by having the community membership attestment section
of the bundle also include a decryption key that is encrypted using a
key derived from the shared secret of the community. This would
prevent people outside of a community from being able to search the
service and resource offers and requests of the community, which is
probably a good step to take.
</p>
<p lang="en-AU" style="font-weight: normal;">This approach also has
the advantage that members of a community can choose to change the
shared secret at any point, without requiring any central
coordination. They could choose to advertise membership with both the
new and old shared secret for a period of time, to allow others time
to update their shared secrets. Or alternative when the goal is to
exclude an anti-social member, the old shared secret could be
immediately discarded, and any updates to the service discovery
bundles would render them unreadable by the excluded user (although
they could still read any older version of the bundles for which they
have access to the shared secret). This effectively provides a kind
of forward secrecy for the communities. It also helps to expire stale
members of the communities, who are not participating, helping with
the data trimming aspects of the system.</p>
<p lang="en-AU" style="font-weight: normal;">Finally, we need to
address the optimisation of bundles in the network, so that content
for a community is prioritised over content from outside the
communities that a device owner has joined. This will require
tweaking of the Rhizome Bundle synchronisation process, rather than
any changes to the data structures used. It thus falls outside the
scope of this project, although it is highly desirable to have this
implemented, as it will help with the scalability of Serval Mesh
networks greatly.</p>
<p lang="en-AU">So let’s now draw up a rough outline of what this
service discovery bundle would contain, so that we can follow their
contents for the example users in our previous scenario.</p><p lang="en-AU">The key information types are:</p><p lang="en-AU">1. Publication time and date of the bundle. This is used to indicate the beginning of validity of the contents of a bundle, so that a malicious user can't publish data that is "eternally valid", and thus clutters up search results forever.</p><p lang="en-AU">2. A record for each community that a user has joined, and in which their service discovery bundle should be considered valid. These may have individual periods of validity, but always within the hard-limit of 7 days from the publication date of the bundle. Thus to remain in a community, the service discovery bundle has to be updated at least once per week. Again, this is to stop stale data clogging the network and search results. The maximum number of communities that can be listed in a bundle should be restricted to a relatively small number, perhaps 5.<br /></p><p lang="en-AU">3. A record for each resource request, service request, resource offer or service offer, again with each entry having a validity period. These records should include both a textual description of the request or offer, as well allowing the specification of a number of emojis to provide a language independent description that can be used to support both low-literacy and through the use of an "emoji reader" vision impaired users. For low-literacy users it should be allowed to specify a request or offer that consists solely of the describing emojis. The emojis can also be used to provide a convenient visual description of a record for placement on a crowd-sourced map display. To support this, these records should also include a PlusCode location identifier, making it easy to draw maps containing this information.</p><p lang="en-AU">4. Records for the crowd-sourced mapping interface, that are not service or resource requests or offers, but rather situational-awareness markers, e.g., to indicate damaged roads or other infrastructure. These are otherwise similar to the request and offer records.</p><p lang="en-AU">A data model with just these few record types are sufficient to support the various use-cases explored in the scenario in section 2. These will be fleshed out in a later milestone, once we have assured ourselves that they really are sufficient.<br /></p>
<h2 class="western" lang="en-AU">3.2 User Story Work-Throughs</h2>
<p><style type="text/css">h2 { margin-top: 0.35cm; margin-bottom: 0.21cm; background: transparent; page-break-after: avoid }h2.western { font-family: "Liberation Sans", sans-serif; font-size: 16pt; font-weight: bold }h2.cjk { font-family: "Noto Sans CJK SC Regular"; font-size: 16pt; font-weight: bold }h2.ctl { font-family: "Noto Sans Devanagari"; font-size: 16pt; font-weight: bold }p { line-height: 115%; margin-bottom: 0.25cm; background: transparent }a:link { color: #000080; so-language: zxx; text-decoration: underline }</style></p><p>To verify that the data model is adequate, I went through the scenario from section 2, and mapped out the information that would be required in the service discovery bundle for the key users in the scenario. To keep this manageable, I have focussed only on the service discovery bundle, and excluded any Serval MeshMS or other communications. I have also left out the expiration and validity duration information, as this is logically fairly straight-forward, and is orthogonal to the search and discovery requirements.</p><p>The result is the following image, where we have the key contents of the service discovery bundle listed for each of the sub-sections of the scenario. New pieces of data that are added to a user's service discovery bundle are highlighted in yellow when they are first added. The image is pretty big, so it is shown smaller here. Click on it to open it in full-resolution:<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7Kpu7WDdXqv7cXmyZU7OZgBgAhHsZn6HT0mULyi4KEP8DrsYUN_YkGYlq2PMV0GypaULPcgDSQy6GMOdKG4godZXezz4n3I6Lok963e6PMlymObJcoQzqWdbgiqNvD_s77x0C7Zmpl9oMWD0T_RT5_SWbYYfQ0ztDEcxJXIqxHQ2EQi5j28J1oVmI/s7016/20220929-NLnet-Service-Discovery-Scenario-Data-Mapping-Walk-Through.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="4961" data-original-width="7016" height="283" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7Kpu7WDdXqv7cXmyZU7OZgBgAhHsZn6HT0mULyi4KEP8DrsYUN_YkGYlq2PMV0GypaULPcgDSQy6GMOdKG4godZXezz4n3I6Lok963e6PMlymObJcoQzqWdbgiqNvD_s77x0C7Zmpl9oMWD0T_RT5_SWbYYfQ0ztDEcxJXIqxHQ2EQi5j28J1oVmI/w400-h283/20220929-NLnet-Service-Discovery-Scenario-Data-Mapping-Walk-Through.png" width="400" /></a></div><p>Using this diagram, we can trace through each user's activity. For example, we can trace Niere connecting to the network in 2.3, and then offering freight services in 2.5. Where she then seeks diesel fuel, we can see the offers of fuel from a Saua community member in 2.5, and then later from Boli in 2.6.</p><p>This approach to representing the data purposely doesn't show the act of searching for information, as that doesn't result in the further injection of data into the network. Rather, we can verify that the search will succeed, by visually confirming that the data required for the search is available at that point in time. For example, that Boli's offer for fuel is published in his service discovery bundle at or before the time that Niere searches for fuel in 2.6. That is, we have taken the approach of showing the data that resides in the Serval Mesh network to facilitate the searches -- which we can thus see that it does. Implementing the search facilities will come in a later milestone.</p><p>So that all looks sensible, so now we just need to document this on the Serval Project Developer's website, <a href="https://developer.servalproject.org/dokuwiki/doku.php?id=content:servalmesh:searchanddiscovery">which I have just done</a>, and that's all the items from this first milestone complete:<br /></p><p style="background: #95be00; margin-bottom: 0.11cm; margin-top: 0.78cm; page-break-after: auto;">
<span style="color: white;"><span style="font-family: Liberation Sans;"><span style="font-size: medium;"><b>1.
Define Search and Discovery Services and Data Model</b></span></span></span></p>
<p class="western"><span style="font-family: Liberation Sans;">This sub-task focuses
on defining the user-facing search and discovery services that will
be created, and how they will be mapped to the underlying Serval Mesh
network primitives.</span></p>
<p align="center" style="background: #c0c0c0; margin-top: 0.1cm;"><span style="font-size: medium;"><b>Milestone(s)</b></span></p>
<ul><li><p class="western"><span style="font-family: Liberation Sans;">Identify the
precise search and discovery services to be provided, including from
both data provider and consumer perspectives.</span></p>
</li><li><p class="western"><span style="font-family: Liberation Sans;">Create several
user-stories for how the search and discovery systems might be used
in practice.</span></p>
</li><li><p class="western"><span style="font-family: Liberation Sans;">Map these
services to the Serval Mesh Rhizome Delay Tolerant Networking
Primitives, and work though several user-stories to confirm that the
model is fundamental workable.</span></p>
</li><li><p class="western"><span style="font-family: Liberation Sans;">This milestone
will be considered complete when the above elements have been
completed, culminating in worked-through user-stories and documented
data model, both of which shall be documented on the Serval Project
Developer’s website (which will be upgraded to HTTPS in the
process), as well as on the Serval Project blog
(<a class="western" href="https://servalproject.org/">https://servalproject.org</a>)</span></p>
</li></ul>
<p>
</p>
<p></p><p><br /></p><p><br /></p>Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com0tag:blogger.com,1999:blog-9068638994894102968.post-19782543376216205552021-06-16T17:43:00.001-07:002021-06-16T17:43:23.680-07:00Succinct Data in the Pacific for COVID19 Symptom Monitoring<p>Over the past year or so, we have been talking with some of our partners in the Pacific about helping Pacific Island Nations to improve their capacity to respond to the COVID19 pandemic -- but also to support them with their ability to manage the existing neglected tropical diseases in the region, including <a href="https://www2.health.vic.gov.au/public-health/infectious-diseases/disease-information-advice/dengue-fever">Dengue Fever</a> and <a href="https://www.cdc.gov/malaria/about/biology/index.html">Malaria</a>. </p><p>What these three diseases all have in common, is that tracking who has and doesn't have the disease (or symptoms of the disease) can be really helpful at stopping the chain of transmission. If this is all sounding strangely familiar in our post-COVID19 world, that's because it is quite similar to the trace and quarantine approach that Australia and NZ have taken with great success -- enabled by being relatively small island nations able to control their borders. </p><p>This approach doesn't work for all countries, however, e.g., with very large dense populations and porous borders. However, that is not the case for Pacific Island Nations: The natural advantages of Australia and NZ were able to leverage are greatly amplified for most Pacific Island Nations, as they have relatively few international passenger movements, a very wide moat called the Pacific Ocean, and even within most of these countries, large distances between adjacent islands and communities, that further enable natural quarantining, without great inconvenience.</p><p>Where the challenge lies for these nations is that their relatively small economies and lower levels of development and logistical adversities make it difficult to build these kind of public health tracking and situation awareness gathering tools.</p><p>We developed Succinct Data back in about 2012 with NZ Red Cross precisely to be able to provide this kind of functionality, and we now have a sprint of effort to attempt to resurrect that system, and get it operational for use in the Pacific by the end of 2021. The process of doing this will be documented in blog posts over coming weeks.<br /></p>Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com0tag:blogger.com,1999:blog-9068638994894102968.post-11491319519700162042020-09-28T15:16:00.005-07:002020-09-28T17:10:35.183-07:00900MHz 802.11ah WiFi modules<p> Hmmm... So it looks like there are the first 900MHz wifi modules, which claim 1 - 1.5km range outdoors, with data rates of megabits to tens of megabits in many circumstances. The module I have found is this one:</p><p><a href="https://www.silextechnology.com/connectivity-solutions/embedded-wireless/sx-newah">https://www.silextechnology.com/connectivity-solutions/embedded-wireless/sx-newah</a></p><p>The only problem I see so far, is that there is no comment as to whether they can do ad-hoc mode, or only AP or station mode.</p><p>If they can do ad-hoc mode, then this is potentially very interesting for use in Serval Mesh Extenders.</p><p>More when I discover it... But if anyone else has experience working with these, I would love to hear from you.</p><p>UPDATE: It looks like it uses an NRC7292 chipset for the 802.11ah, for which source code is available:</p><p><a href="https://github.com/newracom/nrc7292_sw_pkg">https://github.com/newracom/nrc7292_sw_pkg</a></p><p>They already have monitor mode implemented in the chip, it seems, so it looks like it *should* be possible to implement ad-hoc mode on it.</p>Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com0tag:blogger.com,1999:blog-9068638994894102968.post-74067698765098563302020-09-24T22:53:00.001-07:002020-09-24T22:53:30.590-07:00Chasing down the last LBARD HF TX queue bugs and Initial Performance Graphs<p>To recap, in the last blog post I found and fixed a lot of bugs in LBARD over HF data modem operation. This has taken much longer than hoped, in part due to the 4 to 5 hours it takes to do a complete run.</p><p>The last fixes I discussed in the previous post have resolved the remaining TX queue shuffling bugs, so far as I can see. But in some runs I am still seeing that a few bundles are not transferred in one or both directions. These are typically among the lowest priority bundles, but not strictly the lowest priority bundles.</p><p>For example, in the last test run, the following bundles are not transferred:</p><p><span style="font-family: courier;">peer4random79</span></p><p><span style="font-family: courier;">peer4random94<br /></span></p><p><span style="font-family: courier;">peer4random96</span></p><p>The files are named in increasing file size, from peer4random00 to peer4random99. Thus the 94 and 96 files are among the lowest priority bundles, and the 79 file is reasonably close.</p><p>The first step to tracking down what happened with them is to get the bundle IDs (BIDs) that correspond to these files:</p><p><span style="font-family: courier;">peer4random79 - 6CE2CB42* - bundle #21 on the sending side<br /></span></p><p><span style="font-family: courier;">peer4random94 - 469E2B18* - bundle #6 on the sending side<br /></span></p><p><span style="font-family: courier;">peer4random96 - 338FC557* - bundle #4 on the sending side<br /></span></p><p>Armed with that information, I can see if those bundles get transferred at all. Let's start with the 79 file.</p><p>At 17:53.27 the sender first realises that the recipient does not have the bundle. However, the TX queue is already full, so it doesn't make it onto the TX queue.</p><p>At 18:28.39 it gets queued again, and this time there is space in the TX queue, and it does get queued in position 3 in the TX queue.</p><p>At 18:28.39 bundle #66 gets queued, which has a higher priority, so our bundle is demoted to position 4 in the TX queue.</p><p>At 18:29.06 bundle #53 gets queued, which also has a higher prioity, so our bundle is now demoted to position 5 in the TX queue.</p><p>At 18:29.06 bundles #27, #71 and #56 also get queued, and our bundle is now in position 8.</p><p>At 18:29.29 bundles #51 and #55 get queued, and our bundle is now in position 10. One more higher prioity bundle, and it will get bumped from the TX queue altogether until the tree-sync process gets restarted. This happens immediately, as bundle #59 gets queued.</p><p>At 19:06.57 our target bundle, #21, gets requeued, however the TX queue is at that time already full of higher-priority bundles, so does not make it ont othe TX queue.</p><p>At 19:40.05 it does finally make it back onto the TX queue in position 8.</p><p>At 19:40.14 bundle #41 gets queued, bumping our bundle to position 9.</p><p>Then at 19:40.49 bundle #27 gets queued, bumping our bundle to position 10, followed immediately by bundle #23, which causes our bundle to get bumped from the TX queue.</p><p>Our bundle is next attempted to be requeued at 20:13.42, but the TX queue is already full of bundles with equal or higher priority, so it doesn't get queued.</p><p>The next attempt is at 20:43.25. And this looks like it might be the problematic instance. The TX queue is not full, and our bundle has a higher priority than the TX queue's entries, and also compared with the bundle currently being sent. As a result, it is immediately selected for transmission. However, then bundle #28 also gets queued for sending, which bumps #21 from being the bundle currently being sent. However, there is no code that then puts the bundle into the TX queue. </p><p>As the TX queue had not overflowed since the last restart of the tree-sync process, no flag is raised to require the tree-sync to restart one last time, and so this bundle will never finish being sent. And indeed this is what happens. Similar things happen for the other two bundles.<br /></p><p>So the fix is relatively simple: When de-queuing a bundle that should still get sent, we need to insert it into the TX queue. </p><p>I'm heartened that this fault seems to describe the last situation where this problem of bundles not being forwarded can occur.</p><p>The only immediate problem I can see, is that there is already code that is supposed to do this. It turned out the problem was that I attempted to re-queue the bundle being sent before removing it as the bundle being currently sent. The queuing logic was noticing this, and not adding the bundle back into the queue, because as far as it was concerned, the bundle was already being sent. </p><p>By reordering this sequence of events, I was able to solve the problem, and it now reliably delivers every single bundle. Together with all the performance fixes, we now have things at a state where they work reliably, which is great. So lets get some basic performance statistics from these runs:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivPJiu6hESr7Cl0UM09oJ3RloS_RVrpnHyuwmGevskzXTM781L0cb5bcdAlFdHprVtXN2xB1xcphX9HnYzFMC978BCWr2yMmTob6mCpEYstaJYnRZEMEVlmsX5XFSSrrR34oTrsSKnhS0/s640/lbard-bundle-speeds.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="480" data-original-width="640" height="363" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivPJiu6hESr7Cl0UM09oJ3RloS_RVrpnHyuwmGevskzXTM781L0cb5bcdAlFdHprVtXN2xB1xcphX9HnYzFMC978BCWr2yMmTob6mCpEYstaJYnRZEMEVlmsX5XFSSrrR34oTrsSKnhS0/w483-h363/lbard-bundle-speeds.png" width="483" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVVqsr5vGF0Uaab1jP2ppJkWujefCC5WdkGYGkI_QnSpTFvRy27hC4ih180W6NTFCbA-eu5WI6Q09XwNKON__w20_9y1pelThktjPv0ciZD8fw5HJ1Unmg5ufvrWr76kkgg7nDb6BqM1Q/s640/lbard-bundle-times.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="480" data-original-width="640" height="375" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVVqsr5vGF0Uaab1jP2ppJkWujefCC5WdkGYGkI_QnSpTFvRy27hC4ih180W6NTFCbA-eu5WI6Q09XwNKON__w20_9y1pelThktjPv0ciZD8fw5HJ1Unmg5ufvrWr76kkgg7nDb6BqM1Q/w500-h375/lbard-bundle-times.png" width="500" /></a></div><br /><p>We can see that there is considerable variation in the transmission times (and thus speed), with respect to the bundle sizes. That is, for bundles of about the same size, the delivery time can vary by a factor of 5 or so. </p><p>This is not unexpected, as the prioritisation of bundles during the tree-sync protocol can cause some bundles to be fully or partially transmitted, before tree-sync realises that there is a higher priority bundle. When this occurs, the bundle that pre-empts the lower-priority bundle will appear to take longer to deliver, because part of one or more other bundles will have occurred during the time accounted to the higher-priority bundle. Conversely, if a bundle has been partially transferred before being pre-empted by a higher priority one, the pre-empted bundle will appear to deliver more quickly, because only the time from when it resumes transmission will be accounted.</p><p>Otherwise, we see that the average data throughput, excluding the approximately 400 bytes of Bundle Manifest, improves gradually with increasing bundle size. For larger bundles we see transfer speeds of around 40 bytes per second, which is about half of the channel capacity of ~93 bytes per second we discovered previously. This is still not ideal, but given that it includes all the overheads for the transmissions to occur, to be prioritised and for synchronisation of bundle lists between the two parties, its not too bad. Note that these figures are for one direction only, with similar results for the opposite direction.<br /></p><p>If we look at the overall bundle transfer times, we see that this varies between about 50 and 200 seconds, only somewhat influenced by the size of the bundle's payload. This corresponds to message delivery rates between 2x (3600/50) = 144 and 2x (3600/200) = 36 bundles per hour over an HF link. That is, between ~850 and ~3,500 bundles per day. Again, while higher figures would be welcome, being able to deliver one to a handful of text messages very few minutes would seem to be sufficient for delivering a useful level of communications between small isolated communities. Round-trip message send and reply cycles would likely be of the order of 2 to 10 minutes, which again is sufficient to facilitate useful communications.</p><p>So at this point we have reliable and reasonably performant delivery of Serval Mesh traffic over Codan HF data modems. The next step will be to try to setup a real HF link with two radios spaced many kilometres, or perhaps a couple of hundred kilometres apart, and see how it performs in practice with some real message traffic.</p><p>Otherwise, I'll now be turning my attention to implementing LoRa radio support for Mesh Extenders to help make lower-cost and more internationally portable Mesh Extenders, so that we can experiment with more interesting mixed-radio networks. Because the Wi-Fi links between Mesh Extenders are self-organising, this will allow such mixed-media networks to be easily built. It will then be interesting to see how the system behaves with such networks.</p><p>After that, I will start looking at implementing "affinity tags" so that when we start connecting large numbers of such networks together, that we can prevent traffic being replicated places it doesn't need to go. We should also finally implement the logic for detecting whether senders or recipients are likely to be located on the far end of an HF link, so that we can also factor that into the prioritisation.<br /></p>Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com1tag:blogger.com,1999:blog-9068638994894102968.post-28471757720119629542020-09-21T22:28:00.000-07:002020-09-21T22:28:04.279-07:00Continuing working on the LBARD HF protocol efficiency problems<p>Ok, back from the last post where I was thinking about some longer-term future approaches, to trying to get the current code working as well as it can.</p><p>The problem I am seeing at the moment, is that the same blocks are being sent repeatedly, while other blocks are not being sent at all. </p><p>One problem was that pieces of the manifest were being marked as sent, when they hadn't. This was due to an out-by-one error in the comparison of the extent of a block being sent, compared with the actual data. The effect was when a block was sent, both it, and the following block were marked as having been sent.</p><p>This seems to be a common factor in many of the slow transfers. It has only started showing up now, because we were previously testing with payload-less bundles, and so the manifest would keep being resent, which would then have a higher chance of re-sending the missing part through the normal cyclic re-transmission.</p><p>So let's set a test run going, and see if it solves the problem.</p><p>Well, its helped, but close examination of the log files reveals that there are still pieces being counted against the wrong blocks, due to a duplicate code-path. I've now de-duplicated that. I also notice that sometimes manifest or body pieces would be counted against the other as having been sent, which of course is also causing problems. So I have fixed those as well. </p><p>On the plus side, its pretty how these sorts of bugs can be causing these problems: If we are marking things that have not been sent as having been sent, then we are not going to send those any time soon. Instead, we will keep sending the same pieces over and over, because we think we haven't sent them.</p><p>I'm also seeing the situation that the sending of a bundle can get reset part way through, which then makes it forget the transfer bitmap information for that bundle, resulting in it re-sending pieces it has already sent. This might be because an old feature that was built in that causes the tree-sync process to restart periodically, to avoid lock-up in the protocol, which in turn causes the selection of the currently being sent bundle to potentially change. However, the root causes for those lock-up have all been fixed now, so that can probably be removed, if it is still there. I'll investigate that while the new run with the previous bug fixes in place runs.<br /></p><p>Yes, it now looks like the resetting of sending bundles is the most significant remaining problem. As mentioned, this causes the knowledge of the state of a transfer to be lost. There are two approaches to solving this: </p><p>1. Don't switch which bundle we are transferring. </p><p>2. Remember the state of other bundles, so that we can continue from where we left off, if we do switch which bundle we are transferring.</p><p>Option 1 sounds great in theory, but the synchronisation process may well interrupt the transfer of a lower priority bundle with a higher priority one, for example. So this is probably not possible to achieve in practice. Although I will instrument the changing of the current bundle being sent, to find out why it is changing.</p><p>Thus it probably makes sense to implement something to remember the progress of a bundle when switching. This can be kept in a nice little cache somewhere. Of course, this is one of several complications that would go away if/when I get to implementing the new approach I wrote about in the previous blog post. But for now, the cache will have to do.</p><p>I've now implemented this caching of transfer progress. However, the first run after implementing in it showed a weird situation where bundles transferred quite regularly in one direction, but none in the other. In the direction where traffic was flowing, I saw 6 bundles received fairly quickly, with all of them transferring in 50 to 80 seconds. That's the good news. However, there would then be a gap ranging from a couple of minutes to half an hour with nothing being transferred. Then the same six bundles would be transferred again.</p><p>I think what happened is that one of the servald instances crashed or didn't start properly. As a result one party had no bundles to share, and could not store them into its Rhizome database, and so the same transfers happened repeatedly. That might have just been some freak event that caused one of the servald instances to not have started properly. To find out, I'll run the test again, and see what happens.</p><p>Second run worked properly in that regard. Some bundles transferring at a reasonable speed, but others takeing a VERY long time, upto an hour or so. </p><p>One bug spotted: We don't mark the final body block as sent if it is a partial block. This looks to be able to cause the final block to be sent many times repeatedly, which is clearly not good. I've tried to fix that now.</p><p>The other bigger problem, is that the bundle keeps getting de-queued and re-queued, which of course is causing a lot of delay in transfer. It also seems that at least one code path can de-queue the current bundle being sent, without stashing its progress for later retrieval. </p><p>Found at least one: The sending of a new generation ID by the sender, which causes us to restart the tree-sync process. In that code, we stop sending the current bundle. However, we should just keep on sending the same bundle, unless and until we find a better bundle to send. This is likely the cause of considerable delays, because the synchronisation will run for potentially several minutes before it finds a new bundle to send. That's likely to be it. We should probably reduce the generation ID rotation rate for HF, anyway, since it doesn't make sense to reset the tree-sync process so frequently on a high-latency link.</p><p>All these problems keep making me think about the new tree-based approach I wrote about recently, as there would be no need to do any of this, and resumption of transmission would happen in a much more natural and efficient manner -- even if the lbard process really did die and restart.</p><p>Anyway, back to the existing code, it now looks like transfers are happening reasonably sensibly. In trying to make a script to show the sequence of transfers, I am however still seeing that the currently selected bundle for transmission tends to change more often than it seems it should. This led me to look more deeply into the TX queue management, and I'm seeing some odd things there, where some bundles added to the queue seem to disappear, and others that were never reported as having been added to the queue turn up in there. So I'll add some more debugging for that.</p><p>Yes: The TX queue is being completely purged from time to time. This is again the resetting of the generation ID, the unique identifier that is used to indicate when an LBARD instance restarts. Apparently I was resetting the generation ID <i>every four minutes</i>. Thus it is no wonder that the TX queues are getting purged and we are ending up stuck in a synchronisation hell. Instead, the generation ID should only be changed when the TX queue at one end or the other has overflown, and thus been emptied, and so the sending side doesn't know which bundles it should have sent when the TX queue overflowed. </p><p>Given that the transfers are now otherwise working fairly efficiently, I'm hopeful that this will get us to some kind of reasonably reliable transfer with steady transfer times for bundles of a given size.</p><p>Running more tests, I have found and fixed a few more problems:</p><p>1. If the carrier is lost in a HF modem call, we can end up in a situation where no more normal packets get sent, presumably because the number of unacknowledged packets is high, causing the packet rate to be practically zero.</p><p>2. Particularly with bundles that require many data packets, they end up slowing down in transfer over time, and eventually transfer halts. The halting is probably due in part to (1), but the reducing transfer efficiency must have another cause.<br /></p><p>3. Some received pure data packets are corrupted, which is a probable significant contributor to (2).<br /></p><p>To fix (1), I have reset the TX sequence number and last acknowledged sequence number whenever a HF call starts.</p><p>For (2), I need to examine what is going on in the logs of runs, which will probably make more sense once I have fixed (3).</p><p>For (3), I suspect that the modem's TX buffer is being easily overwhelmed. This is in spite of the fact that I have hardware flow control enabled. It is possible the hardware flow control doesn't work properly with the USB serial adapters. In any case, I have implemented a check before sending pure data packets, so that they will not be sent if no normal packet has been seen in the last 30 seconds. This should stop the buffers getting too over full. But a fuller investigation will require that I log every byte sent to and received from each modem, and then compare the two streams to see exactly what kind of data errors we are experiencing. Is it lost bytes, as I am suspecting, or is it bit errors, for example?</p><p>So adding the logging of bytes sent/received, I can see that indeed something is going wrong with the pure data packets almost immediately. This makes it much easier to see what is going on, since I don't have to wait an hour or more each time. Now to look at the TX and RX logs to see what is happening...</p><p>We are indeed missing large slabs of data. <a href="https://www.ftdichip.com/Support/FAQs.htm#HwGen3">Taking a look at the FTDI USB serial adapter documentation</a>, it looks like you cannot safely write more than 32 bytes at a time, if using hardware flow-control, as that is the size of the FTDI chips internal buffer. So I've modified the serial writing code to write 16 bytes at a time, to see if that fixes the missing bytes problem. However the code to back-off due to hardware flow control doesn't seem to get triggered at all, and the problem with corrupt data continues. Found one problem with the TX and RX logs, where the pure data packets weren't being included. Fixed that, now lets see how it looks...</p><p>Ok, so now I am not seeing any more lost bytes at the start, but the pure data packets were still being munged. That problem turned out to be caused by the truncation of the data packets on the receive side. I also found and fixed a couple of segmentation fault bugs in the process. Now re-running...</p><p>Still seeing corrupted packets. It turns out the pure data packets were not being properly escaped before sending, thus if they contained a "!" character in their data or header, it was messing things up. Fix that now, too. This could well be causing the failure to transfer larger bundles efficiently, as it would effectively stuff up the transfer of larger pieces more often. So its satisfying to have found this problem. Let's see what it does to the transfer rates...</p><p>Yes, bundles now seem to transfer in between 30 and 200 seconds. However, now I am seeing that the same bundles get sent more than once. I suspect that this is due to the synctree reset that happens when the TX queue is emptied, but overflow of the TX queue was previously recorded. This tells LBARD that it needs to start the sync again. However, if the bundles were received, then they should not get transferred again. Looking in the logs, I am seeing HTTP 400 errors when trying to insert the bundles into the rhizome database. Thus we are still having a problem with corruption during transfer. I'll have to look at the reassembly of bundles. My suspicion is that the problem will be with the pure data packets somewhere.</p><p>The corruption problems have now been solved. Part of it was that writing in 16-byte pieces helps the serial over USB transfers, but messes up the HTTP transactions. So I have split the code to only do that for the serial transfers. That fixed the problem with bundles not making it into the Rhizome database, and transfers now proceed progressively through more than just the same 10 bundles repeating. </p><p>However, I am still seeing the HF call dropping out sometimes. Those tend to recover fairly quickly, and might be legitimate loss of carrier in the radio setup.</p><p>More concerning though, is that there can be a period of half an hour or more where no transfers in either direction occur. These are quite mysterious, as the HF modem doesn't seem to drop out, but the logs of both sender and receiver simply show nothing at all for a period of 46 minutes in this particular case. I guess I will need to run it again, and try to catch when this happens. </p><p>This time we hit a segfault after almost an hour and transferring 80 bundles (between both directions) in about 64 minutes. The problem is that sometimes rubbish ends up in the TX queue. More specifically, the priority of a bundle ends up being stored in the bundle ID field of a slot in the TX queue. </p><p>So I've put more debug instrumentation in to find out how the TX queue list is getting corrupted. In the process, I also fixed the case where if a bundle is currently being sent, it could still end up being queued for later transmission as well, e.g., if the tree-sync process is restarted at some point. </p><p>I've also added code that detects when the top of the TX queue ends up containing an illegal bundle number to reduce the impact. However, I would really like to fix the error at its cause. What I believe is most likely the problem, is that the shuffling of the TX queue has an out-by-one error somewhere, as the bundle numbers occur immediately before the list of priorities. Thus if one too many entries were copied down, it would copy a bundle priority. <br /></p><p>So time to start a new test run...</p><p>Its now running through a lot happier. 157 of the ~220 bundles have transferred, but it has taken about 4 hours to get that far. But it's not as bad as it sounds, as it seems that my laptop somehow stops USB communications while the display is locked. This means that if I don't periodically prod the computer, long periods of no transfers result. I'm pretty sure that this is the cause of this problem that I noticed previously.<br /></p><p>That said, I am also seeing it able to get into a hang up and redial loop, where no packets get sent in the meantime. I assume that the time to next packet TX is somehow being set wrongly. Or more the point, it can grow large, and wasn't being reset when a call was re-established. I've examined all the code-paths that can get there, and added explicit resetting of the next packet TX time.</p><p>I tracked down the "nothing sent for an hour or more" problem: It's actually not with LBARD, but with the laptop I am using. It automatically suspends after 2 hours of inaction, even when connected to mains power. This doesn't really help when it happens mid-run. So I've disabled that in the laptop, and should now be able to run transfers without this problem. <br /></p><p>I've also looked through and found the problem that was causing the illegal bundle numbers ending up in the TX queue, and fixed that. The problem was indeed an out-by-one with the list shuffling code, as expected. I'm expecting that this will likely fix the problem where only 199 or so of the 202 bundles were being transferred, as it could result in the situation where the TX queue loses an entry or two without actually overflowing to trigger the re-start of the sync process to re-discover the bundles that were lost from the queue. Thus I'm hoping that runs now will correctly transfer all 202 bundles -- 101 in each direction. It might also end up being marginally faster, because fewer sync restarts will be required.</p><p>In short, I'm now hoping that I am at the point where transfers run reliably and in reasonable time. I'll do a few runs to get some data on the typical speed and variation, but then we should be good, I think. What I would like to do next is to plan some actual long-distance tests using the HF radios, with one here at the house in Akaroola and another some distance away. Perhaps in a neighbouring town. However to do that, I will need to get some antennae and coordinate with folks in the amateur radio community to help with that.<br /><br />But as this post has already gone on for ages, I'll stop here, and start a new post looking at the remaining TX issues.<br /></p>Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com0tag:blogger.com,1999:blog-9068638994894102968.post-22595590000965511312020-09-08T16:33:00.001-07:002020-09-08T16:33:45.805-07:00Thinking about different ways to do the tree-sync for LBARD<p>In the last few blog posts I've been fighting my way through fixing bugs with LBARD's synchronisation algorithms. While both the tree-sync and the actual transfer of bundles have been problematic, the tree-sync side of things has been niggling at me for HF use, in particular. One of the reasons is that tree-sync has to start its synchronisation again from scratch every time a radio connection is made. This already takes several minutes, with only relatively small numbers of bundles. Even if neither end has new bundles, the entire synchronisation process has to occur again. Also the very high latency of the HF radios doesn't really agree with tree-sync, and can result in considerable delays in synchronisation.<br /></p><p>Way back around 2014 we had started thinking about reworking the way that Rhizome stores data, by having it use a block-store to represent the bundles. That is, instead of keeping a manifest and payload, we would have a kind of virtual file-system in which the bundles would be represented. Data blocks would be identified by the hash of their contents, thus providing automatic de-duplication, and making it much easier to cache data.</p><p>Now, reworking all of Rhizome to do that would require significant effort. However, what *might* be doable in reasonable time, is to make the list of bundles an instance holds be represented in such a way. This would allow each end of an HF link to cache this virtual file-system representation, and thus re-synchronise very rapidly if none or few new bundles have appeared at each end.</p><p>My current thinking is along the following lines:</p><p>1. Blocks should be small enough to fit in a single LBARD packet. This realistically means a limit of about 200 bytes.</p><p>2. We want to minimise the number of nodes in the tree, as each additional node requires an extra round-trip.<br /></p><p>3. The small block size means that we need to use relatively small hashes for each block. Even if they are too short for strong cryptographic security, they should still be strong enough to keep the chance of accidental collisions practically zero.<br /></p><p>4. Each block can consist simply of leaf nodes or pointers to child nodes, or a mix of the two. The position of the node in the tree is implicit.</p><p>5. It would be nice to have one hex digit of identifier covered per block. This means 16 pointers to sub-blocks have to be able to fit in a block. 200 bytes / 16 = 12.5 bytes per entry. This probably means that 12 byte hashes = 96 bit hashes will be the longest possible. That's ample to keep us safe from Birthday Paradox accidental node collisions (should be safe for up to 2^48 blocks).</p><p>6. If all the child nodes for a block can fit in a block, then that is where they are placed. This avoids having unnecessary intermediate nodes in the tree that would otherwise have to be synchronised.</p><p>7. If all the child nodes can't fit in a single block, then we need to have a deterministic way to represent this, so that we don't end up with a proliferation of blocks as we fill nodes.</p><p>8. The simplest approach to (7), is to require that an overfull block child blocks for each of the 16 entries it can hold. This may mean that some child blocks point to the empty node (which because of the hashing, will always be the same).</p><p>9. The above formulation means that each bundle (or more correctly, version of bundle), will have a deterministic node, which each side can enter into their block cache, thus avoiding the need to ever transfer a block that corresponds to a bundle that the other party already has.</p><p>10. Each side can request blocks from the other, to begin exploring the tree structure. Any common sub-trees between the two sides will be implicitly available.</p><p>11. The block cache can persist between executions of LBARD, thus allowing the complete state of the other party to be remembered between radio sessions, thus greatly speeding up the initial synchronisation when a radio session commences.</p><p>12. Multiple blocks can be requested at a time, thus helping to mask high latency. In fact, simple optimisations like "please send me block X, plus any children (and optionally further descendents) of it that you don't think that I have" can be used to save further round-trips.</p><p>13. When a block is received, its position in the tree is known. This means that in some cases, it will be possible for the receipient to work out the content of the child nodes, if the sender has a strict sub-set of the sender's bundles that are in that part of the tree. At its simplest, a leaf node can be constructed for every combination of the bundles that it contains. If any of those match the block ID of the child node indicated in the received block, then there is no need to receive those child-blocks. Instead, not only can we avoid the need to send those blocks, we can actually immediately infer that the sender is missing those bundles not in the combination that resulted in the matching child node. </p><p>14. The approach from (13) can be extended further up the tree if desired, to help save bandwidth, at the expense of storage space and computation. At some point, however, the number of combinations that need to be tested becomes excessive, as each additional bundle in the portion of the tree being considered potentially doubles the number of combinations to be computed. However, because each leaf node is restricted in its potential content, the growth in complexity is actually rather different (leaf nodes are only formed when the collective leaves don't fit in a single block). The exact complexity of this will need to be modelled.</p><p>15. In theory, if the complexity of the above can be handled, then sending just the top of the tree will allow the recipient of any node that contains only a subset of the nodes the recipient possesses, without any further data exchange. This is more of theoretical interest than practical effect, except for the leaf-node optimisation described in (13).</p><p>16. The bundles themselves can also be represented as tree structures, so that similar techniques can be used. This also allows for implicit transfer of common data sections, including in journal bundles that have invariant starts. Of course, this ability to transfer common data sections will only work for those that have such commonalities that are aligned with the block size, or better, aligned with a leaf-node of blocks.</p><p>17. This approach allows for implicit caching of all transfers, making resumption of communications with other nodes and of bundles much more efficient.</p><p>18. This approach also solves many of the authenticity issues with transfers in LBARD, where the transfer of a large bundle could be poisoned with a fake block, or even just by a bit error that escaped the error correcting code. This is a significant advantage of this approach.<br /></p><p>19. This approach effectively trades-off storage space (which is cheap and plentiful) for algorithm complexity and RAM requirements (which is in limited supply on the kinds of small embedded devices we want to target, such as low-power meshing phones etc). This is thus a very nice side-effect of this approach.</p><p>Anyway, I'll have a think about whether I implement this now or not, depending on how feasible it looks to get acceptable performance with the existing algorithm.<br /></p>Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com0tag:blogger.com,1999:blog-9068638994894102968.post-49018863403122103822020-09-07T20:37:00.002-07:002020-09-07T20:37:19.048-07:00More work on LBARD over Codan HF Data Modem<div>In the last post, we got to the point where we had Rhizome bundles flowing over the modem, but not as fast as we would like. Also, we were still seeing some problems where bundle sending would get stuck. I've been away for a couple of weeks, but now am back at the bench with the HF radios, and can investigate further.</div><div><br /></div><div>The first thing I realised that I should investigate, is what the actual throughput of the modems is, so that I can determine what fraction of the theoretical throughput we have available.</div><div><br /></div><div>To do this, I instrumented LBARD to count the number of bytes, and divide that by the number of seconds since the first byte was received. This yielded results of about 91 bytes per second in each direction. Given the channel theoretical capacity of 2,400 bits / sec = 300 bytes / sec, this is noticeably lower than expected. It means that we are only actually transmitting data (91+91)/300 = ~60% of the time. <br /></div><div><br /></div><div>Now, if we were just transferring in one direction, instead of switching all the time, we could surely improve that, but we know that there is less than a factor of 2 available through such optimisation. So I'm not too worried about that right now.</div><div><br /></div><div>What is more interesting is that we should be able to get near to 90 bytes per second, but are still languishing down around the 20 bytes per second mark due to protocol issues. That is, we have a factor of 4 that we can improve, without having to increase the channel capacity or use strategy at all.</div><div><br /></div><div>Having spoken with one of my students who is working on a debug framework for LBARD, it is sounding to me like the packet bitmap that is used for acknowledgement and scheduling of parts of a bundle that the receiver doesn't already have is not working properly. If this is the case, it would be causing the sending side to send parts of the bundle that the receiver already has, effectively wasting bandwidth. <br /></div><div><br /></div><div>Looking through logs of previous runs, I can see one problem already: We schedule sending a bundle at the end of the file, even if we have already sent to the end of the file. That is, we send 0 bytes of data, positioned at the end of the bundle payload. This is clearly silly, and should be fixed. What we need to watch out for, though, is that we use end-of-payload packets to implicitly indicate the payload length in LBARD transfers. <br /></div><div><br /></div><div>We are also seeing the 4 data bytes being sent quite a number of times, before the missing parts of the manifest are sent through. That resending needs to be fixed. But it also occurred to me, that if we randomly select a point in the last 64 bytes of the manifest or payload, when we will only send <64 bytes, when typically we have space in the packet for 128 or 192 bytes. I suspect that this has the potential to greatly improve latency for small bundles, by basically eliminating such very low efficiency packets. Together with preventing the re-transmissions mentioned in the last paragraph, this should be able to get the latency of a sub-1KB bundle down to perhaps 15 seconds, which would be great improvement.</div><div><br /></div><div>I'll try these improvements, together with some tweaking of debug output, so that I can more easily follow when particular bundle pieces are sent, and what the list of candidate blocks at a given packet transmission are.<br /></div><div><br /></div><div>This should have been a simple-rerun, but then I hit a segfault caused by something unrelated, and now running it in gdb all sorts of strange things are happening, causing one of the instances to not properly switch to the online data mode when the modems connect. Well, I found what caused the whole crazy problem to start: When I copy-pasted the arguments into gdb, I left the "1" off the front of "127.0.0.1", thus causing long delays while waiting for "27.0.0.1" to resolve, which caused the protocol to time out in various exciting ways. So let's try that all again...</div><div><br /></div><div>Ok. Now it is running again, after fixing that. I also found and fixed some other bugs. It is now transferring small bundles in typically 20 - 50 seconds. This could be quicker, but it is a start. However, I am seeing instances where a bundle is received, but then fails to insert into the Rhizome database with an error 422. This means that either the manifest is too big, or that the payload doesn't match the hash. The bundle is then received a second time, and gets inserted. This means that there is some problem with the integrity of the transfer. <br /></div><div><br /></div><div>My first suspicion was that the pure data packets I implemented recently might be the problem. But in this case, no such packets had been sent at all for the bundle in question. As the packets are in any case protected by Reed-Solomon error correction, this would seem to suggest that the contents of the packets were being incorrectly assembled.</div><div><br /></div><div>Actually, it turns out that this happens fairly often. This is good, in that it probably explains a lot of the poor transfer performance: Some bundles are having to be sent two or more times, before actually being accepted. So now to find out why.</div><div><br /></div><div>It looks like decompression of the bundle manifests fails, resulting in the end of the manifest being messed up, and not passing its cryptographic self-consistency checks. It is the end of the manifest that gets messed up, so I am guessing that somehow another, either the length of the manifest, or the last partial 64-byte block of data for the manifest doesn't get properly received. It should hopefully not be too hard to find, as it is happening quite frequently, with probably 20% of bundle transfers being affected by it.</div><div><br /></div><div>To track it down, I'll make both sender and recipient parties display the compressed manifest data, so that I can play spot-the-differences.</div><div><br /></div><div>So looking at the compressed manifest on reception, it looks to be the right size, but has all zeros in the last data blocks. The problem was that when I made the optimisation to send not just the last fractional block, but as many 64-byte blocks that come before it that can fit, I hadn't adjusted the start point of the packet data being read, thus it was missing the extra bytes. So time for another run, to see how it goes.</div><div><br /></div><div>Right. So now we don't seem to have the problem with the code 422s, which is good. However, we have the problem that sometimes we keep receiving a bundle repeatedly after receiving and registering it. This seems to be because we don't get a confirmation message from servald that the bundle is now in our rhizome database. As a result, we don't cull the duplicate reception of the bundle. It also means that we can't immediately start sending the bundle to other peers. <br /></div><div><br /></div><div>This isn't such an issue for the HF modem driver, because we don't have other peers to talk to. However, for Mesh Extenders with the UHF packet radios, this is a big problem, as bundles won't get forwarded on to other peers until this occurs. Thus we need to find and fix this problem.</div><div><br /></div><div>LBARD communicates with servald via servald's HTTP RESTful interface. In particular, it listens for new bundles via an asynchronous API, which means that we should get indication from servald essentially immediately on inserting the bundle. So I presume something is fishy in that. To investigate this, I have instrumented the code that communicates with servald, to find out where the problem is.</div><div><br /></div><div>While looking at the logs for these delayed bundle notifications, I am also seeing that some bundle transfers are slow, as one end is not sending any data packets at all to the other end for 10 -- 20 seconds at a time. This should never happen, especially once the initial synchronisation has occurred, that lets each end know what the other needs. In the current case, I am seeing a delay of 24 seconds between when successive pieces of a bundle are sent. So we know that the remote end thinks it has a bundle that is worth sending.</div><div><br /></div><div>I'll start by investigating one of these intervals, from 17:51.02 to 17:51.26.</div><div>During that time, we do receive several packets from the sender at 17:51.10, 17:51.12, 17:51.15, and 17:51.21. That is, we have received four consecutive packets that seem to contain no pieces of bundles. Okay, so it seems that the synchronisation process is continuing to run, and is sending packets of ~200 bytes of synchronisation information. <br /></div><div><br /></div><div>Thus these delays are a false alarm, caused by the natural synchronisation process. What is a bit worrying though, is that the synchronisation procedure is taking a long time. With 100 unique bundles on each side, synchronisation is taking at least tens of minutes. In this example, I am seeing of the order of 300 sync messages of close to 200 bytes each -- and it shows no sign of settling down.</div><div>This amounts to somewhere around 68 KB of data out of a total of 84KB sent.</div><div><br /></div><div>Well, that's very interesting: It explains why our goodput is so low, and generally, why transfers are going so slowly.</div><div><br /></div><div>Thus together with the Rhizome bundle notification delay, we have to bugs that are able to greatly slow things down. I now have something quite definite to concentrate on. <br /></div><div><br /></div><div>I'll still work on the Rhizome problem first, because I am much more familiar with that code. <br /></div><div><br /></div><div>The synctree code is going to be trickier to fix, because I didn't write that. What I suspect might be the issue with it, though, is that the sync tree code doesn't expect to be used in an environment where multiple packets get queued up. This means that when it sees a response to and old packet of its own, it might be getting confused, and starting the tree traversal from higher up the tree again.<br /></div><div><br /></div><div>Thus the potential fix for this is to find a way to make sure we have only one packet in flight in each direction at a time. This would be a good idea in any case, because it would help to minimise the latency when a bundle is received, so that the sending party doesn't keep needlessly sending parts of that bundle after it has been received.</div><div><br /></div><div>Well, actually I am going to fix this one-packet-in-flight issue first. But first, a brief explanation as to why we do this syncing this way in the first instance, rather than, say, just exchanging lists of bundles, and then sending the bundles that the other side needs to get -- because that would surely be more efficient.</div><div><br /></div><div>The reason is that a simple "exchange lists" and then bundles method would be fine if no new bundles ever arrive during a communications session. But we want to be able to deliver messages with reasonable latency, even if they arrive while a connection is active. That is, we would like to have live HF links between Serval Mesh islands, and have those act as near-real-time data conduits.<br /></div><div><br /></div><div>Now, back to the need for the sync algorithm to not be several packets behind, I have implemented this, and it seems to be working somewhat: The sync traffic is flowing, and is no longer filling the packets. However, the rate of packet exchange is VERY slow. The average raw data rate in each direction is only about 8 bytes per second. It looks like each end thinks that it has 2 unacked packets nearly the whole time, which is causing it to throttle back the packet rate. I am suspecting that the low data rate is interacting in a bad way with the modems trying to be efficient, and waiting until there is enough data to be bothered sending.</div><div><br /></div><div>But back to thinking about having only one packet in flight in each direction at a time, this means we will always end up with some wasted time, because one end or the other will almost certainly be waiting most of the time. Investigating further, the round trip time for a packet turns out to be around 15 seconds (!!).</div><div>Even allowing for two maximum size packets of 255 bytes, the propagation time would still only be (255x2 bytes)/(2400 bits per second) = 1.7 seconds. This makes me all the more suspicious about the modems trying to batch data together for efficiency, which in other cases would work.<br /></div><div><br /></div><div>So we need a solution to this problem. The previous work I did on inserting the occasional pure data packet might actually be a good solution here: Basically it means that we have a background level of sync exchange through the existing packets, and then we greatly accelerate the exchange of bundles via the pure data packets, with the added benefit that they should help to flush the modem buffers.</div><div> <br /></div><div>I'm now reading through the manual for the modems, with a view of disabling compression, in case the compression is causing increased latency. In the process, on page 6-2 of the "HF Data Modem 3012 Reference Manual", I have found some helpful information: While the data channel can in theory carry upto 2,400 bits per second, the maximum effective throughput is only 1,475 bits per second, i.e., 184.375 bytes per second. This explains why I was seeing data rates of about 92 bytes in each direction. While the extra bandwidth would have been nice, it does at least give me a clearer idea of what the maximum realistic capacity is.</div><div><br /></div>On page A-3, I also found *T, which allows setting the timeout before sending data packets. This is important, because any non-zero value will increase the latency of packet transmission. This setting can be set from 0ms to 120000ms. The default is apparently 0ms. I'll set it explicitly in any case, just to make sure... Except the modem complains about the command. It is oddly listed as only *T, not AT*T, but *T also does not work. I tried *0, in case the T is supposed to be the value itself.<br /><div><br /></div><div>Also AT&M=5 is supposed to select the highest data rate and optimise for interactivity, i.e., reduced latency. That was already set.</div><div><br /></div><div>Doing some more direct tests with the modems, I timed how long it takes after I paste a 62 byte string into one modem, before it appears at the other end. This reliably takes 7 to 8 seconds, thus confirming this latency problem.<br /><br />The issue we have, is that our synchronisation protocol currently does not support having multiple packets in flight, as it was designed with radios in mind that can reliably indicate when they are ready to send a packet, and then send the packet immediately, thus allowing each packet to be build just-in-time, to ensure it has the most appropriate content in it. <br /><br />In fact, just pressing a letter and waiting for it to appear at the end has the same latency of 7 - 10 seconds, so it isn't a bandwidth thing. Rather, it seems that the modems make at least one full turn-around each, before the data gets sent. Is this normal?<br /></div><div><br /></div><div><div>I did also find the command to disable data compression, AT%C0, and
I am now testing that to see if it makes any noticeable difference.<br /></div><div>(Another useful command I have found is AT&C=n,n,n,n,n, which allows providing the modem with a list of channels on which to listen for incoming calls, which I'll use later, and record here so that I don't forget about it ;).<br /></div></div><div><br /></div><div>But back to AT%C: Disabling data compression DOES make a difference: It drops minimum latency from around 7 - 10 seconds, to only about 4 seconds. That's a big help, and should greatly improve our throughput. With 4 seconds in each direction, and sending packets of up to 232 bytes, this should get us a data rate of up to 58 bytes per second of the theoretical 92 bytes per second available. <br /></div><div><br /></div><div>Thus, while it is a good step forward, it isn't enough to completely solve the problem. We still will need to send some pure data packets to make full use of the bandwidth. Part of me also wonders if disabling data compression might help with the buffer overrun problem we were having previously, that was limiting the size of the data packets we could send between normal LBARD packets. <br /></div><div><br /></div><div>But before we make further changes, we should time how long it is taking to send small bundles (without payloads in this case), so that we have a base line to compare against. Having ~4 second latency in each direction instead of ~8 seconds, means that we will detect the end of bundle transmission on average 8 seconds sooner, which should provide a noticeable improvement.<br /></div><div><br /></div><div>Sending only the normal LBARD packets, without any extra data packets, and it fails, because of the previously reported problem with payload-less bundles. I've now put a fix in for that, and that helps.</div><div><br /></div><div>I'm also seeing another problem which is when the receiver tries to send a Bundle Announce Response (BAR) to indicate to the sender that it has received the bundle, we hit a problem if the packet contains another piece of the same bundle. That 2nd piece causes the receiver to emit an ack for that piece, which replaces the BAR in the transmission queue. As a result the sender never hears that the sender has the bundle, and keeps sending pieces of it. This is likely a key contributor to the highly variable receive times we have been seeing for bundles.</div><div><br /></div><div>There are two fixes for this that should both be done:</div><div>1. ACKs should never replace BARs in the progress report TX queue.</div><div>2. We should simply ignore pieces of bundles we have marked as recently received, and never generate the ACKs for them, but instead produce BARs for them, to indicate that we have already received that bundle.</div><div><br /></div><div>With that fixed, along with some other tidy-ups, we are now ready to do the comparison again. We will count the number of zero-length payload bundles (which thus involve sending about 200 -- 220 bytes of compress bundle manifest, and all the house-keeping) that we receive in 1,000 seconds, and the total bytes transferred in that time, so that we can calculate the average bundle transfer time, and the goodput. <br /></div><div><br /></div><div>Again, we will do this with no extra data packets, then with a 128 byte data packet after each normal LBARD packet, and then again with sending a data packet every 2 seconds, independently from the LBARD packets. We will average for both directions over our test between the two radios.</div><div><br /></div><div>With 0 byte payloads, i.e., just manifests:<br /></div><div><br /></div><div><b>No data packets:</b></div><div>A -> B : 16.66 bytes per second, 13 bundles transferred<br /></div><div>B -> A : 16.93 bytes per second, 13 bundles transferred<br /></div><div><b>Sending 1 x 128 byte data packet after each normal LBARD packet:</b></div><div>A -> B : 25.74 bytes per second, 19 bundles transferred</div><div>B -> A : 25.26 bytes per second, 21 bundles transferred</div><div><div><b>Sending 1 x 256 byte data packet after each normal LBARD packet:</b></div><div>A -> B: 33.22 bytes per second, 26 bundles transferred<br /></div><div>B -> A: 33.06 bytes per second, 33 bundles transferred<br /></div><b><br /></b></div><div><b>Sending 1 x 256 byte (including headers) data packet every 2 seconds, as well as normal LBARD packets:<br /></b>A -> B : Modems hang up after a while, possibly due to data overrun, as bytes per second climbs to ~90 bytes per second.<br /></div><div>B -> A : Modems hang up after a while, possibly due to data overrun, as bytes per second climbs to ~90 bytes per second.</div><div>(this makes sense<br /></div><div><b>Sending 1 x 200 byte (including headers) data packet every 3 seconds, as well as normal LBARD packets:<br /></b></div><div>A -> B: 48.13 bytes per second, 29 bundles transferred<br /></div><div>B -> A: 48.18 bytes per second, 26 bundles transferred</div><b>Sending 1 x 256 byte (including headers) data packet every 3 seconds, as well as normal LBARD packets:<br /></b><div>A -> B: 78.37 bytes per second, 23 bundles transferred<br /></div><div>B -> A: 80.62 bytes per second, 21 bundles transferred</div><div><div><b>Sending 1 x 200 byte (including headers) data packet every 2 seconds, as well as normal LBARD packets:</b></div><div>A -> B: 68.59 bytes per second, 23 bundles transferred</div><div>B -> A: 68.86 bytes per second, 23 bundles transferred<br /><b></b></div></div><div><br /></div><div>Note that just because we allow a 256 byte data packet, doesn't mean that one will get sent: The actual packet sent might be quite a bit smaller, if there isn't enough data for the bundle currently being sent to stuff in.</div><div><br /></div><div>We also see that in some cases, the bytes per second is higher, while the number of bundles transferred does not increase, or in fact goes down. This will most likely be due to the re-transmission of the same data blocks over and over, while waiting for acknowledgement of reception of the whole bundle. Indeed, switching from 256 byte to 200 byte data packets with the same 3 second interval, we see no decrease in bundles transferred, although the number of bytes sent per second reduces.<br /></div><div><br /></div><div>Once we move to bundles with non-zero-length payloads, this effect should reverse. Another effect that is likely at play, is that if we send lots of data blocks between the LBARD packets, then we will increase the rount-trip-time of the LBARD packets, which will slow acknowledgement of bundles, and thus reduce the number of bundles that can be received per unit time.<br /></div><div><br /></div><div>Another factor that must be taken into account here, is that the first bundle transfer does not typically take place until around 120 -- 180 seconds into the 1,000 second test. Thus once actively communicating, the actual time per bundle will be lower than suggested here.</div><div><br /></div><div>So now to repeat those tests with non-zero payloads for the bundles... Which showed up a bug in payload handling, which was quick to fix. More critically, it showed up an issue where receivers sometimes don't recognise packets as being addressed to them. This means that when the other party sends them pieces of a bundle that they have already received, they don't tell the sender that they already have it.<br /></div><div><br /></div><div>The mechanism that is playing up stuffs each message containing a piece of a bundle with the first two bytes of the SID of the receiver. This works fine for some time, but after a while, the receiver finds garbage in this field. What is not clear is whether it is a data handling bug, that corrupts it on the receive side or during assembly on the transmit side, or whether the field that contains a peer's SID on the sender side is getting corrupted. This takes of the order of 15 minutes to occur.<br /></div><div><br /></div><div>LBARD maintains both ASCII-hex and binary representations of each peer. If these end up with different values, then we can assume that memory corruption has occurred. I'll add instrumentation that will help me to see if that is what is going on.<br /></div><div><br /></div><div>Interestingly, the data that overwrites the binary representation seems to often begin with $47, suggesting that some piece of code might be putting stuff there by mistake. The "Generation ID" message type does put a $47 followed by 4 random bytes. This message type also contains no bounds checking -- it just assumes it has been given a buffer with enough space in it. However, it is only called at the start of the packet assembly process, when it can be safely assumed that there is enough space in the buffer to fit the message.</div><div><br /></div><div>A few more runs confirms that it is always $47 in the first byte, so almost certainly something screwing up with the Generation ID messages. I'll add some more instrumentation to see what the new generation ID is each time one is received, and if it matches what I am seeing in the packet, then I will know that its to blame.</div><div><br /></div><div>Ok, while waiting for the run to hit the problem again, I've taken a closer look at the code that parses the generation ID messages, and discovered that it does indeed put the generation ID into the SID prefix field. I.e., a straight-forward bug.</div><div><br /></div><div>This fixed the problem, and got everything going, but it still seems to be rather slower than it should be sometimes. But investigating the remaining problems can go into the next blog post.<br /></div>Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com0tag:blogger.com,1999:blog-9068638994894102968.post-27406385958235848622020-07-19T15:33:00.000-07:002020-07-19T15:33:09.630-07:00LBARD over Codan HF modem and debugging LBARD sync problems<div>This week I am finally getting to working on the remaining work for supporting the Codan 3012 HF data modem in LBARD. This was started a long time ago, but has stalled at various points along the way. <br /></div><div><br /></div><div>The purpose of using this data modem is that it comes as a software option in the Codan Envoy 2210 HF radios, and can in theory transfer close to 100x more data than the horrible ALE text message method that we had demonstrated a few years ago. That said, we are still talking about a data link that <i>in theory</i> can do 2,400bits per second. I say in theory, because that relies on good atmospheric conditions for the HF communications, and because the link is actually half-duplex with quite a noticeable turn-around time. That is, the modem at one end sends a blob of data, stops transmitting, and begins listening. The modem at the other end receives the blob of data, switches to transmit, transmits its blob of data, stops transmitting and begins listening. This is a good thing, but not so helpful for our data throughput. Thus it takes perhaps a tenth of a second or more to do the turn-around. Because this is all happening with radios that can transmit at 125W, it
involves relays and various safe-guards in the TX to RX turn-around. <br /></div><div><br /></div><div>The modems seem to send a data blob lasting 2 seconds. So in every ~(2 + 0.1 + 2 + 0.1) seconds we get to send one blob of data. Thus while the channel can carry 2,400 bits per second, we are limited to something below 1,200 bits per second from each end. We'll estimate this to be about 1,000 bits per second for ease of calculation. This should give us an effective data rate of a bout 125 bytes per second in each direction.</div><div><br /></div><div>This is pretty modest by modern internet standards, but it does correlate to about one text message per second. Of course, by the time we have overhead for routing, authentication and other house-keeping elements, a text message might be closer to 1KB, which would require about 8 seconds to send under ideal conditions. This means that a link that is active 24/7 could send <i>and</i> receive about 86,400 seconds/day / 8 seconds = ~10,000 messages per day. Even with just an hour a day this means that we should be able to send and receive about 400 messages. Given our goal is to connect small isolated island communities in the Pacific, and to provide backup text messaging capability during disasters, this sounds like a pretty reasonable capacity -- and certainly better than the 10 to 15 minutes it was taking to send an authenticated text message via the ALE text messaging channel!</div><div><br /></div><div>The radio setup I have here is a pair of Codan 2210s kindly loaned to us by Codan themselves. I have these hooked up to a laptop here via USB serial adapters.</div><div><br /></div><div>The first step is to enable the data modems on the Envoy radios. If you have obtained these radios without enabling this feature, you will need to give Codan a call to purchase an unlock code. You shouldn't need any hardware changes, which is handy, if you are already in the middle of nowhere, needing to setup communications.</div><div><br /></div><div>Next, use the handset to select "Menu", "User data", "Peripherals" and "RFU GP port", and select "2.4kbps data modem". It is important to NOT select "3012 mode", unless you really are using an external modem, rather than the built-in one. The menus are shown in the following figures:</div><div><br /></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgoBX6bOGmJ9cs74C4-2lITIKij8B5I0jvXa2RadT27CElFHFSKNLh8umsnf2rqTg2jsC15-RWfV_ph5_mN9-BIFxqxLLNEeTQMRMu2GHU0k9adxbbKlEQedU3I957cdJQ1BoZkO1vlV74/s1040/1594510824762.JPEG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="780" data-original-width="1040" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgoBX6bOGmJ9cs74C4-2lITIKij8B5I0jvXa2RadT27CElFHFSKNLh8umsnf2rqTg2jsC15-RWfV_ph5_mN9-BIFxqxLLNEeTQMRMu2GHU0k9adxbbKlEQedU3I957cdJQ1BoZkO1vlV74/s320/1594510824762.JPEG" width="320" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzZbShwA8ygKnfl1f990j7cLK3vPb3VlewT3-wSBqlsifdV90UltJyiz_gfkh9djaB2DgtVsnX9SgNX5h77y1y3pXc7z8HQyT3mqCTYm4mtONuJZ_wamEo6KYqHNtm6icZAQ3Heapg-98/s1040/1594510825761.JPEG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="780" data-original-width="1040" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzZbShwA8ygKnfl1f990j7cLK3vPb3VlewT3-wSBqlsifdV90UltJyiz_gfkh9djaB2DgtVsnX9SgNX5h77y1y3pXc7z8HQyT3mqCTYm4mtONuJZ_wamEo6KYqHNtm6icZAQ3Heapg-98/s320/1594510825761.JPEG" width="320" /></a></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjouKTGOi_CmyQETq3bKwnQfBgwatewgmI_3CP8mEM8hBu6D1E21KtX7vtuSrbrAbIqsgy_-XzyYqzfSCi1URJ5ioZ9Q4Wfhc83VSAxuXKNtpEM3foGrj9e3BX0eBQqftzPI0npa2-u3pg/s1040/1594510824025.JPEG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="780" data-original-width="1040" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjouKTGOi_CmyQETq3bKwnQfBgwatewgmI_3CP8mEM8hBu6D1E21KtX7vtuSrbrAbIqsgy_-XzyYqzfSCi1URJ5ioZ9Q4Wfhc83VSAxuXKNtpEM3foGrj9e3BX0eBQqftzPI0npa2-u3pg/s320/1594510824025.JPEG" width="320" /><br /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: left;">The modems themselves have an interesting feature, where they report themselves when you connect to them via the USB serial port. However, if you power the Envoy on with the USB already connected, this doesn't always occur, and sometimes I had problems getting anything out of the modem in that context. The solution is to disconnect the USB port from the computer and reconnect it. <br /></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">I'll have to think about a robust solution for that, as it can cause LBARD's auto-detection of the modem to fail, which would be unfortunate for automatic operation, which is what we want to achieve. It might be possible to do a software USB disconnect and reconnect, for example. Or there might be some magic option I need to setup on the radio. Anyway, we'll come back to that later.</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">The modems need only a few simple commands to control them -- assuming the HF radios are sitting on the same channel. I haven't yet used the commands that the modems offer for selecting channels, although it should be quite possible.<br /></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">The first thing to do is to give the modem a station ID. This is a bit like a phone number. It can have upto 6 digits, but for reasons I don't immediately recall, the last two digits cannot be 00. This is set using the <b><span style="font-family: courier;">AT&I=<i>number</i></span></b> command, e.g.:</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: courier;"><b>AT&I=2</b></span></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">would set the station ID to 2. Once that has been setup, you can tell a modem to try to dial it using the good old <b><span style="font-family: courier;">ATD</span></b> command, e.g., <br /></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"><b><span style="font-family: courier;">ATD2</span></b></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">would call a modem that had been configured as previously described. This will cause the standard <span style="font-family: courier;"><b>RING</b></span>, <span style="font-family: courier;"><b>NO ANSWER</b></span>, <span style="font-family: courier;"><b>NO CARRIER</b></span>, and <span style="font-family: courier;"><b>CONNECTED</b></span> messages to be displayed based on whether the remote end answers with <span style="font-family: courier;"><b>ATA</b></span> or not, or later disconnects with <span style="font-family: courier;"><b>ATH0</b></span>. In short, Codan have done a great job of making this modem masquerade as an old-fashion 2400 bps data modem. This helps it be compatible with various existing software, and helps me, because I have previously had many adventures programming such modems.<br /></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div>The modems default to having compression enabled, although I don't know what kind of compression they have, nor how effective it is. For now, I am just ignoring it, and leaving it in the default setting of enabled. <br /></div><div><br /></div><div>Armed with the information on how to control the modem, I set about creating an LBARD driver for it. This consists mostly of an auto-detection routine, the state machine for managing the call states, and routines for sending and receiving packets. This is all in the <b><span style="font-family: courier;">src/drivers/drv_codan3012.c</span></b> file.</div><div><br /></div><div>Initialisation just consists of sending a few AT commands to set the modem's HF station ID, and enable hardware flow-control:</div><div><br /></div><div><span style="font-family: courier;">int hfcodan3012_initialise(int serialfd)<br />{<br /> char cmd[1024];<br /> fprintf(stderr,"Initialising Codan HF 3012 modem with id '%s'...\n",hfselfid?hfselfid:"<not set>");<br /> snprintf(cmd,1024,"at&i=%s\r\n",hfselfid?hfselfid:"1");<br /> write_all(serialfd,cmd,strlen(cmd));<br /> fprintf(stderr,"Set HF station ID in modem to '%s'\n",hfselfid?hfselfid:"1");<br /><br /> snprintf(cmd,1024,"at&k=3\r\n");<br /> write_all(serialfd,cmd,strlen(cmd));<br /> fprintf(stderr,"Enabling hardware flow control.\n");<br /><br /> // Slow message rate, so that we don't have overruns all the time,<br /> // and so that we don't end up with lots of missed packets which messes with the<br /> // sync algorithm<br /> message_update_interval = 1000;<br /> <br />}</span><br /><br /></div><div>Flow control is a bit interesting to manage. The big problem is that we don't want the buffer of the modem to get too full, as this makes the round-trip time for control messages too long, which slows things down. For example, if the far end doesn't quickly confirm that a given bundle has been received, the sender will keep sending pieces of it. Thus we need to keep well away from hitting the flow control-imposed limit. The packet sending and receiving routines keep track of this by having sequence numbers that are then returned to the sender, so that the sender can get an idea of the number of packets in flight.</div><div><br /></div><div>LBARD modulates packet flow, not by the number of packets in flight, but by the average interval between packets. As a first-cut approach, I set the packet interval dynamically based on the number of outstanding packets in flight. Basically I set the packet interval to some multiple of the number of outstanding packets. 250ms per outstanding packet seems to work ok, with reasonable throughput balanced against not having too many unacknolwedged packets to waste outgoing bandwidth with needless retransmissions. I can refine that a bit more later.<br /></div><div><br /></div><div>The state machine for the modem is also fairly simple:</div><div> <br /><span style="font-family: courier;"> switch(hf_state) {<br /> case HF_DISCONNECTED:<br /> if (hfcallplan) {<br /> int n;<br /> char remoteid[1024];<br /> int f=sscanf(&hfcallplan[hfcallplan_pos],<br /> "call %[^,]%n",<br /> remoteid,&n);<br /> if (f==1) {<br /> hfcallplan_pos+=n;<br /> fprintf(stderr,"Calling station '%s'\n",remoteid);<br /> char cmd[1024];<br /> snprintf(cmd,1024,"atd%s\r\n",remoteid);<br /> write_all(serialfd,cmd,strlen(cmd));<br /> call_timeout=time(0)+300;<br /> hf_state=HF_CALLREQUESTED;<br /> } else {<br /> fprintf(stderr," remoteid='%s', n=%d, f=%d\n",remoteid,n,f);<br /> }<br /> <br /> while (hfcallplan[hfcallplan_pos]==',') hfcallplan_pos++;<br /> if (!hfcallplan[hfcallplan_pos]) hfcallplan_pos=0;<br /> } <br /> break;<br /> case HF_CALLREQUESTED:<br /> if (time(0)>=call_timeout) hf_state=HF_DISCONNECTED;<br /> break;<br /> case HF_ANSWERCALL:<br /> write_all(serialfd,"ata\r\n",5);<br /> hf_state=HF_CONNECTING;<br /> call_timeout=300;<br /> break;<br /> case HF_CONNECTING:<br /> // wait for CONNECT or NO CARRIER message<br /> break;<br /> case HF_DATALINK:<br /> // Modem is connected<br /> write_all(serialfd,"ato\r\n",5);<br /> hf_state=HF_DATAONLINE;<br /> call_timeout=time(0)+120;<br /> break;<br /> case HF_DATAONLINE:<br /> // Mode is online. Do nothing but indicate that the modem is<br /> // ready to process packets<br /> if (time(0)>call_timeout) {<br /> // Nothing for too long, so hang up<br /> sleep(2);<br /> write_all(serialfd,"+++",3);<br /> sleep(2);<br /> write_all(serialfd,"ath0\r\n",5);<br /> hf_state=HF_DISCONNECTED;<br /> }<br /> break;<br /> case HF_DISCONNECTING:<br /> break;<br /> default:<br /> break;<br /> }<br /></span> <br />In short, we make a call if we have been instructed to, and am not currently on a call. Otherwise stay on the call unless we get no data for too long, in which case, hang up, and repeat the process.<br /></div><div><br /></div><div>With the above and the necessary glue, I was able to get packets flowing between the two modems fairly easily. But when I started trying to transfer bundles, I started seeing the protocol get confused with both sides trying to send each other the same bundle endlessly. This should never happen, as when one end realises that the other has a given bundle, it should stop trying to send it, because it has pretty clear evidence that the other end has it already. <br /></div><div><br /></div><div>I'd seen this problem before in the field with UHF radios in the Mesh Extenders, but it had stubbornly refused to be reproduced reliably in the lab. So while it is on the one hand annoying, its actually a great relief to have found the problem, and to be able to progressively work on solving it.</div><div><br /></div><div>It turns out to be caused by a number of things:</div><div><br /></div><div>First, the TreeSync algorithm seems to not realise when the other end receives a bundle, even though it should. That module is quite fancy and efficient at doing its job, and I don't understand it well enough to figure out what subtle problem it is suffering from. The mechanism by which this causes the lockup took a little while to realise, but it made a lot of sense once realised it: Once both sides have as their highest priority bundle for the other a bundle that the other side actually has, and that trips this TreeSync problem, then they will continue to try to send it forever.<br /></div><div><br /></div><div>It self-corrects eventually, but only briefly, because when it produces a new generation ID, the tree is re-built, which allows it to send the next few bundles before locking again. To work around this, I now maintain a list of bundles that we have seen the other end try to send. If they try to send a bundle, then they must by definition have it. Therefore we should never try to send that bundle to them again. This works to stop the TreeSync algorithm from re-scheduling the same bundle over and over.</div><div><br /></div><div>That fixed the protocol lock-up. Then I was seeing that the scheduling of which data blocks to send was being completely random. Now, it is supposed to be rather random, so that if multiple senders are available for a given bundle, a single receiver can receive pieces from all of them, and thus receive the bundle as a whole sooner. <br /></div><div><br /></div><div>The problem was that the system for keeping track of which blocks we think the recipient might already have was not being activated, because it was getting confused about which bundle we thought we were sending them, or should be sending them. As a result very small bundles consisting of only a few blocks would send in reasonable time, as the probability of all their blocks being sent would approach 1 very quickly. But larger bundles consisting of more than a dozen or so blocks would be extremely slow to send, because it was basically pot luck waiting for the last few holes to get filled in.<br /></div><div><br /></div><div>This required a bit of fiddling about in a few different places to pick up all the corner cases of when we should reset it, and making sure that we clear the bitmap of sent blocks and other minor details. I also fixed the situation where we see another party sending a bundle to someone, and we haven't yet worked out what to send to them. In that case, we now join in sending the same bundle, so that the receiver can hopefully receive the whole bundle that bit quicker. <br /></div><div><br /></div><div>These multi-way transfer optimisations are not really applicable to the HF radios, as the modems are designed for point-to-point, but it made sense to implement them, anyway. Also, down the track, it would be possible for us to look at ways to make a radio listen to multiple HF radio channels in parallel, and receive all the content being delivered on them. This promiscuous mode of listening could greatly increase the effective capacity of the system, and basically is just a sensible use of the true broadcast nature of radio.</div><div><br /></div><div>Back to the bitmap sending, even after I got it working, it was still not particularly efficient, as even when there were a relatively few pieces left to send, it would still chose them at random, potentially resending the same block many times until the next bitmap update came through. As the round-trip latency is typically 10 - 20 seconds, by the time all the modem buffering is done, this can add significant delays through redundant sending. Also, it can get stuck sending manifest pieces for a long time, which again can be quite wasteful, as the manifest is typically less than 300 bytes, and will keep getting resent until the bitmap update comes through indicating that the manifest has been fully received. <br /></div><div><br /></div><div>What would be better, would be to keep track of the set of pieces that are outstanding at any point in time, and just randomise their order, and then send them in that randomised order, and then start over. That way we will avoid redundant sending. The order can be randomised afresh each time we get a bitmap update.</div><div><br /></div><div>First implementation of this, seems to work fairly well: It sends the blocks in random order. It's still not as fast as I would like, averaging only about 20 bytes per second in each direction. This will be partly because I am sending 64 byte blocks at random, but the packets can hold 128 or sometimes even 192 bytes. But if there are no contiguous outstanding blocks, then it will end up being very inefficient. I'll likely fix that by allowing a message type that can contain more than one piece of the same bundle. Actually, I figured out an easier quick fix: Consider all odd-numbered blocked to have been send once already, so that we try to send all the even-numbered ones at least once. <br /></div><div><br /></div><div><br /></div><div>The bigger problem, though, is that the indication of when the other end has received a bundle is not working properly, and so we are back to our protocol lock-up situation.</div><div><br /></div><div>So looking at a particular run when this is occurring, we have LBARD2 and LBARD8 talking to each other.</div><div><br /></div><div>The begin by exchanging two small bundles with each other, before moving on to transfer bundles of several KB. <br /></div><div><br /></div><div>LBARD2 receives bundle 4B47* (4870 bytes long) from LBARD8. LBARD8 receives C884* (4850 bytes long) from LBARD2. <br /></div><div><br /></div><div>So both now have C884*. The problem is that both think that the other doesn't have it. As C884* is smaller than 4B47*, it has a higher TX priority, and thus both sides keep trying to send it to the other.</div><div><br /></div><div>LBARD8 received C884* at 11:42.38. By 11:42:47, LBARD8 has received confirmation that the bundle was inserted into the Rhizome database, and thus knows definitively that it has the bundle.<br /></div><div><br /></div><div>Even though it knows that it has this bundle, it continues to accept receiving the bundle afresh from LBARD2. <br /></div><div><br /></div><div>Part of the problem is that LBARD8 doesn't immediately note that LBARD2 has the bundle. It should do this, whenever it sees LBARD2 sending a piece of the bundle. <br /></div><div><br /></div><div>In fact, there is no indication whatever that LBARD8 is receiving the pieces of the bundle after it has received it the first time. I'll need to re-run with more debugging enabled. Perhaps part of LBARD is noticing that it already has the bundle, thus is ignoring the messages, but without scheduling the indication to the other end that it already has the bundle. But even if that is not the case, the receiver shouldn't be willing to again start reception of the same bundle, after it knows that it has it itself. <br /></div><div><br /></div><div>Each run has randomised bundle IDs, which is a bit annoying. So I have to trawl through the logs again to see what is going on.</div><div><br /></div><div>So this time, LBARD2 receives bundle D386* from LBARD8 at 12:18.26.</div><div><br /></div><div>Interestingly, the "recent senders" information is correctly noting that LBARD8 has sent a piece of this bundle. So I can make use of that to update the other structures, if necessary.</div><div><br /></div><div>At the point where LBARD2 receives D386*, it is sending bundle 3A7C* to LBARD8. <br /></div><div>Ah, I have just spotted at the point where we realise that the other party has the bundle, instead of dequeuing the bundle to them, we are actually queueing it to them. That should be fixed. This is also the point at which we should mark that we know that that peer has the bundle already. So we will try that in the next run. Simultaneously I have added some instrumentation for the "prioritise even over odd" optimisation, as it doesn't seem to be doing exactly what it should.</div><div><br /></div><div>Fixed that, and some other related bugs. It now does a fairly reasonable job of scheduling, although there is still room for improvement. The big problem is that it is still sitting at around 20 bytes per second of effective throughput. <br /></div><div><br /></div><div>Now, this isn't as bad as it sounds, as that includes the overhead of negotiating what needs to be sent, and packetising it all, so that if the modems do lose packets, we don't lose anything. It also reflects the reality that we are only sending one packet in each direction every 3 seconds or so. With an effective MTU of 220 bytes, we only end up fitting one or two 64 byte blocks of data in each. Thus our potential throughput is limited to about 64 or 128 bytes every 3 seconds = ~20 -- 40 bytes per second. <br /></div><div><br /></div><div>So at the current state of affairs, we have something that is at least 10x faster than the old ALE messaging based transport -- and its is bidirectional throughput. We are seeing message delivery latencies for short messages that are about 30 to 50 seconds. This compares very favourably to the ~3 minutes that sending a bundle was taking with the ALE transport. But it still isn't as good as we would like or, I think, we can achieve. We know that we should have something like 125 bytes per second of bandwidth in each direction. Even with the suboptimal scheduling of pieces to send, we should be able to achieve at least half of that figure.<br /></div><div><br /></div><div>The first port of call is to try to optimise the packet sending rate, as it seems that one packet of ~260 bytes in each direction every ~3 seconds or so is probably not quite filling the 2,400bps channel, even allowing for the change-over. Well, that sounded like a great theory. Looking closer, the packet rate is closer to one every 2 seconds, not every 3 seconds. 2400bps / 2 = 1200bps = 150 bytes per second raw rate in each direction, before subtracting a bit for turn-around. Thus sending a 255 byte packet every 2 seconds = ~128 bytes per second, which is pretty much spot on the channel capacity.</div><div><br /></div><div>This means we will have to look at other options. The most obvious is to send large slabs of bundles, without all the other protocol fluff. This is only really possible in this context, because the modem link is point-to-point, so we don't have to worry about any trade-offs. In fact, the whole packet format and message types are really a bit of an over-kill for this configuration. But short of re-writing the whole thing, we can instead get some substantial gain by just sending a slab of useful bundle data after every normal LBARD packet. This will get us half of the channel use at close to 100% goodput, and thus bring the average up to >50%. If we send large contiguous slabs, then that will also reduce the opportunity for the block scheduling to get too much wrong, so it may well be even more effective. <br /></div><div><br /></div><div>The modem has no natural packet size limit, so we can even send more than 256 bytes at a time -- if we trust ourselves to the modem's error correction to get the data to the other end intact. If errors creep in, and we don't have a means of detecting them, it will be really bad. The current error correcting code works on blocks of 232 bytes. We don't want to send huge amounts, though, as we want to maintain the agility to respond to changing bundle mixes. <br /></div><div><br /></div><div>The largest number of 64 byte blocks we can thus fit would be 192 bytes. 256 would have been nicer, so that it would be an even number of blocks. That's a bit annoying, so I'll have to make a different type of packet that can carry upto 256 bytes of raw data. It will assume that the data is for the currently selected bundle, and thus only have a 4 byte offset and 2 byte length indicator, plus the escaping of the packet termination character. Thus it should have an average overhead of less than 10 bytes, and thus be better than 90% efficient.<br /></div><div><br /></div><div>Well, the next challenge was that some bytes were getting lost, as though buffer overflow was occurring. However, as I have hardware flow-control enabled, I'm not sure of the cause here. My work around is to send only 192 bytes, after all, and to make sure I throttle back the packet sending rate, to try to keep the data flowing at a reasonable rate, without overwhelming the buffers. This is a bit worrying, as in real life it is quite likely that we will get data rates below 2,400bps on the HF links quite often. <br /></div><div><br /></div><div>Reducing the data packet size to 192 bytes has seemed to fix the buffer overflow problem. Also, I am now seeing the larger bundles go through more quickly. However, I am still seeing the problem where the same bundle is received more than once. Presumably it starts being received a 2nd time before we get confirmation from the Rhizome data base that it has been inserted. <br /></div><div><br /></div><div>The solution would be to filter the receive list whenever we notice a bundle has been added to the rhizome database. That way such duplicate transmission of the same bundle should get suppressed. Examining the log file of when this happened, it turns out that the problem was actually that the bundle was corrupted during reception. This is possible if the bundle being transferred changes part way through, as our pure data packets lack bundle identifying information to save space. However, it looks like we will will need it after, to avoid problems with inferring which bundle is being sent by the remote end.<br /></div><div><br /></div><div>There is also a pattern where some pieces of a bundle are failing to get sent in the data packets. I'm suspecting that this is due to escaping of characters causing the encoded size of the data packet to become too large. So I might need to trim the packet size again. (This is all part of why I don't like it when systems try to hide packetisation behind a pretend stream interface. You end up having to put framing back in the stream, which adds overhead, and there is no way to make sure your packets actually end on the underlying packet boundaries which increases latency. But I understand why people implement them.)</div><div><br /></div><div>So I guess I'll have to drop back to 128 bytes per data packet, as well as including the bundle information.</div><div><br /></div><div>Ok, that has helped quite a lot. It is now delivering the packets more reliably. But the throughput is still pretty horrible, with good-put rates still somewhere around 25 bytes per second. I suspect the modems are really not designed for this kind of two-way sustained communications, but rather are optimised for mostly uni-directional transfers. <br /></div><div><br /></div><div>If I can get hardware flow-control working, I should be able to improve things by using larger packet sizes. But I think its also about time I actually wrote a simple test programme that measures the actual bidirectional throughput that these modems can deliver. That way I know whether it is my protocol side of things that is the problem, or whether I really am hitting the limit of what these modems can achieve. <br /></div><div><br /></div><div>But that will all have to wait for the next blog post.<br /></div><div></div>Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com0tag:blogger.com,1999:blog-9068638994894102968.post-34767242412570336112020-01-09T07:03:00.002-08:002020-01-09T07:03:35.584-08:00It's 2020 and Australia is burningIt almost doesn't need saying, but <a href="https://www.nytimes.com/2020/01/08/opinion/australia-fires.html">Australia is burning.</a> Lives have been lost. Houses destroyed. Livestock, farms and livelihood all gone up in smoke, quite literally. I can't even imagine the pain and distress that this is all causing for so many. But what I can do, is try to do something about the technology gaps that are making a bad situation even worse for those affected.<br />
<br />
As readers of this blog will know, I have been devevloping resilient telecommunications solutions for the last decade or so. The best known of those is the <a href="http://servalproject.org/">Serval Project</a>, which is a combination of software for smart phones that can form mesh networks, and small low-cost communications repeaters that we call <a href="https://www.elektormagazine.com/articles/serval-mesh-operating-mobile-phones-without-a-cellular-network">Serval Mesh Extenders</a>. Here is a brief introduction to what, and why, we are making the Serval Mesh:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/WD5rnaMGOGo/0.jpg" src="https://www.youtube.com/embed/WD5rnaMGOGo?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<br />
And to go right back in history to almost the beginning, here is the original motivation of the project:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/Z3p2BYFXBkU/0.jpg" src="https://www.youtube.com/embed/Z3p2BYFXBkU?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<br />
And for another blast from the past, here is the original field test call from back in 2010, at Arkaroola in the Outback, which was also <a href="https://www.abc.net.au/news/2010-07-12/mobile-invention-could-be-desert-lifeline/900762">covered by the ABC</a>:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/ZG0ph6zXa7s/0.jpg" src="https://www.youtube.com/embed/ZG0ph6zXa7s?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<br />
<br />
Here's another video about the Mesh Extenders that we made back in 2013, when we were first developing them:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/30qNfzJCQOA/0.jpg" src="https://www.youtube.com/embed/30qNfzJCQOA?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<br />
Since then, we now have much more mature hardware, and have tested the hardware for long-term operation in the field in a coastal area of Vanuatu, in large part thanks to a grant from DFAT under the <a href="http://pacifichumanitarianchallenge.org/winners/#InfraIndepMob">Pacific Humanitarian Challenge</a> a few years ago.<br />
<br />
While the Serval Mesh was originally developed with disaster situations abroad, it was always also made with more local situations in mind, for example, following cyclones or in very remote areas lacking phone coverage. For example, one variant of the technology that was worked on for a while, was the concept of an emergency network that uses vehicles as the main component, because they are the most ubiquitous infrastructure once you get out of the big cities. Here is a piece that we produced with Toyota on this concept:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/qtMydzfT1ts/0.jpg" src="https://www.youtube.com/embed/qtMydzfT1ts?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
That video is also a good general introduction to the potential for the technology.<br />
<br />
Basically any situation where cellular or other normal communications infrastructure is missing, damaged or disabled for any reason, the Serval Mesh lets local communities form their own local-area digital communications network.<br />
<br />
There are also some branches to this work, for example, <a href="https://media.ccc.de/v/36c3-10800-creating_resilient_and_sustainable_mobile_phones">creating mobile phones that include the Mesh Extender functionality internally, and are designed to be fully "self-sovereign,"</a> that is independent of all power, communications and other infrastructure -- and that can offer security and privacy at least as good as the present state of the art.<br />
<br />
So where are we at now?<br />
<br />
Basically we have proven all the various parts of the technology that we need to make this a reality. What we need to do now, and our plan for 2020 -- and even before the first started -- is to focus time and energy on shaking down the last wrinkles in the system, so that it is ready for deployment by communities in the field. <br />
<br />
This includes revising the Mesh Extender circuit board, and fixing some known problems in the software, and then testing with larger networks of dozens to hundreds of units, that would more accurately reflect the real use of the Serval Mesh by communities in the field.<br />
<br />
Flinders University has kindly granted me a "sabbatical" year to focus on this. I'll be based at Arkaroola to do this, where we did the original field testing, so that we can use the vast rugged landscape there, including over 600 square kilometres of mountainous desert, to be able to deploy realistic test scenarios, and work on this scaling.<br />
<br />
Our goal, is that the Serval Mesh can be ready for individuals and communities who want to use it, to do so, by the end of this year. <br />
<br />
Achieving this will depend on a lot of factors, including the ever present problem of having sufficient funds for the equipment that would allow us to work more quickly, and scale up our tests more meaningfully. It also doesn't answer the question of how we support communities in their use of the technology once it is finished, or who will offer it for sale. But those are issues that we can think further through during the coming year.<br />
<br />
But one thing is crystal clear to me: We each need to consider what we can do to mitigate the effects of the fires, and to adapt to a future where such events are more and more likely, and to do what we can to mitigate this threat. Whether we can see the whole solution or not, is to me secondary: We must simply make sure that we do what we can now. And that is what I am going to do with 2020.Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com6tag:blogger.com,1999:blog-9068638994894102968.post-86496113811021179052018-11-15T01:50:00.000-08:002018-11-15T01:50:27.592-08:00Serval Chat iOS Port finally emerges from the labGoodness me, this is probably the single piece of the Serval Project that has taken the longest compared to what was expected. We have had a number of misadventures along the way with false starts to make a working port of Serval for iOS, but today, we finally have it.<br />
<br />
This adventure began back in about 2014 when the NLnet provided us with a grant to achieve this. Needless to say, we didn't make our original planned timelines. We had some turn over of engineers and students on the project, and Apple changed from Objective C to Swift, and we hit some early dead ends and speed humps. But we now have something that works quite nicely on iOS 12.<br />
<br />
This port runs the core serval-dna source in a thread in the background, so that it is fully capable and fully compatible. Getting serval-dna to compile for iOS, and produce binaries that can run on all iPhone CPU types, and that could be sensibly accessed from Swift, and that would behave properly was non-trivial to say the least. We got to that point earlier in the year, and then the remaining time has been spent making a proof-of-concept app, and working through the remaining issues, like not having the thread die when the app looses focus, and working around a pile of funny little problems, including using the properly containerised paths in strange places we had overlooked it, and discovering that named sockets just don't work on the iPhone.<br />
<br />
So today we reached the happy point of having something working, just hours before our French exchange student was due to finish up. His last great act for us was to help prepare the following short video to celebrate this milestone.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/-1np0RzAD0k/0.jpg" src="https://www.youtube.com/embed/-1np0RzAD0k?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<br />
There is still some missing functionality to implement, and then the real fun begins: Trying to get the app approved for release on the Apple App Store. But that will be an adventure for another day.Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com4tag:blogger.com,1999:blog-9068638994894102968.post-67104370616235949632018-11-12T17:30:00.000-08:002018-11-12T17:30:00.737-08:00Slashing the cost of tsunami (and bushfire) early warning systems<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: left;">
We have just about finished off a project that we have had funded by the Humanitarian Innovation Fund (HIF), which is funded by the UK's foreign aid program. This has been a bit of a complementary variation on our existing work on the Serval Mesh. Rather than on-ground peer-to-peer communications, the goal is to get communications out in to very remote locations, in particular, to get earaly warning for tsunamis, cyclones, bush fires and other hazards.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The tragic events in Sulawesi last month are a stark reminder of the need for tsunami early warning systems. Following the earthquake, a tsunami warning was generated, but there was no way to get the message out to the impacted communities, in part because the earthquake had knocked out the cellular network, and in part, because many of the smaller villages likely lacked cellular coverage to begin with, or in the absence of grid electricity, many phones would have been turned off to conserve power.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
These circumstances are not by any means unique to Indonesia -- they occur all around the Indo-Pacific in various forms and permutations. The question is how to solve the problem.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The traditional approach is big tsunami early warning towers. However, those are really expensive, especially to install in remote areas, often costing $10,000 or more per tower. For isolated villages of a hundred or so people, the per-capita cost is simply infeasible. Maintenance is also problematic both financially, and logistically</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
We have tackled this problem from the perspective of making a solution that works best in these resource-constrained environments. In the end, our solution is rather simple: Use off-the-shelf satellite TV receiver parts combined with some clever signal processing work by <a href="http://othernet.is/">Othernet.is</a>, so that no dish is required -- just point it to within about 10 degrees of where the geostationary satellite is. This gives us a low-bit-rate digital broadcast signal that we can receive over a wide area. Add to this direct-leasing of satellite capacity, so that the operating cost of the system is low and fixes, i.e., there is no monthly charge per user, so that it can be scaled up region wide. Then all that is missing is a cheap automotive air-horn, so that the alarm can be heard within a village. </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The result is receiver hardware that costs maybe $200 in modest quantities, i.e., around 100x cheaper than the old approach, and that <i>can be installed by communities themselves</i>. That is, we don't have to rely on big programs to roll these things out. Once people know about them, they are cheap enough for a village to buy, and easy enough for them to install themselves.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
This just leaves the problem of disaster resilience/warning technologies not getting maintained during the year, and thus often having failed before they were needed. Our solution here is to include a low-power FM radio transmitter, so that news, weather, climate change mitigation and other information can be received in each village. This means when the unit stops working at some point (nothing lasts forever in the tropics), they know immediately (no more radio signal, and the services it was providing them), and they can organise themselves to get it replaced and reinstalled, again, without having to rely on an NGO or aid program to do it.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
In short, it is not only way cheaper, it is also way more appropriate to the needs of the Pacific. It turns out that these characteristics also make it a great potential solution for bush-fire early-warning in Australia, where cell networks are often taken out by large bush fires (= wild fires, for those on the other side of The Pond), being cheap enough for house-holds to install themselves, and simple enough for them to self-install.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
We are at the proof-of-concept stage and are looking for ways to fund the next stage. Hopefully we will find a funder who can help us bring it to operational readiness in the next year or so.</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/IDeG-DUetdI/0.jpg" src="https://www.youtube.com/embed/IDeG-DUetdI?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<br />Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com0tag:blogger.com,1999:blog-9068638994894102968.post-32011385705093844762018-04-04T23:46:00.000-07:002018-04-04T23:46:07.859-07:00Building an indoor test network and testing LBARD bug fixes<div class="separator" style="clear: both; text-align: left;">
After the successful bug hunting last week, this week I have turned my attention to setting up to easily verify whether the bugs have been fixed. This means setting up a multi-hop Mesh Extender network in a convenient location. </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The building I am based in was purpose-built for the College of Science & Engineering, and we were able to have a number of specific provisions included during the design phase to make this kind of test network easier to deploy, and we are now finally able to start making fuller use of that. More specifically, we have 13 points around the building where we have copper and fibre directly to our lab, and also have some funky mounting brackets, on which we can fit radio hardware, all nicely protected from the outside weather. </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
So the obvious approach was to fit several Mesh Extenders to some of those points, and form a multi-hop UHF network. Here are some photos from various angles showing the units:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
First up, we have Mesh Extender serial #23 with SID BCA306B0*, which is at point NE1. Note the extra long 6dB antenna we fitted to this one to make sure it could talk to the one at N4 (1A15ED32*). Even with that antenna, the link is quite poor, often getting only one packet every 5 to 10 seconds. While on the one hand annoying, this is actually quite helpful for testing the behaviour of Mesh Extenders in marginal link conditions.</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDxoNh8eGezGs9e8Z3ygd_V695IjeE8rve5b8xX3Wfnl1_0m7KAn_nlx7b39okjPymYqoP_fkCw4wQw-PEw8cFyifRIfwAhaw9YFYYkGkfHN_5trX5kdToj5k9r5p7uySg6PqXAgjTM5I/s1600/P1020009.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDxoNh8eGezGs9e8Z3ygd_V695IjeE8rve5b8xX3Wfnl1_0m7KAn_nlx7b39okjPymYqoP_fkCw4wQw-PEw8cFyifRIfwAhaw9YFYYkGkfHN_5trX5kdToj5k9r5p7uySg6PqXAgjTM5I/s400/P1020009.JPG" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjU_7K9ikWrpm5V31sYyTK4KBIp5TwW0ue_NeyTcYctGFX3XXwxq0p6W-8OFQYipDvthbc4oVJhyphenhyphenKd5briorF6ZBaMstu0NieefibefS6Yt7qELqo_nWygtumdVn_6hP6RuGZtp4Nz8Zqw/s1600/P1020010.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjU_7K9ikWrpm5V31sYyTK4KBIp5TwW0ue_NeyTcYctGFX3XXwxq0p6W-8OFQYipDvthbc4oVJhyphenhyphenKd5briorF6ZBaMstu0NieefibefS6Yt7qELqo_nWygtumdVn_6hP6RuGZtp4Nz8Zqw/s400/P1020010.JPG" width="400" /></a></div>
<br />
Mesh Extender serial #34 is at point N4, and has SID 1A15ED32*:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFf3w8oOvg01bDV-XChWkuftw7wE_lLRz0s-1ZfBvFAif-1hyphenhyphen6mNqxYh3Y6UOTnpR8tA5-jeulslKbZwDKwcc47Sl1YysfIJTen0rl860MexqzcbKat7pQvCUpZ9RP-cDW2vs7CwuDNcQ/s1600/P1020013.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFf3w8oOvg01bDV-XChWkuftw7wE_lLRz0s-1ZfBvFAif-1hyphenhyphen6mNqxYh3Y6UOTnpR8tA5-jeulslKbZwDKwcc47Sl1YysfIJTen0rl860MexqzcbKat7pQvCUpZ9RP-cDW2vs7CwuDNcQ/s400/P1020013.JPG" width="400" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYt30Jn_lnwBXICFfvx1AXxrOpSnmnWhEpIBTvgJ0TWUD7WG5P56MDmq1RaL5k-E1Opl-n6m1QqE68YwVrzx2Pq09bL2UqD7uO0dGivi8EsrnW6wcVRKCW2lknKa8gtioiI53VW89o6eQ/s1600/P1020012.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYt30Jn_lnwBXICFfvx1AXxrOpSnmnWhEpIBTvgJ0TWUD7WG5P56MDmq1RaL5k-E1Opl-n6m1QqE68YwVrzx2Pq09bL2UqD7uO0dGivi8EsrnW6wcVRKCW2lknKa8gtioiI53VW89o6eQ/s400/P1020012.JPG" width="400" /></a></div>
<br />
Note that we have removed the Wi-Fi antenna from this one. This was necessary to make sure that it can't communicate via Wi-Fi with any of the other units, which it was otherwise able to do intermittently, due to being relatively near to the one in the lab, which is on the same floor, and the one on level 5, to which it almost has line of sight up to via the stair case.<br />
<br />
Mesh Extender serial #50 is at NW5 and has SID 5F39F182*:<br />
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtKzFCEI0XV9WsYCO6r8cO0X5RcImPKhytXuD9Xn2ca-lOX_ArT-QeSYzQoNv32wseS1Cz69FkLGXLcNIahKtcV-Kpsotu9xu8csSkzS_rGnOL4oZ9r4VNJsmWR8jEJx9lMbD45ZkskrE/s1600/P1020015.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtKzFCEI0XV9WsYCO6r8cO0X5RcImPKhytXuD9Xn2ca-lOX_ArT-QeSYzQoNv32wseS1Cz69FkLGXLcNIahKtcV-Kpsotu9xu8csSkzS_rGnOL4oZ9r4VNJsmWR8jEJx9lMbD45ZkskrE/s400/P1020015.JPG" width="400" /></a></div>
<br />
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhStLsq9s4kOSehe5KnluKJOs6wTNkkJ_LFYkYgue1ZnTRR23ARnCx6k1mqtlSWaDVAbsxlTEcHUQFuWM1Oey6kVWr-UjeVkb4p65CyEXDi6SrzifBasWSIZDnN1aAETnlWZaInYRRFme4/s1600/P1020016.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhStLsq9s4kOSehe5KnluKJOs6wTNkkJ_LFYkYgue1ZnTRR23ARnCx6k1mqtlSWaDVAbsxlTEcHUQFuWM1Oey6kVWr-UjeVkb4p65CyEXDi6SrzifBasWSIZDnN1aAETnlWZaInYRRFme4/s400/P1020016.JPG" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZ9LPcwyhw2rDE9DPLaf_OC5JC9U4styEt2OUhoqHYBLSfDk6YktYiqOJSmY89qoeSeMbMzsWzNjP0WGnnPr1QviZFWHv_IL7DuBXMYmrjJv-lsNaVLTRJZCWgJEU_x__GM2AiCp9ME_8/s1600/P1020017.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZ9LPcwyhw2rDE9DPLaf_OC5JC9U4styEt2OUhoqHYBLSfDk6YktYiqOJSmY89qoeSeMbMzsWzNjP0WGnnPr1QviZFWHv_IL7DuBXMYmrjJv-lsNaVLTRJZCWgJEU_x__GM2AiCp9ME_8/s400/P1020017.JPG" width="400" /></a></div>
<br />
And finally, Mesh Extender serial #17 (6ABE0326*) is sitting on the rack in our lab:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6dKI_mKVipK10aYcqBSgVCpCqcR7_BFuIXjmNv5NZhWFZDSqz-ubBHVaK8eHg8A5Cpx4jmWgplGUPtBUY1vpWieZKOe8xfHzNA8iMGzo7FizSIFxdYQunW22R9XYOQsbeVhtJZvO58x4/s1600/P1020018.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6dKI_mKVipK10aYcqBSgVCpCqcR7_BFuIXjmNv5NZhWFZDSqz-ubBHVaK8eHg8A5Cpx4jmWgplGUPtBUY1vpWieZKOe8xfHzNA8iMGzo7FizSIFxdYQunW22R9XYOQsbeVhtJZvO58x4/s400/P1020018.JPG" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLc8ZdF5HlNDSB6gq4hGmLtQLhFHKrYR104rw0YrlwK-ZwWwBDsCdS0hQ6sVLjQKEniAJqUZW2VuPUKqWgHR5WdeYvxcY4yvetVTejyYjYKCMmyvqa1iOIrtZKy0LHa9FGmi1fspLoG-U/s1600/P1020019.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLc8ZdF5HlNDSB6gq4hGmLtQLhFHKrYR104rw0YrlwK-ZwWwBDsCdS0hQ6sVLjQKEniAJqUZW2VuPUKqWgHR5WdeYvxcY4yvetVTejyYjYKCMmyvqa1iOIrtZKy0LHa9FGmi1fspLoG-U/s400/P1020019.JPG" width="400" /></a></div>
<br />
Mesh Extender #34 (1A15ED32*) is the only node that can see all the others, which we confirmed by using the debug display on LBARD:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjy9YfSjMXTl96nTfu24D40m6P0IBTIqSsp5qGdnqleaQiH97qfdtncdtSa5zlFJAPUvY5yc2dqaQxwO0cc7q2KjIV5exCypmuFNo7dbiG_sZT4vn9shBUIDy69R1WFfbG9tkRKszOCqqs/s1600/P1020014.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjy9YfSjMXTl96nTfu24D40m6P0IBTIqSsp5qGdnqleaQiH97qfdtncdtSa5zlFJAPUvY5yc2dqaQxwO0cc7q2KjIV5exCypmuFNo7dbiG_sZT4vn9shBUIDy69R1WFfbG9tkRKszOCqqs/s640/P1020014.JPG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
I was still fiddling around with things at this point, but you can see already that the RSSI for BCA306B* is quite a lot lower than for the others. Although here it looks like the link is healthy, it really does come and go a lot, and goes seconds at a time without a single packet arriving. </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The last of the four rows, the one showing 39F1821A* is actually a known-bug manifesting. This is a ghost connection, caused by receiving a packet from 5F39F1821*, with the first byte having gone walk about for some reason. We are still to work out what the cause of this problem is. It's on our bug list to find and fix.</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
Now, to give a better idea of how the network is laid out, here are a couple of shots from the outside of the building. The first is looking along the northern facade to give a sense of the length of the building. It is from memory about 50m long.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZMJVKXV9AHPZ8uc-XRHavkj6FWJyurrpN2N_lqvFbABxtNfSpS-OmG6welXM8sK6t4pQrN01QPVV7-Hl5uV-xB-2jkjs5GiTa3wY7VpDWrUyiNkdjudChvkdYCMKL41-gX9KUQgdvVIo/s1600/P1020020.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZMJVKXV9AHPZ8uc-XRHavkj6FWJyurrpN2N_lqvFbABxtNfSpS-OmG6welXM8sK6t4pQrN01QPVV7-Hl5uV-xB-2jkjs5GiTa3wY7VpDWrUyiNkdjudChvkdYCMKL41-gX9KUQgdvVIo/s400/P1020020.JPG" width="400" /></a></div>
<br />
And here is the whole facade:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEPX2YGqMVG_yH08KqclxvQRSpR5TjxYRR2uuLgRFXuxBDCUm9ddA5xxg_oYTeEu0_UFckfAZkyuJtiKky4NMTOb7GE6nIWU-J0xC5bbavv-DLelzB8rUa4hMJJ7UjbgZNtoQZjiyPMZI/s1600/P1020021.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEPX2YGqMVG_yH08KqclxvQRSpR5TjxYRR2uuLgRFXuxBDCUm9ddA5xxg_oYTeEu0_UFckfAZkyuJtiKky4NMTOb7GE6nIWU-J0xC5bbavv-DLelzB8rUa4hMJJ7UjbgZNtoQZjiyPMZI/s400/P1020021.JPG" width="400" /></a></div>
And, here is the facade showing, in red, the locations of the three units that are located on the mounting points just behind the three layers of metal-film coated glass that acts as a partial Faraday Cage. The black spot shows the approximate position of the lab, as projected onto the northern facade, although it is in reality on the southern side of the building, approximately 25 metres south of the northern facade. Green lines indicate strong UHF links, and the yellow line the poor link between the first and fourth floors.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3bHprePm3GYFyMSicf1CtLKv05TTV-7X4Lizd1TUfqPkw9TwywkVWWLAerQ93bWgc8jygtjmG6eOJ9foK6YZEzYHjrRmR0DN8sjSgFYIR6cXEPxTSHb1KbbIrjbyEhpZ3X3wdKzBzz8I/s1600/TonsleyMeshExtenderLocations.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3bHprePm3GYFyMSicf1CtLKv05TTV-7X4Lizd1TUfqPkw9TwywkVWWLAerQ93bWgc8jygtjmG6eOJ9foK6YZEzYHjrRmR0DN8sjSgFYIR6cXEPxTSHb1KbbIrjbyEhpZ3X3wdKzBzz8I/s640/TonsleyMeshExtenderLocations.JPG" width="640" /></a></div>
<br />
The glass treatment means that very little signal can actually get out of the building, bounce of something, and find its way back in. We estimate the attenuation to be about 15dB at 923MHz, so a total of 30dB loss to pass out of the windows, and back in again, not counting any free space losses etc. Thus we suspect that the links between the units are due to internal reflections in the building, and perhaps a bit of common mode radiation along metal parts of the building interior. It is almost certainly not due to transmission through the ~30 - 50 cm thick reinforced concrete floor decks of the building.<br />
<br />
Given that we are in a building with such poor RF propagation properties, it is rather pleasing that we are able to get any link at all between the first and fourth floors, given the multiple concrete decks and other obstacles that the link has to face.<br />
<br />
So, with the network in place, it was time to do a little initial testing with it. For this, I had a phone running Serval Chat in the lab, and I went down to level 1 by BCA306B0 and sent text messages, using the automatic delivery confirmation mechanism to know that a message has got to the other end, and indeed that a bundle has made its way back containing the acknowledgement.<br />
<br />
First, we tested using the old version of LBARD, and were able to see the failure of messages to deliver that we had seen in the field. Then it was a visit to all the Mesh Extenders to update LBARD (we will in the near future supply them all with ethernet connections back to the lab, so that we can remotely upgrade them). The result after upgrading LBARD was messages were reliably delivering, taking between about 1 and 4 minutes to get the delivery confirmation. This was of course a very pleasing and comforting result, as the inability to reliably deliver messages was the most critical issue we were seeing in Vanuatu.<br />
<br />
The large variation in time we expect to be simply due to the very high rates of packet loss that we see on the link between BCA306B0 and 1A15ED32 -- something that we will confirm was we continue testing. The main thing is that we now have a test environment that is both convenient, and realistic enough, for us to reproduce bugs, and confirm that they have been fixed.<br />
<br />Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com7tag:blogger.com,1999:blog-9068638994894102968.post-51100644616648928982018-04-01T00:03:00.002-07:002018-04-01T00:03:28.761-07:00Fixing bugs and structure of LBARDDuring our trips to Vanuatu last year we identified a number of bugs with LBARD, which have been sitting on the queue to be fixed for a while. <br />
<br />
At the same time, we have had a couple of folks who have tried to add support for additional radio types to LBARD, but have struggled due to the lack of structure and documentation in the LBARD source code.<br />
<br />
Together, these have greatly increased the priority of fixing a variety of things in LBARD. So over the past few days I have started pulling LBARD apart, taking a look at some preliminary refactoring by Lars from Germany, and tried to improve the understandability, maintainability and correctness of LBARD. This has focussed on a few separate areas:<br />
<br />
<h3>
Generally improving the structure of the source code</h3>
The most obvious change here is that the source files now live in a set of reasonably appropriately named sub-directories, to make everything easier to find. This is supported by the factoring out of a bunch of functionality into the radio drivers and message handlers described below. <br />
<h3>
Drivers for different packet radio types are now MUCH easier to write. </h3>
Previously you had to hunt through the source code to find the various places where the radio-specific code existed, hoping you found them all, and worked out how they hang together. This also made maintenance of radio drivers more fragile, because of the multiple files that had to be maintained and that could give rise to merge conflicts.<br />
<br />
In contrast, the process now consists of creating a header and source file in <span style="font-family: "Courier New", Courier, monospace;">src/drivers/</span>, that implement just a few functions, and has a single special header line that is used by the Makefile to create the necessary structures to make the drivers usable. <br />
<br />
The header file is the simplest: It just has to have prototypes for all of the functions you create in the source file.<br />
<br />
The source file is not much more complex, excepting that you have to implement the actual functionality. There are a few assumptions, however, primarily that the radios will all be controlled via a UART. Using the RFD900 driver as an example, let's quickly go through what is involved, beginning with the magic comment at the start:<br />
<br />
<b><span style="font-size: x-small;"><span style="font-family: "Courier New", Courier, monospace;"><br />/*<br />The following specially formatted comments tell the LBARD build environment about this radio.<br />See radio_type for the meaning of each field.<br />See radios.h target in Makefile to see how this comment is used to register support for the radio.<br /><br />RADIO TYPE: RFD900,"rfd900","RFDesign RFD900, RFD868 or compatible",rfd900_radio_detect,rfd900_serviceloop,rfd900_receive_bytes,rfd900_send_packet,always_ready,0<br /><br />*/</span></span></b><br />
<br />
In order for the compilation to detect the driver and make it available, it requires a line that begins with <b><span style="font-family: "Courier New", Courier, monospace;">RADIO TYPE:</span></b> and is followed by the elements for a struct radio_type record, the details of which are found in <span style="font-family: "Courier New", Courier, monospace;">include/radio_type.h.</span> You should provide here:<br />
<br />
1. the unique radio type suffix (this will get <b><span style="font-family: "Courier New", Courier, monospace;">RADIOTYPE_</span></b> prefixed to it, and #defined to a unique value by the build system).<br />
2. The short name of the radio type to appear in various messages.<br />
3. A longer description of the radio type.<br />
4. An auto-detect routine for the radio.<br />
5. The main service loop routine for the radio. LBARD will call this for you on a regular basis. You have to decide when the radio is ready to send a packet, as well as to manage any session control, e.g., link establishment for radio types that require it.<br />
6. A routine that is called whenever bytes are ready to be received from the UART the radio is connected to.<br />
7. A routine that when called transmits a given packet.<br />
8. A routine that returns 1 when the radio is ready to transmit a packet.<br />
9. The turn-around delay for HF and other similar radios that take a considerable time to switch which end is transmitting.<br />
<br />
These drivers are actually quite simple to write in the grand scheme of things. Even the RFD900 driver, which implements a congestion control scheme and an over-complicated transmit power selection scheme requires is still less than 500 lines of C. The current proof-of-concept drivers for the Barrett and Codan HF radios are less than 300 lines each.<br />
<br />
What hasn't been mentioned so far is that if you want your radio driver to get exercised by LBARD's test suite, you also need to add support for it to <span style="font-family: "Courier New", Courier, monospace;">fakecsmaradio</span>, LBARD's built-in radio simulator. This simulator implements a number of useful features, such as the ability to implement unicast and broadcast transmission, the ability to detect packet collisions and cause loss of packets, adjustable random packet loss and a flexible firewall language that makes it fairly easy to define even quite complex network topologies. The drivers for this are placed in <span style="font-family: "Courier New", Courier, monospace;">src/drivers/fake_</span><i>radiotype</i><span style="font-family: "Courier New", Courier, monospace;">.h</span>, and will be described in more detail in a future blog post.<br />
<br />
<h3>
LBARD Messages are now each defined in a separate source file</h3>
Whereas previously the code to produce messages for insertion into packets, and the code to parse and interpret them were found scattered all over the place, they now all live in a source file each in <span style="font-family: "Courier New", Courier, monospace;">src/messages/</span>. For further convenience, these files are automatically discovered, along with the message types they contain, and hooked into the packet parsing code. This makes creation of message parsing functions quite trivial: One just needs to create a function called message_parser_XX, where XX is the upper-case hexadecimal value of the message type, i.e., the first byte of the message when encoded in the packet. Where the same routine can be used to decode multiple message types, #define can be used to allow re-use of the function. Apart from performing their internal functions, these routines need only return the number of bytes consumed during parsing. For example, here is the routine that handles four related message types that are used to acknowledge progress during a transfer:<br />
<br />
<b><span style="font-family: "Courier New", Courier, monospace;">#define message_parser_46 message_parser_41<br />#define message_parser_66 message_parser_41<br />#define message_parser_61 message_parser_41<br /><br />int message_parser_41(struct peer_state *sender,char *prefix,<br /> char *servald_server, char *credential,<br /> unsigned char *message,int length)<br />{<br /> sync_parse_ack(sender,message,prefix,servald_server,credential);<br /> return 17; // length of ACK message consumed from input<br />}</span></b><br />
<h3>
Fixing LBARD transfer bugs</h3>
In addition to the structural improvements described above (and in many regards facilitated by them), I have also found and fixed quite a few bugs with Rhizome bundle transfers. The automated tests now all pass, with the exception of the auto-detection of the HF radios, which don't work yet because we are still writing the drivers for <span style="font-family: "Courier New", Courier, monospace;">fakecsmaradio</span> for them. Here are a pair of the more important bugs fixed:<br />
<br />
1. The code for tracking and acknowledging transfers using acknowledgement bitmaps now actually works. This is important for real-world situations where packet loss can be very high. In contrast to Wi-Fi that tries to hide packet loss through repeated low-latency retransmissions, we have to conserve our bandwidth, and so we keep track of what a recipient says that they have received, and only retransmit that content when required. <br />
<br />
This also plays an important role when there are multiple senders, as they listen to each other's transmissions, and use that information to try very hard to make sure that they don't send any data that anyone else has recently sent. This helps tremendously when there are multiple radios in range of one another.<br />
<br />
This bug is likely responsible for some of the randomness in transfer time we were seeing in Vanuatu, where some bundles would transfer in a few seconds, while others would take minutes.<br />
<br />
2. There were some edge-cases that could cause transmission to get stuck resending the last few bytes of a bundle over and over and over, even though they had already been received, and other parts of the bundle still needed sending.<br />
<br />
This particular bug was tickled whenever the length of a bundle, when rounded up to the nearest multiple of 64 bytes was an odd multiple of 64 bytes. In that case, it would mistakenly think that there was 128 bytes there to send, and in trying to be as efficient as possible, send that instead of any single 64 byte piece that was outstanding (since it results in reduced amortised header cost). However, the mishandling of the bundle length meant it would keep sending the last 64 bytes forever, until a higher priority bundle was encountered. <br />
<br />
It is quite possible that this bug was responsible for some of the rest of the randomness in latencies we were seeing, and also the "every other MeshMS message never delivers" bug, where sending one MeshMS would work, sending a 2nd wouldn't, but sending a 3rd would cause both the 2nd and 3rd to be delivered -- precisely because each additional message had a high probability of flipping the parity of the length of the bundle in 64 byte blocks.<br />
<br />
So the net result is that the various tests we had implemented in the test suite, including delivering bundles over multiple (simulated) UHF hops, holding a MeshMS conversation in the face of 75% packet loss, transferring bundles to many recipients or receiving a bundle efficiently from many nearby senders all work just fine. <br />
<br />
The proof of the pudding is in the real-world testing, so I am hoping that we will be able to get outside with a bunch of Mesh Extenders in the coming week and setup some nice multi-hop topologies, and confirm whether we can now reliably deliver MeshMS and MeshMB traffic over multiple UHF radio hops. <br />
<br />
In parallel to this, we will work on the drivers for the HF radios, and also for the RFM96-based Lora-compatible radios that are legal to use in Europe, and get that all merged in and tested. But meanwhile, the current work is all there to see at <a href="https://github.com/servalproject/lbard/">https://github.com/servalproject/lbard/</a>.Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com2tag:blogger.com,1999:blog-9068638994894102968.post-9585879061440700942018-03-21T20:39:00.001-07:002018-03-21T20:39:14.102-07:00Early access to Serval Mesh Extender hardwareA quick post in response to a few enquiries we have had recently, where folks have been asking about getting their hands on some prototype Serval Mesh Extender devices.<br />
<br />
So far, we have only produced 50 of these units, which have our (hopefully) IP65/IP66 injection-moulded case, IP67 power/external radio cable, integrated solar panel controller and battery charger if you wish to operate them off-grid. Most of those are currently deployed in Vanuatu, or are in use in the lab here or with a few other partner organisations.<br />
<br />
Although we are still in the process of testing them, they have already been in some interesting places in Vanuatu and beyond, as the images at the end of the post show.<br />
<br />
Thus, we would like to gauge the interest in us having another production run, so that folks who would like to work with these experimental units, including to help us identify, document and fix any problems with them, can do so.<br />
<b><u><br /></u></b>
<b><u>A reminder and warning: </u>The devices would be <u>experimental</u>, and <u>have known software issues</u> that need to be resolved before they can be sensibly deployed. It is also possible that hardware issues will be found in the process, and that these units may well never be deployable by you in any useful way, i.e., cannot be considered as a finished product or fit for <u>merchantability</u>. Rather, you are being given the opportunity to purchase hardware that would allow you to participate in the ongoing development of these devices.<br /><br />If you want to buy Mesh Extenders to use in a serious way, this is not the offer you are looking for. As soon as we are ready to make them commercially available, we will be sure to let everyone know, and will update this post as well.</b><br />
<br />
Because of the small production size that we would anticipate for this, the per-unit price will likely be quite high -- potentially as high as AU$700 each (and you really need a pair for them to be useful). If we were able to get 100 or more manufactured in a single run, then this cost would come down, quite likely a lot. (But please note, this is not at all representative of the final price of these units when we hopefully bring them to market -- we intend that the end price to be a LOT lower than this. Again, the purpose of this current offer now is to provide early access that would support the finalisation of these devices, and allow us to get a few extra devices made for our internal use at the same time.)<br />
<br />
So, given the above, is there any one else out there who would also like to get their hands on one or more Mesh Extenders, in your choice of Wholly White, Rather Red or Generally Grey?<br />
<br />
(We realise the price at this stage is rather high, and that this might preclude interest from some folks from participating. During this pre-production stage, we have a couple of options to reduce the cost a little: First, if you already own an RFD900 or RFD868 radio, or want a Wi-Fi-only Mesh Extender, you can subtract AU$100 from the price. Second, if you want to make an offer contingent on a production run of at least 100 units, which should get the price down by AU$100 - AU$200 per unit (subject to confirming with our suppliers), you are welcome to indicate this. We will in any case come back to the community before we proceed with any production run.)<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiy1gn7k3waBjFuSjSdBduLB65N24LCK5QKD4PSMi7PU11V56ecdR7R76ZsT78_7_7h17E5g0Hj9zX9xPiigWvqajFN9EaqhRZg52T_6O6UHY0NKtQKIRgxKc2FcVuHv0LxtPgptlIjX-c/s1600/P1210379-Mt-Yasur-Volcano-lava-couldrens-MeshExtender.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiy1gn7k3waBjFuSjSdBduLB65N24LCK5QKD4PSMi7PU11V56ecdR7R76ZsT78_7_7h17E5g0Hj9zX9xPiigWvqajFN9EaqhRZg52T_6O6UHY0NKtQKIRgxKc2FcVuHv0LxtPgptlIjX-c/s400/P1210379-Mt-Yasur-Volcano-lava-couldrens-MeshExtender.JPG" width="300" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjvyB0xY7_nq15-oW1fjUKjF0RE5Qsfhl_O6tFU-4ULEY_l7JO7qxEiWdHoF0ZnNRnkPXTcbzIc4VYMzlfP7LfkvDErmlHFk1KDeV3BpN_7ozRJyGdPwkV8arxEpY1r_U1rNb5z66yDKJo/s1600/P1000413-Mesh-Extender-outdoor-testing.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjvyB0xY7_nq15-oW1fjUKjF0RE5Qsfhl_O6tFU-4ULEY_l7JO7qxEiWdHoF0ZnNRnkPXTcbzIc4VYMzlfP7LfkvDErmlHFk1KDeV3BpN_7ozRJyGdPwkV8arxEpY1r_U1rNb5z66yDKJo/s400/P1000413-Mesh-Extender-outdoor-testing.JPG" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhB2Jq2EWwZOKWrvu_NZmVnBgPn6H6bhxGV3L4kjs0cFCwzn6qyEX0yJcczM1rNitNkKy7VgxvOdArLbKLrBgJZ9PkV-KoTuv3lPiWurIQB0j9Ftyv2FXGyjUxbktFAPjgOIg3ss-HBOp0/s1600/P1200957-Epau-Village-installing-Mesh-Extenders.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhB2Jq2EWwZOKWrvu_NZmVnBgPn6H6bhxGV3L4kjs0cFCwzn6qyEX0yJcczM1rNitNkKy7VgxvOdArLbKLrBgJZ9PkV-KoTuv3lPiWurIQB0j9Ftyv2FXGyjUxbktFAPjgOIg3ss-HBOp0/s400/P1200957-Epau-Village-installing-Mesh-Extenders.JPG" width="300" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVAPiDFtY5i6mrVuT9hzi_HTSeAhI_ISlR7xhQ6N5MuHeebBNa0RgklLe6dixKxpB7BHCjg8IGCZwpN6hdkt0l5-S17ASjfpK_3SSvhFkLIK_I1riwNMwrI2Z76-EJj8k-RI78GXTeGhs/s1600/P1200971-Epau-Village-installing-Mesh-Extenders-Epau-Village-installing-Mesh-Extenders-up-a-tree.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVAPiDFtY5i6mrVuT9hzi_HTSeAhI_ISlR7xhQ6N5MuHeebBNa0RgklLe6dixKxpB7BHCjg8IGCZwpN6hdkt0l5-S17ASjfpK_3SSvhFkLIK_I1riwNMwrI2Z76-EJj8k-RI78GXTeGhs/s400/P1200971-Epau-Village-installing-Mesh-Extenders-Epau-Village-installing-Mesh-Extenders-up-a-tree.JPG" width="300" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhwI9wCBkRAzBKtBiyInttEZCKktpTSAl0uMwHO5Qu8waZtWFeqF8EQnZGhMJEv-rxzE_0pafNeBuy4Fue2tN5MhBpe4pxTod9NgxZ_n1uQjVUMh_pU5fJU_Z_qvQODzhiAp9WLt0GlTM/s1600/P1200432-PangPang-Village-MeshExtender-on-Roof.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhwI9wCBkRAzBKtBiyInttEZCKktpTSAl0uMwHO5Qu8waZtWFeqF8EQnZGhMJEv-rxzE_0pafNeBuy4Fue2tN5MhBpe4pxTod9NgxZ_n1uQjVUMh_pU5fJU_Z_qvQODzhiAp9WLt0GlTM/s400/P1200432-PangPang-Village-MeshExtender-on-Roof.JPG" width="300" /></a></div>
<br />Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com8tag:blogger.com,1999:blog-9068638994894102968.post-91666189204663614712018-02-22T12:31:00.004-08:002018-02-22T12:31:53.646-08:00Getting back up to speed for 2018It's a been a while since the last post, in part due to summer holidays here in Australia, and in part due to various other things taking my time and energy. But, we are now ramping back up for 2018. There are a few interesting things going on that are worth commenting on:<br />
<br />
1. Porting Serval to Ubuntu Touch<br />
<br />
A group has formed to port Serval to <a href="https://ubports.com/ubuntu-touch/about-ut">Ubuntu Touch</a>, in their own words:<br />
<br />
"The Serval Mesh for UBports group is dedicated to the task of launching a fully functional Serval app for Ubuntu Touch and aiding in the expansion of community-built mesh networks."<br />
<br />
and<br />
<br />
"Hi! We are busy making Serval Mesh work on UT and it's going WELL (they say)"<br />
<br />
In just a couple of days, they already have servald running on the phone (this is one of the nice things about Ubuntu Touch basically being a port of Ubuntu Linux, and thus brings with it many benefits of convergence, i.e., where something made for one platform works on the other, possibly without any changes. Here are some early screenshots showing their current progress (which, given they have been working on it for only two days, is very nice):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjqEtHUFVatCkT6Sa71nr8KjHEP32YYc30F4q6WWGPMdE5dF1D402IhMfdOSrGvpILVQkv1v-zs0JriePGLUnlipFxODkwZm9nvDNjnUzXr0VL8FjidjSphenBNAe8hUgcln4ohDI0HfCg/s1600/screenshot20180222_121249319.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="900" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjqEtHUFVatCkT6Sa71nr8KjHEP32YYc30F4q6WWGPMdE5dF1D402IhMfdOSrGvpILVQkv1v-zs0JriePGLUnlipFxODkwZm9nvDNjnUzXr0VL8FjidjSphenBNAe8hUgcln4ohDI0HfCg/s400/screenshot20180222_121249319.png" width="225" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjafw67p2sQy62dG_FR9qp1b1zv_QlHeXIbGAUlcGHgsTety4oWKqhXAdaiw-ywJoxIMdyIYYIlwSfy8hx7m046Hnkkhp66ciFv70JOH8ABdPrSNcqtNKtPU_Enuc2od94D5AWWItQHhJ0/s1600/screenshot20180222_121252625.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="900" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjafw67p2sQy62dG_FR9qp1b1zv_QlHeXIbGAUlcGHgsTety4oWKqhXAdaiw-ywJoxIMdyIYYIlwSfy8hx7m046Hnkkhp66ciFv70JOH8ABdPrSNcqtNKtPU_Enuc2od94D5AWWItQHhJ0/s400/screenshot20180222_121252625.png" width="225" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGIDxh0tuSbGBAMPfgm6NMmy8SmMJ87xPUk2lCz4edrXt8xvgpwgZkSI-mv6uf9rGAmSGcR-37Q9nKhRbsiZwxXVEHJO9tN1F-0aRBjs3TdmN-JPk9qN6lYigRmI3kX8_c1QfVHioGkms/s1600/screenshot20180222_121312069.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="900" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGIDxh0tuSbGBAMPfgm6NMmy8SmMJ87xPUk2lCz4edrXt8xvgpwgZkSI-mv6uf9rGAmSGcR-37Q9nKhRbsiZwxXVEHJO9tN1F-0aRBjs3TdmN-JPk9qN6lYigRmI3kX8_c1QfVHioGkms/s400/screenshot20180222_121312069.png" width="225" /></a></div>
In that last one, you can also just see the cat's ears poking up in the bottom of the list of apps, as well :)<br />
<br />
Currently the list of devices that support Ubuntu Touch is relatively limited. That said, my FairPhone 2 is on the supported list, but I am not sure if I will switch over just yet. I might just get a second FP2 for testing for now. The FP2 is of course the perfect phone for Ubuntu Touch in my view, as it is an open-hardware phone as far as is possible, and allows for end-user servicing -- very much in line with the Linux ethos.<br />
<br />
2. Outernet integration<br />
<br />
This is a project supported by the Humanitarian Innovation Fund (HIF) out of the UK. While we have a couple of roadblocks to solve, as a result of Outernet moving back from L-band (~1.5GHz) to Ku-Band (~12GHz), the end result will be VERY exciting, in part because the bandwidth will be higher, and in part because it looks like we will be able to incorporate the entire receiver into the existing Mesh Extender cases, thus offering a single unit that does both functions. The new Ku receiver antennae are also nicely shaped to allow for strapping to a tree pointing in the general direction of the satellite. <br />
<br />
More to come on this as it progresses.<br />
<br />
3. We have our next cohort of excellent students from INSA Lyon, France, arriving as I type. They will be working on a number of very useful sub-projects, including bedding down the HF radio support we first demonstrated a couple of years ago, fixing bugs we identified in Vanuatu last year and so on. I am VERY excited about seeing progress on these fronts, so that Serval Mesh Extenders can be usefully deployed by our various partners in a variety of ways.<br />
<br />
Paul.Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com4tag:blogger.com,1999:blog-9068638994894102968.post-16569540435390974082017-11-23T15:16:00.003-08:002017-11-23T15:16:37.319-08:00Android has been a bit naughty with its location trackingI was pointed to this article today:<br />
<br />
<a href="https://qz.com/1131515/google-collects-android-users-locations-even-when-location-services-are-disabled/">https://qz.com/1131515/google-collects-android-users-locations-even-when-location-services-are-disabled/</a><br />
<br />
Basically it points out that Android has been tracking location of phones for the past year or so, even when location tracking is disabled. More specifically, it tells Google whenever you come in range of a cell tower. By doing this for each cell tower a phone can hear, can provide a fairly good location, especially if you integrate it over time.<br />
<br />
The use of spyware in mobile devices is a topic we have talked about previously, both for people living in dangerous places, as well as for victims of domestic violence and other contexts where being able to locate someone further compounds their vulnerability and tips the power-imbalance in the favour of an abusive person, organisation or otherwise.<br />
<br />
The really naughty part in this current situation, is that this was happening even without a SIM card in the phone, and even when location services were disabled in Android: There was no way to know it was happening, and no way to disable it, even if you knew. In fact, Google realised it was naughty by more or less immediately phasing it out as soon as they had been called out on it.<br />
<br />
This leads me to a topic that we have been quietly working on in the background for the past couple of years, that is, how can we trust modern computers and communications devices, when they are so complex that it almost requires accidental discovery by dedicated researchers to find these significant privacy and safety damaging functions, which have been silently introduced to our devices -- often through software updates long after the initial purchase.<br />
<br />
Our response to this is to explore the creation of "simply secure" communications devices, i.e., communications devices so simple, that their security can be quickly and confidently audited by a reasonably determined user, rather than requiring a team of researchers to explore. Such devices should also make it much easier to be assured that the device cannot communicate with the outside world -- including getting a location fix -- when you don't want it to. <br />
<br />
Such devices are easy to make. After all, a brick is a secure communications device, in that there isn't really any way to subvert the function of a lump of burnt clay. But it isn't useful. This is the opposite extreme from current devices, that are almost omnipotent, but are so easy to subvert.<br />
<br />
The challenge is to design and create devices that sit on some sweet spot in the middle, where they are still simple enough to be confident in their correct function, yet not so simple as to be practically useless.<br />
<br />
This is exactly the kind of device that we are currently designing, in the form of a specialised smart-phone, that will still be capable of secure email, telephone calls, SMS and so on, while being much more resistent to attack or subversion, due to its simplicity and transparent auditability. <br />
<br />
For example, it will have physical switches to power off the cellular modem, and the cellular modem will be completely sandboxed from the rest of the phone -- including the GPS receiver, microphone and so on. Many of these modules will also be completely removable.<br />
<br />
It will also allow full out-of-band memory inspection of the entire system, transparent to, and independent of the processor, and provide a secure compartmentalised architecture that allows a paranoid process, for example an email decryption program, to be sure that even the hypervisor cannot interrupt it to exfiltrate private information.<br />
<br />
We know that there are some other folks active in similar spaces, including the excellent folks at <a href="https://puri.sm/">Purism</a>. We love what they are doing, and see our thinking in this space as complementary. The Purism laptops (and soon phone) use all open-hardware, so that if you need a full-function computer, it is as trust-worthy as possible. What we are looking to do is a little different: We want to see how simple we can go, while preserving enough function to be useful. We are expecting the core operating system to fit in kilo-bytes of memory, not mega-bytes, and applications to be tens to hundreds of kilo-bytes, not mega-bytes. <br />
<br />
There are lots of questions unanswered, not the least whether the thing will actually be useful enough for anyone, but we are exploring, and all going well, hope to be able to produce a few prototype devices by the end of 2018. We have also secured the necessary defence-related export clearance for such a device, precisely because its combined security measures place it in risk of tipping over into the category of dual-use equipment, so we have a green light there.<br />
<br />
So my questions for all of you reading:<br />
<br />
<br />
<ol>
<li>Would any of you buy a "phone for the paranoid" along the lines of what I am describing?</li>
<li>What are the absolute core functions that you would require, compared to the list below:</li>
<ul>
<li>Make and receive telephone calls (en claire, and quite possibly end-to-end encrypted).</li>
<li>Send and receive SMS messages (en claire or encrypted).</li>
<li>Send and receive Email, including GPG or similar encrypted.</li>
<li>Very basic web browsing, using a purposely cut-down browser.</li>
<li>Ability to run 3rd-party apps in a sand-box environment.</li>
</ul>
</ol>
<br />
<br />
<br />Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com22tag:blogger.com,1999:blog-9068638994894102968.post-10257344722033745822017-10-17T17:41:00.000-07:002017-10-17T17:41:05.256-07:00Setting up Mesh Extender capability within NZ Red CrossI am briefly in Wellington, NZ, visiting NZ Red Cross on my way to the Global Humanitarian Technology Conference where we have a bunch of papers to present at the end of the week.<br />
<br />
One purpose of the visit was to update the firmware on the Mesh Extenders we had previously provided NZ Red Cross with, and to transfer the knowledge of how to flash the Mesh Extenders to their IT & Telecomms Emergency Response Unit (IT&T ERU), so that they can do it themselves in the future.<br />
<br />
As the ERU does not normally carry laptops running Linux, we found an old disused laptop, and installed Ubuntu on it, and replicated the build and flashing environment from my laptop.<br />
<br />
The important parts were to setup a TFTP server on the laptop, copy the firmware files in there, and clone the <a href="https://github.com/servalproject/openwrt-packages/">Mesh Extender openwrt-packages repository from github</a>, checkout the MeshExtender2.0 branch, compile the auto-flash program.<br />
<br />
After that, it is just a case of running the auto-flash program with a USB to serial adapter connected to a specially made adapter cable, and connecting the Mesh Extenders and watching the output of auto-flash to see when a unit has been flashed.<br />
<br />
Natalie from the ERU was super-helpful being our guinnea pig, and also in documenting the process. Hopefully we will get the documentation up on the wiki in the near future, at which point I will link to it from this post.<br />
<br />
But in the meantime, the following photo shows the completed kit, with the USB serial adapter cable, ethernet cable for TFTP, USB memory stick with Ubuntu so that it can be cloned to other laptops in the future, all in a fashionable marigold laptop case. <a href="https://en.wikipedia.org/wiki/Cat_(Red_Dwarf)">The Cat</a> may object wearing gold and marigold at the same time, but we are quite happy with the result for now.<br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHpLx102UVWEXg_Y-TEioc9roKxhRtAtYl0QQm0s4oZ6uDsIJqR7q_1wZ2sONQnX_t3OaFFSTlu6CBHdTcqhT8rsUVIxGHrivc7saqBD7DMj4Umft9l9Gl58EQYb0OoH3GI4PSnkKUlkM/s1600/IMG_20171018_105157-NZRC-ERU-MeshExtender-Flashing-Kit.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHpLx102UVWEXg_Y-TEioc9roKxhRtAtYl0QQm0s4oZ6uDsIJqR7q_1wZ2sONQnX_t3OaFFSTlu6CBHdTcqhT8rsUVIxGHrivc7saqBD7DMj4Umft9l9Gl58EQYb0OoH3GI4PSnkKUlkM/s400/IMG_20171018_105157-NZRC-ERU-MeshExtender-Flashing-Kit.jpg" width="300" /></a></div>
<br />Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com0tag:blogger.com,1999:blog-9068638994894102968.post-88531546607409466902017-10-08T02:08:00.002-07:002017-10-08T02:08:58.952-07:00Pandanus cable ties and a Mesh Extender treeWe are now in the process of installing Mesh Extenders into Epau, the second of the two villages we are targeting here on the island of Efaté in Vanuatu. As with Pang Pang, the community have been very gracious and enthusiastic in working with us.<br />
<br />
It is always interesting and educational to watch how the folks here go about installing Mesh Extenders, in ways that are appropriate for them, rather than what we might naturally think of in an infrastructure-rich first-world context.<br />
<br />
This series of images follows the process of installing a Mesh Extender in Epau. After talking with the community, they decided that this Mesh Extender would be better mounted in a tree next to the house, than on the house itself.<br />
<br />
First step: Attach the Mesh Extender to a long bamboo pole, which is a convenient locally available material:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFrsW2QlIEtjtQt1SECG6nzfd1tajJPyzImT3w6es8w9-jc42XeQRmCeOAALh8WpmnNzFjF4bwDVQyZ46nCTGjoKwZapv0G69fH1et-IukV5UmWhpBauob9X1vp3T60z7MaaKZcwfdid0/s1600/P1200956-Epau-Village-installing-Mesh-Extenders.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFrsW2QlIEtjtQt1SECG6nzfd1tajJPyzImT3w6es8w9-jc42XeQRmCeOAALh8WpmnNzFjF4bwDVQyZ46nCTGjoKwZapv0G69fH1et-IukV5UmWhpBauob9X1vp3T60z7MaaKZcwfdid0/s400/P1200956-Epau-Village-installing-Mesh-Extenders.JPG" width="300" /></a></div>
<br />
Second step: If you don't have enough cable ties, make a make-shift cable-tie from a dried Pandanus leaf (the same leaves that the Vanuatuans use to weave mats, bags and other useful things):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjP03Qg0DWHwy5OU2zHkZQahPlFjqIxKOFrjnvezZaJgt_L-NzVyt5OHbtenX2NZQi7fFMTeBB2EJpy2hAZdqoaPozIOKewgN9JP8EMosn9VT8Ep7VhvxfHEe2DLJkHjVpzRog2z7mYwv4/s1600/P1200957-Epau-Village-installing-Mesh-Extenders.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjP03Qg0DWHwy5OU2zHkZQahPlFjqIxKOFrjnvezZaJgt_L-NzVyt5OHbtenX2NZQi7fFMTeBB2EJpy2hAZdqoaPozIOKewgN9JP8EMosn9VT8Ep7VhvxfHEe2DLJkHjVpzRog2z7mYwv4/s400/P1200957-Epau-Village-installing-Mesh-Extenders.JPG" width="300" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
Here you can see it closer up, with the light-brown Pandanus leave around the lower part of the Mesh Extender:</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjk5RPe0fr2FPEFDcG4OUFJLeVsPeTwgIjSwkl5T9mZ_iqGVJH50y2CEdKxAOC5UWDIM_QcPbIfRMmLFHttGGYNBkvrx48LrHT_ms82JKPSa9dAiAtTlNHH7sYI35358hc0C_4twGDn8M4/s1600/P1200959-Epau-Village-installing-Mesh-Extenders-Pandanus-strap.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjk5RPe0fr2FPEFDcG4OUFJLeVsPeTwgIjSwkl5T9mZ_iqGVJH50y2CEdKxAOC5UWDIM_QcPbIfRMmLFHttGGYNBkvrx48LrHT_ms82JKPSa9dAiAtTlNHH7sYI35358hc0C_4twGDn8M4/s400/P1200959-Epau-Village-installing-Mesh-Extenders-Pandanus-strap.JPG" width="300" /></a></div>
<br />
Third step: Explore the tree to work out how to get the > 6 metre long bamboo pole up there, and firmly attached:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHdoomamhmAcx701DZFUzwovbdGsSd6iXfu7atKT47qVuNAZf2ggK0Bk74GGaotJsb9BYwkPL3b-baz4cdt_uKQX9hlxLFvZwSHjyFFdlD4gh8eCJUZJMOiO-5YmVsfvMHDbkOcJfEYaU/s1600/P1200963-Epau-Village-installing-Mesh-Extenders-Epau-Village-installing-Mesh-Extenders-up-a-tree.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHdoomamhmAcx701DZFUzwovbdGsSd6iXfu7atKT47qVuNAZf2ggK0Bk74GGaotJsb9BYwkPL3b-baz4cdt_uKQX9hlxLFvZwSHjyFFdlD4gh8eCJUZJMOiO-5YmVsfvMHDbkOcJfEYaU/s400/P1200963-Epau-Village-installing-Mesh-Extenders-Epau-Village-installing-Mesh-Extenders-up-a-tree.JPG" width="300" /></a></div>
<br />
It didn't take very long for said person to disappear even higher up the tree:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHHZVzaFLgTz-cn9S3T7c3XePhYADgObIDVbWg8LXiSpKaDyWZjZqwMq8udcAQx23h4wVQwOMKuH7EnDHAFxDggEJzWTohign891c0TU6-NN-FeH-55wkGwBbTxuFR4P5kmaW0GAQu1Kg/s1600/P1200965-Epau-Village-installing-Mesh-Extenders-Epau-Village-installing-Mesh-Extenders-up-a-tree.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHHZVzaFLgTz-cn9S3T7c3XePhYADgObIDVbWg8LXiSpKaDyWZjZqwMq8udcAQx23h4wVQwOMKuH7EnDHAFxDggEJzWTohign891c0TU6-NN-FeH-55wkGwBbTxuFR4P5kmaW0GAQu1Kg/s400/P1200965-Epau-Village-installing-Mesh-Extenders-Epau-Village-installing-Mesh-Extenders-up-a-tree.JPG" width="300" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
Then the pole was passed up, and it wasn't long until the Mesh Extender tree flowered: </div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkYFow1BXOlk_6fljBepl7AGHOT-Qh0oBr8IntIEMp-PX8mRTAW0g3X-aTYB0HRWSKGNFEodQvuYbFHy5iUZoC5wjB3dcT-wIkx3lmwKFe7S3k-uMNNt0NFByWXpVI2acR3VPNCx7wilE/s1600/P1200968-Epau-Village-installing-Mesh-Extenders-Epau-Village-installing-Mesh-Extenders-up-a-tree.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkYFow1BXOlk_6fljBepl7AGHOT-Qh0oBr8IntIEMp-PX8mRTAW0g3X-aTYB0HRWSKGNFEodQvuYbFHy5iUZoC5wjB3dcT-wIkx3lmwKFe7S3k-uMNNt0NFByWXpVI2acR3VPNCx7wilE/s400/P1200968-Epau-Village-installing-Mesh-Extenders-Epau-Village-installing-Mesh-Extenders-up-a-tree.JPG" width="300" /></a></div>
<br />
The power lead was fed back down through the tree:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhSc6gAWDCCZOqw2o2HMkTtSXeq3hOgIAVBjFQSuVuanIs7DOVuoZgGs6JW8UXJJwD3zRJZjp5YDVXjv1r8iZ6k624TCKVTS5PSrNN_qrltmTICXH4dveuh6c6bYIsLRY-O2vIYgtn7os/s1600/P1200969-Epau-Village-installing-Mesh-Extenders-Epau-Village-installing-Mesh-Extenders-up-a-tree.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhSc6gAWDCCZOqw2o2HMkTtSXeq3hOgIAVBjFQSuVuanIs7DOVuoZgGs6JW8UXJJwD3zRJZjp5YDVXjv1r8iZ6k624TCKVTS5PSrNN_qrltmTICXH4dveuh6c6bYIsLRY-O2vIYgtn7os/s400/P1200969-Epau-Village-installing-Mesh-Extenders-Epau-Village-installing-Mesh-Extenders-up-a-tree.JPG" width="300" /></a></div>
<br />
And the Mesh Extender lofted a bit higher, to ensure it was well clear of the foliage (and hopefully will remain so for a few months before needing adjustment, due to tree growth):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJweYG597sT59cVJq4uX8Qdy8tyJSB37gYxBRt-aBjPSWk6WVv0mKWaeD2-NoISfsdiGKidNNTU27Sa6B_ocBI7DHRhyJD_Y70TNqsik7bysJuij5ZCmL_Fag5YHnx7tSYsEMsXli_t_0/s1600/P1200971-Epau-Village-installing-Mesh-Extenders-Epau-Village-installing-Mesh-Extenders-up-a-tree.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJweYG597sT59cVJq4uX8Qdy8tyJSB37gYxBRt-aBjPSWk6WVv0mKWaeD2-NoISfsdiGKidNNTU27Sa6B_ocBI7DHRhyJD_Y70TNqsik7bysJuij5ZCmL_Fag5YHnx7tSYsEMsXli_t_0/s400/P1200971-Epau-Village-installing-Mesh-Extenders-Epau-Village-installing-Mesh-Extenders-up-a-tree.JPG" width="300" /></a></div>
<br />
The tree itself is not that small, either: The crown would be at least six metres tall, on top of a few metres of relatively bare trunk below:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAL6fpJcHTJMBC2I8PDzIU30bQ7Ot7vaQMHaVH7D8sDc-aAH7Tj9KBhskA_7RsJxA0B8Nc-s3Ba5URWp1WHwYmCbFnpLvL3-NxfqIlCT5ypVUiS5_r4xAkcdU-chMcuJFrz1q39Azux3k/s1600/P1200972-Epau-Village-installing-Mesh-Extenders-Epau-Village-installing-Mesh-Extenders-up-a-tree.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAL6fpJcHTJMBC2I8PDzIU30bQ7Ot7vaQMHaVH7D8sDc-aAH7Tj9KBhskA_7RsJxA0B8Nc-s3Ba5URWp1WHwYmCbFnpLvL3-NxfqIlCT5ypVUiS5_r4xAkcdU-chMcuJFrz1q39Azux3k/s320/P1200972-Epau-Village-installing-Mesh-Extenders-Epau-Village-installing-Mesh-Extenders-up-a-tree.JPG" width="240" /></a></div>
<br />
With about 1.5 metres of clearance, I'd estimate that this Mesh Extender is about 10 metres above the ground:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRPnlT2yA0U-BxjLVOh92UEA6W2xXoir9mKbl0uSshb0AUCi7YGqDaY6XKdlglTfTSuY2JgIjpTIHgWJwRs5NpdE8_VFHsuEpVy3nQPuHllT35pgmM48XJmCGn5DDa0cDpuhJF38JP-C8/s1600/P1200994-Epau-Village-installing-Mesh-Extenders-up-a-tree.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRPnlT2yA0U-BxjLVOh92UEA6W2xXoir9mKbl0uSshb0AUCi7YGqDaY6XKdlglTfTSuY2JgIjpTIHgWJwRs5NpdE8_VFHsuEpVy3nQPuHllT35pgmM48XJmCGn5DDa0cDpuhJF38JP-C8/s400/P1200994-Epau-Village-installing-Mesh-Extenders-up-a-tree.JPG" width="300" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
What I really love about this installation, is that it was made using local skills, local materials and knowledge: I have every confidence that the community can take it down when a cyclone comes, and then put it back up again, and that they won't be stymied by the lack of things that they need to order from somewhere. </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
All week my children have been accusing me of making "Dad jokes", so I'll continue in that vein here, by calling this installation a blooming success.</div>
<br />
Hopefully in the next few days we will be able to get this community installing and using the app on their phones, and get a few more Mesh Extenders installed around the village.Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com0tag:blogger.com,1999:blog-9068638994894102968.post-62289387634157027082017-10-06T21:58:00.001-07:002017-10-06T21:58:25.621-07:00Third Visit to VanuatuThe past month has been flat out, and I have spent more of it in Vanuatu than at home in Australia. I am now back in Vanuatu on our fourth visit, and figured I really had better write about our third visit, before I get even further behind, especially since my wife has <a href="https://www.cleverchameleon.com.au/everyday-inspiration-pandanus-fruit/">already started blogging about this fourth visit</a> on her blog.<br />
<br />
One of the things that has delayed me blogging more has been the highly variable nature of mobile internet we are experiencing here in Vanuatu. It is possible that part of the problem is that our mobile phones <a href="https://www.frequencycheck.com/country-compatibility/yb5Lv/vanuatu/devices?utf8=%E2%9C%93&q%5Bfull_name_cont%5D=FairPhone&q%5Bdevice_brand_id_eq%5D=&commit=Search">don't support the local 4G frequency</a>, however that shouldn't stop the 3G from working. What we often see is that the data signal seems to disappear, or even when it is there, the throughput drops to zero, or very close to it. Basically between 6 am - midnight it is pot luck as to whether the internet will work at our accommodation, despite having full signal strength, and being able to eye ball the local phone tower. I might have to ask our in-country contacts in the telecommunications industry for clues on this at some point. But in the meantime, it means that our opportunities to upload photos and generally edit blog posts are severely impaired, as you can sense from my <a href="https://www.cleverchameleon.com.au/everyday-inspiration-pandanus-fruit/">wife's frustrations</a>. Indeed, here I am at 1:30am writing this post, for exactly this reason.<br />
<br />
[Update: For some reason our phones won't do 3.5G / HSPDA+. It isn't a FairPhone2 specific problem, as Matthew's completely different phone does the same thing. We have since bought a cheapish Samsung from a local store, which does 3.5G (and soon 4G) on the local network without problem, largely solving our internet problem.]<br />
<br />
Of course, this is all further motivation for us to get Serval working, so that communities for whom even annoyingly variable internet is only a pipe-dream. And let's not even mention the spate of cyclones, earthquakes and naughty volcanoes around the place that also keep reminding us of why we are working on this problem.<br />
<br />
But, first, lets go over what we got up to on the previous visit here to Vanuatu.<br />
<br />
<h3>
Working with the Smart ICT Sistas</h3>
<div>
As part of our efforts to engage all aspects of community in the pilot project, we ran a number of sessions with the Smart ICT Sistas, a school-age group for girls and young women to develop an interest and skills in ICT. This was really interesting from a number of fronts. </div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfvfqkA9gmI8NqM3tJD_fRUiHhCeWWOIC2wCdIDbVNZpfzkgi3cj06mMWp0JxF_Cb98jzL9f8Iezok9x5RqDGR9pK2cGsOTqxizIVPnRGABOVLi_Z8LxyHwKx1sqs0Gl2NvQUDNKMkXoI/s1600/P1000938-Smart-ICT-Sistas-Session-1-Girl-Holding-Mesh-Extender-by-Vanuatu-Flag.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto; text-align: center;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfvfqkA9gmI8NqM3tJD_fRUiHhCeWWOIC2wCdIDbVNZpfzkgi3cj06mMWp0JxF_Cb98jzL9f8Iezok9x5RqDGR9pK2cGsOTqxizIVPnRGABOVLi_Z8LxyHwKx1sqs0Gl2NvQUDNKMkXoI/s400/P1000938-Smart-ICT-Sistas-Session-1-Girl-Holding-Mesh-Extender-by-Vanuatu-Flag.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">This particular Smart ICT Sista was especially keen and active, asking lots of good questions, and giving all sorts of things a try.</td></tr>
</tbody></table>
<div>
</div>
<div>
<br /></div>
<div>
It was great to see some young local people engaging with our technology, to help us work out what works, and what does not in the local context. A couple of key items emerged from this process: </div>
<div>
<br /></div>
<div>
(1) Android phones in Vanuatu almost exclusively have enable installation from untrusted sources enabled by the time we get to see them, because sharing apps from phone to phone is a very common occurrence in Vanuatu, as people save on internet costs. As a result, it wasn't long before we were watching the girls transfer the Serval Mesh app from phone to phone:</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjB9kkjxu_kgNeP7ctEZ5Jw9auzTF9pOaQBTyRtP01tlFUlbHoYw7LMvXX7O63H5mIp-_befeIv5BvqWxgCUXlOrcq-2jaldbIVouIc4xfbuNYyHFyqcX6MrXBHEW1FSP4EAheVAF2G_w/s1600/P1000936-Smart-ICT-Sistas-Session-1.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjB9kkjxu_kgNeP7ctEZ5Jw9auzTF9pOaQBTyRtP01tlFUlbHoYw7LMvXX7O63H5mIp-_befeIv5BvqWxgCUXlOrcq-2jaldbIVouIc4xfbuNYyHFyqcX6MrXBHEW1FSP4EAheVAF2G_w/s400/P1000936-Smart-ICT-Sistas-Session-1.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The Smart ICT sistas getting started with the Serval Mesh</td></tr>
</tbody></table>
(2) We found that the listen before talk settings on the Mesh Extenders were too strict for effective use in urban areas of Vanuatu, where 900MHz point-to-point links are common. We simply would not have found this out, were we not working with the Sistas, and using space provided by the very kind and supportive folks from the Vanuatu national telecommunications regulator (the TRR). Their office is on the 2nd floor in central Port Vila, and seems to be in the direct path of a number of 900MHz point-to-point beams. They also have a very nice spectrum analyser with direction finder antennae, so we were able to confirm that the problem was a very high noise floor (approximately -50dBm) due to directional 900MHz links.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhu6Kw6jAdaz7-hp1_9P0DZbdcZ6ERWQLEaxUolRywlXKJ3_L42g5bbsN7QaWEq3m7YMBLQ83-1iuRDYbfbLrxw4dQMBsfJl9mirEYCoiQpeBR3FOzXChwKNL0jkbVykaV-vTa6qJjx6j8/s1600/P1010152-Smart-ICT-Sistas-TRR-Spectrum-Analyser-900MHz-noise-direction-finder.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhu6Kw6jAdaz7-hp1_9P0DZbdcZ6ERWQLEaxUolRywlXKJ3_L42g5bbsN7QaWEq3m7YMBLQ83-1iuRDYbfbLrxw4dQMBsfJl9mirEYCoiQpeBR3FOzXChwKNL0jkbVykaV-vTa6qJjx6j8/s400/P1010152-Smart-ICT-Sistas-TRR-Spectrum-Analyser-900MHz-noise-direction-finder.JPG" width="400" /></a></div>
<br />
It goes without saying, but Vanuatu is a very pretty place, and we keep on seeing many beautiful and interesting things as we go about our field work here. Here is the local tourist sailing boat that you can pay to be part of the crew of for a few days:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_44u_ewkDk-OFsYKigQhHeES7NRqAiz1jFsLRsIbwKOLRxve7IZD26etVYO4d0Plxi8jsBtdhxF-pWiVh0cUMMYAMPZ3N3oMe_O-j2eDR4QDytW7YNpT_fSlB2YEEYD8LhJgNPSX9ZIg/s1600/P1010188-Square-rigger-sailing-ship-Port-Vila.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_44u_ewkDk-OFsYKigQhHeES7NRqAiz1jFsLRsIbwKOLRxve7IZD26etVYO4d0Plxi8jsBtdhxF-pWiVh0cUMMYAMPZ3N3oMe_O-j2eDR4QDytW7YNpT_fSlB2YEEYD8LhJgNPSX9ZIg/s320/P1010188-Square-rigger-sailing-ship-Port-Vila.JPG" width="320" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
Then it was off to inspect and pay for the 10m mast, that we are hoping to install in Pang Pang, or possibly on top of a hill between Pang Pang and Epau villages, so that we can link them via Mesh Extender:</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAruEgJv7L9PpjsOuuTNtc2QcatEr6cCJhICI5JnqZhL1zqNV37iGYTwfz-ojnJAcyDrYG3GfxSb7JggRGIXQHg-wqJK54dr2DLs1ofweuWm-KbC62v4QTFM03bRz_ncG9g4DkQ8z4SmI/s1600/P1000949-10m-Mast-for-Village.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAruEgJv7L9PpjsOuuTNtc2QcatEr6cCJhICI5JnqZhL1zqNV37iGYTwfz-ojnJAcyDrYG3GfxSb7JggRGIXQHg-wqJK54dr2DLs1ofweuWm-KbC62v4QTFM03bRz_ncG9g4DkQ8z4SmI/s400/P1000949-10m-Mast-for-Village.JPG" width="400" /></a></div>
<br />
I was ably assisted by not only Matthew Lloyd, but also by our two French exchange students, Robin and Raphaël, who also made friends with our neighbours in the SIL compound through their cooking skills.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTgUM8Q0wpk-E-IgNJ2BJbH4RVnGLwsYPRTUcRhD7h7YGnEeiHDpawDeT9IFPhLN6tpyZcK7Pih8NOxObphW87xHX8o0u50rD6KtY36AmvjZD5IN-r0feNejEgBJu2uZA_wYbm4p6599w/s1600/P1000954-Working-on-Software-with-Students.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTgUM8Q0wpk-E-IgNJ2BJbH4RVnGLwsYPRTUcRhD7h7YGnEeiHDpawDeT9IFPhLN6tpyZcK7Pih8NOxObphW87xHX8o0u50rD6KtY36AmvjZD5IN-r0feNejEgBJu2uZA_wYbm4p6599w/s400/P1000954-Working-on-Software-with-Students.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Raphaël, myself and Robin, working on the Mesh Extender firmware.</td></tr>
</tbody></table>
<br />
We also started work on prototyping a satellite-broadcast bridge for Serval, as part of our HIF grant (more on that in up-coming posts). We are building this into a proof-of-concept low-cost tsunami/cyclone early warning system, that would provide weather forecasts and other useful information via Serval year-round, so that they won't become so easily neglected. Our key piece of progress so far, is working out that we can put the 12cm patch antennae inside a poly-carbonate box, and still receive the satellite signal. This will make it much easier to cheaply make well-protected enclosures. We will have to see how it behaves in wet weather, as well.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixYgC4pzfPXRmncJ69d93qa6WO6yk1byXGimxYm8mPE-6WOtVJPHw5L5WiknzyGVOtxEkFCahyNzgjMeKsCeFXOy8SZvWQDpk_Z3IfVAHBvxRt_Y7KsKWkFnkBcXCn0ZTPmJ1WZtuYKnk/s1600/P1000957-Working-on-DreamCatcher-Tsunami-Box.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixYgC4pzfPXRmncJ69d93qa6WO6yk1byXGimxYm8mPE-6WOtVJPHw5L5WiknzyGVOtxEkFCahyNzgjMeKsCeFXOy8SZvWQDpk_Z3IfVAHBvxRt_Y7KsKWkFnkBcXCn0ZTPmJ1WZtuYKnk/s400/P1000957-Working-on-DreamCatcher-Tsunami-Box.JPG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Outernet receiver, with antenna in the grey box. The angle just happens to be close enough with the antenna wedged in the box, and the box sitting flat, which is most convenient.</td></tr>
</tbody></table>
<br />
Then it was time to go around to Pang Pang village again ...<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhU344yFWBJoZtxuzRvbfqMja5j4p2kWbcFiGITGdoXDGo3V2LLtdSJyu6LrHw5fMnX0zQLwSfjGFKnlfrq3DKWENR1_w6TXu2y5Ar3qI6g9R7AcQRGvoj7ptZHhDGMPBKSkDxVKHXxllE/s1600/P1000999-PangPang-House-in-Village.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhU344yFWBJoZtxuzRvbfqMja5j4p2kWbcFiGITGdoXDGo3V2LLtdSJyu6LrHw5fMnX0zQLwSfjGFKnlfrq3DKWENR1_w6TXu2y5Ar3qI6g9R7AcQRGvoj7ptZHhDGMPBKSkDxVKHXxllE/s400/P1000999-PangPang-House-in-Village.JPG" width="400" /></a></div>
<br />
... and go through the University's human ethics approval requirements of obtaining informed concent from the community, before we collect any data. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8WRP5ySzmENqMpsszR2ZoGVckHWWiAhOg6BZajEYknu3VkIGovjohSlFdmUEeSoalxmDMx8tE0L5Eta5COjmxQoMLXB9zuBgaXVHTdyePJgbVLTQWL3vMjXtKOV6SW1HypoDMGq4vkto/s1600/P1010010-PangPang-Pilot-Signing-Group-Shot.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8WRP5ySzmENqMpsszR2ZoGVckHWWiAhOg6BZajEYknu3VkIGovjohSlFdmUEeSoalxmDMx8tE0L5Eta5COjmxQoMLXB9zuBgaXVHTdyePJgbVLTQWL3vMjXtKOV6SW1HypoDMGq4vkto/s400/P1010010-PangPang-Pilot-Signing-Group-Shot.JPG" width="400" /></a></div>
<br />
Places like Pang Pang continue to remind me of the importance that people place on communications. Even though the village itself lacks cellular coverage, there are multiple places you can buy top-up credit for pre-paid phones in the village:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjleF6E1MQosICieGTWFHD9U-VWoKDAm6PJ2nx-rhPobiQchkAyR52ifzKSsYWpMyuvqlBdW1h047ERJ7ARguTZaidSY2YCGHXQc-sS2XL3ATXG1QWpTHDJg3M6xcX67_SDWRoI7lyDV20/s1600/P1010011-PangPang-Digicel-Topup-Here-Sign.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjleF6E1MQosICieGTWFHD9U-VWoKDAm6PJ2nx-rhPobiQchkAyR52ifzKSsYWpMyuvqlBdW1h047ERJ7ARguTZaidSY2YCGHXQc-sS2XL3ATXG1QWpTHDJg3M6xcX67_SDWRoI7lyDV20/s400/P1010011-PangPang-Digicel-Topup-Here-Sign.JPG" width="400" /></a></div>
<br />
Once the formalities were over, it was time to install five Mesh Extenders throughout the village to provide adequate coverage. The goal was to have one Mesh Extender in each of the four distinct parts of the village. So while some people may need to walk a short way to get within wi-fi range of a Mesh Extender, they shouldn't have to go far.<br />
<br />
Here, Donald, the chief's son is installing the solar panel for the Mesh Extender on the chief's house:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVzNa72xjtVeWGM75H884ITu9PEH7MPwwEIkbGKELhjBKoExjxq-gzQtVDzQwN8Nl9NUXf8iDrz-g2CmZH4l0KSQInRkAMkivwKr7tysp1gA-UbAEOE5AASi9N8fsIE8-w8C2kDaHHPmU/s1600/P1010019-PangPang-Putting-Solar-Panel-on-roof.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVzNa72xjtVeWGM75H884ITu9PEH7MPwwEIkbGKELhjBKoExjxq-gzQtVDzQwN8Nl9NUXf8iDrz-g2CmZH4l0KSQInRkAMkivwKr7tysp1gA-UbAEOE5AASi9N8fsIE8-w8C2kDaHHPmU/s400/P1010019-PangPang-Putting-Solar-Panel-on-roof.JPG" width="400" /></a></div>
<br />
Again, the ubiquity of communications and mobile computing devices continues to be amazing. Here the chief is using his Samsung tablet to take a picture of us, while we take a picture of him. Again, remember that there is no cellular coverage in the village.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEil09-Ecg-pYEeSdHnXBH5A5BgPWc3PuClcyhVFQVFTH8q4TIDWJR-lxjLNPoG7AGvg14sSeXYDypwG4ev61wtd6c0xvfx4dmw8rabfcqwA_GQOIHXwid4YCg9JMHBql15Lw4WctAgD8vo/s1600/P1010020-PangPang-chief-filming-us-using-phone-or-tablet.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEil09-Ecg-pYEeSdHnXBH5A5BgPWc3PuClcyhVFQVFTH8q4TIDWJR-lxjLNPoG7AGvg14sSeXYDypwG4ev61wtd6c0xvfx4dmw8rabfcqwA_GQOIHXwid4YCg9JMHBql15Lw4WctAgD8vo/s400/P1010020-PangPang-chief-filming-us-using-phone-or-tablet.JPG" width="400" /></a></div>
<br />
To test out where we needed to put the Mesh Extenders, we attached one to a long bamboo pole, which we then hoisted up attached to the chief's house:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglSbbM2whk7nUiQBvS_E9tG-EzZSRJdD1ayKpCESQg82NZq9PILqU4FI5eNpHOv2l7rHsJwsH0cipBxlFJRpaQ7Rd-dsD2fp0hIjBJ7T_AsaNJCekmiFJvcrSaGaw6CKWElaQWqfKTZ3Y/s1600/P1010022-PangPang-Fitting-MeshExtender-to-bamboo.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglSbbM2whk7nUiQBvS_E9tG-EzZSRJdD1ayKpCESQg82NZq9PILqU4FI5eNpHOv2l7rHsJwsH0cipBxlFJRpaQ7Rd-dsD2fp0hIjBJ7T_AsaNJCekmiFJvcrSaGaw6CKWElaQWqfKTZ3Y/s400/P1010022-PangPang-Fitting-MeshExtender-to-bamboo.JPG" width="400" /></a></div>
<br />
However, we soon found by accident that the extra elevation was not necessary, as the Mesh Extenders were reasonably close to one another (not more than a few hundred metres), and with no really dense vegetation in the way. We found this out on the second day, because we forgot to bring the extension cable that would allow us to use the bamboo again, and had to resort to putting it onto the roof without any extra elevation:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9LarIXrKCLq-Era5Gtab-vl1TeLcEoKF5J_TB6d0Uc8kY3shjLc539tWvVQrEBeTrvopVPMx_wX1GwYP8kTafXMkuxKgVtPHa5ozzi1pbwaej_UPGtXZn38ihjlwGPOpDFy8dc9Y4t8c/s1600/P1010064-PangPang-range-testing-putting-MeshExtender-on-roof.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9LarIXrKCLq-Era5Gtab-vl1TeLcEoKF5J_TB6d0Uc8kY3shjLc539tWvVQrEBeTrvopVPMx_wX1GwYP8kTafXMkuxKgVtPHa5ozzi1pbwaej_UPGtXZn38ihjlwGPOpDFy8dc9Y4t8c/s400/P1010064-PangPang-range-testing-putting-MeshExtender-on-roof.JPG" width="400" /></a></div>
<br />
Then we wandered around the village with a second battery-powered Mesh Extender and laptop to verify that the signal was getting through:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh50kAK1Tmc8f3srumWkBy0tzNFGWWGP8Z25AON5if3We8njjWpYnkPa1ufl2vVBOLXoKWXZ9nztz5-dovDcmMmSF7TyAAEyjk-AwcR_QoSn7UxRT7ZvqTTQn-1cFYhvPuwU8H_KbhyphenhyphenIoc/s1600/P1010072-PangPang-range-testing-good-uhf-link-rugged-laptop-in-jungle-close-up.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh50kAK1Tmc8f3srumWkBy0tzNFGWWGP8Z25AON5if3We8njjWpYnkPa1ufl2vVBOLXoKWXZ9nztz5-dovDcmMmSF7TyAAEyjk-AwcR_QoSn7UxRT7ZvqTTQn-1cFYhvPuwU8H_KbhyphenhyphenIoc/s400/P1010072-PangPang-range-testing-good-uhf-link-rugged-laptop-in-jungle-close-up.JPG" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
The third day we knew where we wanted to install the Mesh Extenders, so together with Donald, we did exactly that. In keeping with our desire that the local communities drive the process as much as possible, we left the physical installation to Donald, who I might add is very adept with a chainsaw. Here he is making pieces of 2x2 from a piece of 2x4 very accurately, using only the chainsaw:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgF3sRWexCtrsAQbh4Oi4XZi9s3Hm9cX05m42HQ54X6VJ8XGG2Djdv5SSB7bDJRdC1GwlFMyJ9ezMFTgXKcF6Yl3P9icqfAFmjlz-7h6GDw6t7oHTJu2LEwLyXgvzn3jB7IQIxjqyN4gpY/s1600/P1010201-PangPang-MeshExtender-install-Donald-Chainsawing-Bracket-with-David.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgF3sRWexCtrsAQbh4Oi4XZi9s3Hm9cX05m42HQ54X6VJ8XGG2Djdv5SSB7bDJRdC1GwlFMyJ9ezMFTgXKcF6Yl3P9icqfAFmjlz-7h6GDw6t7oHTJu2LEwLyXgvzn3jB7IQIxjqyN4gpY/s400/P1010201-PangPang-MeshExtender-install-Donald-Chainsawing-Bracket-with-David.JPG" width="400" /></a></div>
<br />
First up, one Mesh Extender on the chief's house. You can see the elbow shape attachment that Donald used in most cases. If you look very closely, you can see that the nails that attach it to the house beam are not nailed all the way in: This is to allow it to be easily removed when a cyclone comes, so that it can avoid damage, and be put back out again quickly after. A very simple and very Vanuatu solution to the problem of cyclones.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHOIMbAXxe1u0kq6IC4vSeD_U_SAwTCvOeq-v8K2HoFg4n7gJzSlT8XfTNIhfO0sV3az_BgCMwGVOssCn8emuzaMbuQt7vQl4j5zn28lYRbTmV76y00yfHECPCrjI0xBfaSJvs9jBUx3A/s1600/P1010220-PangPang-MeshExtender-installed-Chiefs-house.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHOIMbAXxe1u0kq6IC4vSeD_U_SAwTCvOeq-v8K2HoFg4n7gJzSlT8XfTNIhfO0sV3az_BgCMwGVOssCn8emuzaMbuQt7vQl4j5zn28lYRbTmV76y00yfHECPCrjI0xBfaSJvs9jBUx3A/s400/P1010220-PangPang-MeshExtender-installed-Chiefs-house.JPG" width="400" /></a></div>
<br />
Here is the Mesh Extender itself cable-tied to the 2x2 post on the roof:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbWC2TcLwkqQ-cguITczd-ALINKoUNo1GHvFXG9kotZ5sWak1YWC-4hY1zkCE0DOLFsVnXMYCoqBCvNWgmo6xmVHnhoSyja7AIx8DMlnhndKO3VpMbgXSkY9an_9w2ua_EB-WrfhD5mig/s1600/P1010224-PangPang-MeshExtender-installed-Chiefs-house-close-up.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbWC2TcLwkqQ-cguITczd-ALINKoUNo1GHvFXG9kotZ5sWak1YWC-4hY1zkCE0DOLFsVnXMYCoqBCvNWgmo6xmVHnhoSyja7AIx8DMlnhndKO3VpMbgXSkY9an_9w2ua_EB-WrfhD5mig/s400/P1010224-PangPang-MeshExtender-installed-Chiefs-house-close-up.JPG" width="400" /></a></div>
<br />
Roughly the same process was followed at the other locations, resulting in Mesh Extenders installed on the roofs of five houses over an area of about 1km long by a couple of hundred metres wide.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCDx8RgoqJu7TaHm_Jg-v8B2ul3zJ1oQYp42kvcVLiGayA0SjNi7ACkK14La07UyZjPStUyS49xgdMqKP_yIxpO9IuWPXpoVftgOoeg2vDReZHwvq3BOwXOx1DvgJzPzwIqIVmO9jw93o/s1600/P1010272-PangPang-MeshExtender-install-on-roof.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCDx8RgoqJu7TaHm_Jg-v8B2ul3zJ1oQYp42kvcVLiGayA0SjNi7ACkK14La07UyZjPStUyS49xgdMqKP_yIxpO9IuWPXpoVftgOoeg2vDReZHwvq3BOwXOx1DvgJzPzwIqIVmO9jw93o/s400/P1010272-PangPang-MeshExtender-install-on-roof.JPG" width="300" /></a></div>
<div>
<br />
This was all finished at about 4:30pm the night before flying out at 7am, leaving no time to install the app on people's phones, or to show them how to use it. That would have to wait for our fourth visit, which is now half complete.</div>
<br />Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com4tag:blogger.com,1999:blog-9068638994894102968.post-33633354543549453702017-09-13T09:03:00.001-07:002017-09-13T09:03:13.386-07:00Second visit to Vanuatu<div class="separator" style="clear: both;">
We are getting to the end of our third trip to Vanuatu, so I figured I really ought to post about our second visit.</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
This visit was shorter than the first, as it was primarily focused around the UN World Food Programme's regional Emergency Telecommunications Cluster (ETC) meeting, which has representatives from Vanuatu, Samoa, Solomon Islands, Fiji and Tonga. The purpose of this meeting is to help the relevant folks from each country to get to know one another, as well as to learn more about what is happening globally, and to find solutions to the problems that they are facing in their region and specific countries. This is the second time I have attended this regional meeting.</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
Together with Andrew Bate we spoke about the Serval Mesh and our pilot here in Vanuatu.</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhe98A7lswrLL9dqxk1-3SLqqUL4VhoThRHRpgC63J7W4GtXNSWHJrki5QoAs4dUUCO89AaaG0-yn23qGkvyQ6vDS25gnxxZlTUENV1VII_Jw5dPZMG0uIJQYngFNvGi1v6QTF5Mk-dLIg/s1600/P1000483-PGS-Presenting-at-UN-WFP-EmergencyTelecommsCluster.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhe98A7lswrLL9dqxk1-3SLqqUL4VhoThRHRpgC63J7W4GtXNSWHJrki5QoAs4dUUCO89AaaG0-yn23qGkvyQ6vDS25gnxxZlTUENV1VII_Jw5dPZMG0uIJQYngFNvGi1v6QTF5Mk-dLIg/s320/P1000483-PGS-Presenting-at-UN-WFP-EmergencyTelecommsCluster.JPG" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both;">
There was strong interest with plenty of people handling the Mesh Extender prototypes:</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqFAWgdFYjpy8XJYKJHK1Arkm7AdTdIlPCGBrFRNgJLjq3kzrS5HdTNKpgV-vEVYKaZlrEn_3VyWTz5NVfZudpSsw0svxRgabmFoVgHDTE0aw78CXtdjp-IwVp8xTBdRAgN6Y2_VfGr-0/s1600/P1000484-PGS-Presenting-at-UN-WFP-EmergencyTelecommsCluster.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqFAWgdFYjpy8XJYKJHK1Arkm7AdTdIlPCGBrFRNgJLjq3kzrS5HdTNKpgV-vEVYKaZlrEn_3VyWTz5NVfZudpSsw0svxRgabmFoVgHDTE0aw78CXtdjp-IwVp8xTBdRAgN6Y2_VfGr-0/s320/P1000484-PGS-Presenting-at-UN-WFP-EmergencyTelecommsCluster.JPG" width="320" /></a></div>
<br />
And then giving the Serval Mesh a try. In the Pacific where telecommunications is almost always a challenge, there was no need to convince people of the need: Rather they wanted to know when the Mesh Extenders were available for purchase, and at least one asked why we weren't running the pilot in their country instead!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjhKsb5Jkm2XNDj1MT0u9t-pUQu5gU94P2Ogn2bN8jSIV4Z31NYUiLQY0pltnfQL5sEgyV58tl29-KKgBVAmLjpHe82xi9KQV8YpdyieHr7mNbEQDiHvmGBEKBWT4nXkNHvca_fyu5gwqc/s1600/P1000488-MeshExtender-show-and-tell-at-UN-WFP-EmergencyTelecommsCluster.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjhKsb5Jkm2XNDj1MT0u9t-pUQu5gU94P2Ogn2bN8jSIV4Z31NYUiLQY0pltnfQL5sEgyV58tl29-KKgBVAmLjpHe82xi9KQV8YpdyieHr7mNbEQDiHvmGBEKBWT4nXkNHvca_fyu5gwqc/s320/P1000488-MeshExtender-show-and-tell-at-UN-WFP-EmergencyTelecommsCluster.JPG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both;">
The interactive demo was lots of fun, with many people around the room trying it out.</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxdfZSRQon95lzFRRNQO5yiNpnNbgtx4eA5kBQtwRpByyKEblcz0t7WHz-ePyY7JPRqKkRAqnm9-EgmouxMaLMU-f9fXJ9rLFhXt55pfAw8ED4WPt8EQorBi2Q35Q3W8c6f5SLWhRbMOc/s1600/P1000524-ServalMesh-at-UN-WFP-EmergencyTelecommsCluster.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxdfZSRQon95lzFRRNQO5yiNpnNbgtx4eA5kBQtwRpByyKEblcz0t7WHz-ePyY7JPRqKkRAqnm9-EgmouxMaLMU-f9fXJ9rLFhXt55pfAw8ED4WPt8EQorBi2Q35Q3W8c6f5SLWhRbMOc/s320/P1000524-ServalMesh-at-UN-WFP-EmergencyTelecommsCluster.JPG" width="320" /></a></div>
<br />
But what I really love are situations like that captured in the following photo, where people start showing each other how to use it:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTgTgb1ihncLVlvgZtbH4Szpu1iaflwm7TLP0p2C1nCCxvSJdD183BmMYgTjgsKqY7uCc7_66Cg6TsPxkKEaQIUy82VbM6nuO9Uc9nimpZi-wEdw9tKHEoIwqlBtTqn8eyyZlA0NjYk5s/s1600/P1000527-ServalMesh-at-UN-WFP-EmergencyTelecommsCluster.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTgTgb1ihncLVlvgZtbH4Szpu1iaflwm7TLP0p2C1nCCxvSJdD183BmMYgTjgsKqY7uCc7_66Cg6TsPxkKEaQIUy82VbM6nuO9Uc9nimpZi-wEdw9tKHEoIwqlBtTqn8eyyZlA0NjYk5s/s320/P1000527-ServalMesh-at-UN-WFP-EmergencyTelecommsCluster.JPG" width="240" /></a></div>
<br />
After the indoor demo, we later had a session with other presenters where we went outside to let people have a further hands-on of the various technologies being discussed. So I soldered up the correct connector onto a solar panel to power a Mesh Extender directly from that, without even a battery, using the built-in solar regulator in the Mesh Extender:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHsb3wgzdQbeRIEI0WOqSg8zaz6ldGF_cOhzecovsWtaxCHn3zcoUaJpzR1DZjMtjd207rKtu-jLfDSKrE3xEHeVCgj74cvonDIOBeo3jIfxQDS2lzr4Dgmy7i0_Ck8yNWwgLTmM0Ht94/s1600/P1000551-PGS-preparing-MeshExtender-solar-panel.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHsb3wgzdQbeRIEI0WOqSg8zaz6ldGF_cOhzecovsWtaxCHn3zcoUaJpzR1DZjMtjd207rKtu-jLfDSKrE3xEHeVCgj74cvonDIOBeo3jIfxQDS2lzr4Dgmy7i0_Ck8yNWwgLTmM0Ht94/s320/P1000551-PGS-preparing-MeshExtender-solar-panel.JPG" width="320" /></a></div>
<br />
Then found a suitable coconut palm to attach it to (this is the tropics, after all):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLibN9Uh4afOGIEa93NfMP9I7a9ih6zBVHRMAXeMayHF_nEFke8IhP0SyaF_ve5QimSj4GLB-MHATc5WTGog15dKIfELDYD15V80ZTRJK39rQV2hRej_zkz6AuuV6PLlewZ-zFkkIRVJM/s1600/P1000564-PGS-Trying-Solar-MeshExtender-to-Coconut-Tree.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLibN9Uh4afOGIEa93NfMP9I7a9ih6zBVHRMAXeMayHF_nEFke8IhP0SyaF_ve5QimSj4GLB-MHATc5WTGog15dKIfELDYD15V80ZTRJK39rQV2hRej_zkz6AuuV6PLlewZ-zFkkIRVJM/s320/P1000564-PGS-Trying-Solar-MeshExtender-to-Coconut-Tree.JPG" width="240" /></a></div>
<br />
Held firmly in place with an <a href="https://en.wikipedia.org/wiki/Bungee_cord">octopus-strap</a>, i.e., bungie cord for those of you not from Australia:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhgEh1D7NY2fssKSxy6O2J8mmAtfFdmTOD7c7NzjDGTRxEThmrB8JRJpVUCI-wWJkLIK4rd6-C5RryvltZt2EBSDtJQer4Ow8kLPgJEY-6s89jOL4op2muLIgVligu2Vyq3gGHbuOMxxdc/s1600/P1000565-PGS-Trying-Solar-MeshExtender-to-Coconut-Tree.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhgEh1D7NY2fssKSxy6O2J8mmAtfFdmTOD7c7NzjDGTRxEThmrB8JRJpVUCI-wWJkLIK4rd6-C5RryvltZt2EBSDtJQer4Ow8kLPgJEY-6s89jOL4op2muLIgVligu2Vyq3gGHbuOMxxdc/s320/P1000565-PGS-Trying-Solar-MeshExtender-to-Coconut-Tree.JPG" width="240" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3QMKm0lAwLyjDpCU1bX5M4dgV7nkwzoStHKxWbgvgQ_EN3EAlWaPZZ5jECBoupggsdGIUd8VShh1GrVRTK8Dt0dfinnW6vUzeAAZSLCX6vs4GSTfzHPpCGipnk7-wgNw07Uy0mP7vATo/s1600/P1000567-PGS-Trying-Solar-MeshExtender-to-Coconut-Tree.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3QMKm0lAwLyjDpCU1bX5M4dgV7nkwzoStHKxWbgvgQ_EN3EAlWaPZZ5jECBoupggsdGIUd8VShh1GrVRTK8Dt0dfinnW6vUzeAAZSLCX6vs4GSTfzHPpCGipnk7-wgNw07Uy0mP7vATo/s320/P1000567-PGS-Trying-Solar-MeshExtender-to-Coconut-Tree.JPG" width="320" /></a></div>
<br />
Then it was open-season for the Mesh Extender. Again, it was great to see a wide variety of folks from different countries and organisations in the Pacific trying it out:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWNfpydkIAWzyifEqoxSgtaJiMP6JYZy8gS8tsjCpbubxAQ3IpmjD6nfvEJU_TNoNGzt4yKtmM091G6U088KQhGKoX0v_P_UZtIaw5Gff3D_4Xbxp_hi1Hka6NhCcYQI5L10c_sJDATxo/s1600/P1000572-NGOs-trying-ServalMesh-in-Vanuatu.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWNfpydkIAWzyifEqoxSgtaJiMP6JYZy8gS8tsjCpbubxAQ3IpmjD6nfvEJU_TNoNGzt4yKtmM091G6U088KQhGKoX0v_P_UZtIaw5Gff3D_4Xbxp_hi1Hka6NhCcYQI5L10c_sJDATxo/s320/P1000572-NGOs-trying-ServalMesh-in-Vanuatu.JPG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgseaEZSYFBrOZqJkdGmJCNemyEB-_zlvsCqZqug-JFeaqGGK7SFvyjiLycfpQTVD-Z-YieuJEHYqzVpDO3eyt0fM2tIQga78ewuOVDB4hwjHrQ3iMNAYhLGpNmIAxbAH9elNHeSj4V68Y/s1600/P1000574-NGOs-trying-ServalMesh-in-Vanuatu.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgseaEZSYFBrOZqJkdGmJCNemyEB-_zlvsCqZqug-JFeaqGGK7SFvyjiLycfpQTVD-Z-YieuJEHYqzVpDO3eyt0fM2tIQga78ewuOVDB4hwjHrQ3iMNAYhLGpNmIAxbAH9elNHeSj4V68Y/s320/P1000574-NGOs-trying-ServalMesh-in-Vanuatu.JPG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj62cQQDclZc8tiFlr-8CxSxLADND7nzKyIKGzOp_AE0Mr2D30Egtkjqj2CWqJmGRa7f_EVxu9RL6Q2S0te-G2b2JSklhEsTExKn9nYDCJUAzbxlgdfflybaa9McMjTf_FWx4wNLX-MPoo/s1600/P1000577-NGOs-trying-ServalMesh-in-Vanuatu.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj62cQQDclZc8tiFlr-8CxSxLADND7nzKyIKGzOp_AE0Mr2D30Egtkjqj2CWqJmGRa7f_EVxu9RL6Q2S0te-G2b2JSklhEsTExKn9nYDCJUAzbxlgdfflybaa9McMjTf_FWx4wNLX-MPoo/s320/P1000577-NGOs-trying-ServalMesh-in-Vanuatu.JPG" width="320" /></a></div>
<br />
Again, I just love it when people start showing each other how to use it, without any involvement from me. At that point we have victory!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNdKGMux2KIdvlEEC0OKH4eZK4lMDiM945c5q0uAkLNhpCk5Duj1n1dBohRFlKzRKevmi3h0fbq8KYia1du_ndENMfRlRuTC8hH74RAdBD7rKPCiSjdTiGDimpOEdPMkrDW2k5AsRAFYs/s1600/P1000581-NGOs-trying-ServalMesh-in-Vanuatu.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNdKGMux2KIdvlEEC0OKH4eZK4lMDiM945c5q0uAkLNhpCk5Duj1n1dBohRFlKzRKevmi3h0fbq8KYia1du_ndENMfRlRuTC8hH74RAdBD7rKPCiSjdTiGDimpOEdPMkrDW2k5AsRAFYs/s320/P1000581-NGOs-trying-ServalMesh-in-Vanuatu.JPG" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4l4sw6Rn5oQqPhv-_35CvZ_rEqUXV_MP6Y8RPc-ryg-IyhXqV6toPLrh7KAydBjJ0J2T5zxBixMhZDwID-50V1vyZZWjQiSyJJ-oSGAmmrAdqXUzwfgeyVJ6uv4N9haQ74XhiSMIQTYQ/s1600/P1000606-NGOs-trying-ServalMesh-in-Vanuatu.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4l4sw6Rn5oQqPhv-_35CvZ_rEqUXV_MP6Y8RPc-ryg-IyhXqV6toPLrh7KAydBjJ0J2T5zxBixMhZDwID-50V1vyZZWjQiSyJJ-oSGAmmrAdqXUzwfgeyVJ6uv4N9haQ74XhiSMIQTYQ/s320/P1000606-NGOs-trying-ServalMesh-in-Vanuatu.JPG" width="320" /></a></div>
<br />
Also, we discovered that the Mesh Extender is approximately one hand in size:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh11MlVp0b16bOpQhjIy0VvTHlE0f9UNJyX9n1sTSUwZTZUAe5S-YaWYFChwIc4Lfb6r0mBOhaCJfUKcFjSemymkAww7T9n9szec1MT6wDb4mAqMxd_ILyk9GdHRgsKbVac13LX0sPC-rQ/s1600/P1000598-NGOs-taking-photos-of-MeshExtender-on-Coconut-tree.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh11MlVp0b16bOpQhjIy0VvTHlE0f9UNJyX9n1sTSUwZTZUAe5S-YaWYFChwIc4Lfb6r0mBOhaCJfUKcFjSemymkAww7T9n9szec1MT6wDb4mAqMxd_ILyk9GdHRgsKbVac13LX0sPC-rQ/s320/P1000598-NGOs-taking-photos-of-MeshExtender-on-Coconut-tree.JPG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both;">
Then later in the week we had an impromptu meeting with the Vanuatu Ministry of Health, where we repeated the hand-on test, but this time all I did was supply the equipment: They did the temporary installation themselves: </div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5iGo9H5rZcQ3nkwj5dPEYrCAs9OviBP7Q3twTrdrB-O6uKxzmyX2i0uoDIvDw4NEpndESvk4TW4icjHHrevOT6_OcoQUZHpKqOO6FulQZpbJyJy9qC5oOxFLNAbTpiZCFvpgJIjP1RAA/s1600/P1000670-Vanuatu-Health-Ministry-MeshExtender-Demo.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5iGo9H5rZcQ3nkwj5dPEYrCAs9OviBP7Q3twTrdrB-O6uKxzmyX2i0uoDIvDw4NEpndESvk4TW4icjHHrevOT6_OcoQUZHpKqOO6FulQZpbJyJy9qC5oOxFLNAbTpiZCFvpgJIjP1RAA/s320/P1000670-Vanuatu-Health-Ministry-MeshExtender-Demo.JPG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZd6EQUzhQIQKlitIWJ-4D6tz1-4Yat9llBXkSzhoxoBa-vz8Zqa_RB_lfrcYhw70mkRUW8dU9brukKrUHwfq1mYsRMpTYVzFfy9hT_Gr6V48TzS6db4WSu3aD5sEleZbXfTfQZkHYwvQ/s1600/P1000671-Vanuatu-Health-Ministry-MeshExtender-Demo.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZd6EQUzhQIQKlitIWJ-4D6tz1-4Yat9llBXkSzhoxoBa-vz8Zqa_RB_lfrcYhw70mkRUW8dU9brukKrUHwfq1mYsRMpTYVzFfy9hT_Gr6V48TzS6db4WSu3aD5sEleZbXfTfQZkHYwvQ/s320/P1000671-Vanuatu-Health-Ministry-MeshExtender-Demo.JPG" width="240" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiC9IgZJR9UQ1ybOP-WL3f2WagLzun-lZV_9pDz0khDt_UdrusgHtqRLDr5Zq7xvn9hutRcd0Sqxf4DH1YMA9RybvFfRiiO57_JUXWVB1uBLgK-prIYFnJcAvUBuszl4WWQN3M-G2NMJK8/s1600/P1000672-Vanuatu-Health-Ministry-MeshExtender-Demo.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiC9IgZJR9UQ1ybOP-WL3f2WagLzun-lZV_9pDz0khDt_UdrusgHtqRLDr5Zq7xvn9hutRcd0Sqxf4DH1YMA9RybvFfRiiO57_JUXWVB1uBLgK-prIYFnJcAvUBuszl4WWQN3M-G2NMJK8/s320/P1000672-Vanuatu-Health-Ministry-MeshExtender-Demo.JPG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7VjYjNSvMNtivuFoN8Hk3FUWyNOOoxUh1pXFMHFjHtTrvRTAUawsZmADc8KxjxCaiPhYCcR3NeSY0T-orU_MbQr6M-Yj9eA1KkCg1Y1vCI4-SbiFucfeWxPYXPnratNB_UBKgsgbNBPA/s1600/P1000673-Vanuatu-Health-Ministry-MeshExtender-Demo.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7VjYjNSvMNtivuFoN8Hk3FUWyNOOoxUh1pXFMHFjHtTrvRTAUawsZmADc8KxjxCaiPhYCcR3NeSY0T-orU_MbQr6M-Yj9eA1KkCg1Y1vCI4-SbiFucfeWxPYXPnratNB_UBKgsgbNBPA/s320/P1000673-Vanuatu-Health-Ministry-MeshExtender-Demo.JPG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5TUP628wFKc_sluK2NVFKSMDL73AZzVH9lRlBGMNGWiOhoaOAZs9C_USRKHUwAZedCpujmfp628-tKkGSeNqMdhyphenhyphen8aBMd_Wfh9YdNGxsmkpsEl4HsZLi-RnbmalX9tjfH8DaDZBzNfgU/s1600/P1000674-Vanuatu-Health-Ministry-MeshExtender-Demo.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5TUP628wFKc_sluK2NVFKSMDL73AZzVH9lRlBGMNGWiOhoaOAZs9C_USRKHUwAZedCpujmfp628-tKkGSeNqMdhyphenhyphen8aBMd_Wfh9YdNGxsmkpsEl4HsZLi-RnbmalX9tjfH8DaDZBzNfgU/s320/P1000674-Vanuatu-Health-Ministry-MeshExtender-Demo.JPG" width="240" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAqed4Y_6-17RXJk4AVtmIrHLQv4r3qMlQ_n6biZZnt71hz-QqcOB8Eph5m_6KiCOpnzyM6JAXz_7r5eMHWIYvEAZa3TW6FMAicL9mXn_dFgcw-vT0u98uacA-vTO80yN3jBqBiyDmry8/s1600/P1000684-Vanuatu-Health-Ministry-MeshExtender-Demo.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAqed4Y_6-17RXJk4AVtmIrHLQv4r3qMlQ_n6biZZnt71hz-QqcOB8Eph5m_6KiCOpnzyM6JAXz_7r5eMHWIYvEAZa3TW6FMAicL9mXn_dFgcw-vT0u98uacA-vTO80yN3jBqBiyDmry8/s320/P1000684-Vanuatu-Health-Ministry-MeshExtender-Demo.JPG" width="320" /></a></div>
<br />
Then it was another encouraging session where everyone was trying it out and showing each other how it worked. There were also some folks from the UN World Health Organisation there, who were very interested in how it could be used to help with disease and public health data collection.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinU437JeH_J_1kON3Ue28AhmCMkjNFsMgBhizeZgmYU1rROURCZwdwjQq7AmY3sJl6MMxao_qKt5Z47jGq4oSG5IUKXkg6_WU0ZEU1xqG4CO5qkK-qY7pzOTMXIXCXG2zZ-Ml45u1gww0/s1600/P1000699-Vanuatu-Ministry-of-Health-using-ServalMesh.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinU437JeH_J_1kON3Ue28AhmCMkjNFsMgBhizeZgmYU1rROURCZwdwjQq7AmY3sJl6MMxao_qKt5Z47jGq4oSG5IUKXkg6_WU0ZEU1xqG4CO5qkK-qY7pzOTMXIXCXG2zZ-Ml45u1gww0/s320/P1000699-Vanuatu-Ministry-of-Health-using-ServalMesh.JPG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRkoV-IEJWM4_h-nzlCpjsbhC_z1UqOnI4k27h9mrwizuKfE0TDI2WLMI5OMTchmxZwbek5PzG8cVr_eEmp-6RR2Trr5PTJ_SH73EdcIhagl20I9yL9M07ftb9N0HGIXNZ2jAafizMxAM/s1600/P1000700-Vanuatu-Ministry-of-Health-and-UN-WHO-using-ServalMesh.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRkoV-IEJWM4_h-nzlCpjsbhC_z1UqOnI4k27h9mrwizuKfE0TDI2WLMI5OMTchmxZwbek5PzG8cVr_eEmp-6RR2Trr5PTJ_SH73EdcIhagl20I9yL9M07ftb9N0HGIXNZ2jAafizMxAM/s320/P1000700-Vanuatu-Ministry-of-Health-and-UN-WHO-using-ServalMesh.JPG" width="320" /></a></div>
<br />
One of the other great things is seeing not just men using Serval, but women as well, and women showing other women how to use it. This is important because the more that a technology is used by all kinds of people in a community, the more likely it is to succeed.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjTnydfgL_3BI3lSCTGNCBArczoFcuJBrsCIuO_6pacHz0XNY1dHGLWvmP7ePMvtC-BeXShoY_S8H-hAPHRg-tcp7JN_UsC4dFjcvJ6shxn27Batx8vjdJLivxW6ctFiCIdij4rZuPVtU/s1600/P1000711-Vanuatu-Ministry-of-Health-and-UN-WFP-using-ServalMesh.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjTnydfgL_3BI3lSCTGNCBArczoFcuJBrsCIuO_6pacHz0XNY1dHGLWvmP7ePMvtC-BeXShoY_S8H-hAPHRg-tcp7JN_UsC4dFjcvJ6shxn27Batx8vjdJLivxW6ctFiCIdij4rZuPVtU/s320/P1000711-Vanuatu-Ministry-of-Health-and-UN-WFP-using-ServalMesh.JPG" width="240" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYaZL6jDe-L8W8rO6xgWP_5HDOre67SZ0hg4Hp3kVxX6-cmsMEAPwmUMGwTU7vuwjRJ1PSltv5zpzmBvEF0ms53Z423GeYOI-wDlv-XMlYCXw23HwHYRdPN2yURBi81SbaelfSVEJDu1o/s1600/P1000729-Vanuatu-Ministry-of-Health-and-UN-WFP-using-ServalMesh.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYaZL6jDe-L8W8rO6xgWP_5HDOre67SZ0hg4Hp3kVxX6-cmsMEAPwmUMGwTU7vuwjRJ1PSltv5zpzmBvEF0ms53Z423GeYOI-wDlv-XMlYCXw23HwHYRdPN2yURBi81SbaelfSVEJDu1o/s320/P1000729-Vanuatu-Ministry-of-Health-and-UN-WFP-using-ServalMesh.JPG" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both;">
<br /></div>
Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com2tag:blogger.com,1999:blog-9068638994894102968.post-34102785119970519872017-08-28T20:43:00.002-07:002017-08-28T20:43:59.420-07:00Assembling 50 Mesh Extenders ready for VanuatuTomorrow we fly out to Vanuatu for our third visit. <br />
<br />
This means we need to have several dozen Mesh Extenders ready to take with us, to hopefully begin deploying in Port Vila between various government and NGO sites, and assuming we get the final authorities through, to also connect a couple of outlying villages.<br />
<br />
From a practical stand point, this means flashing 50 Mesh Extender boards, 50 RFD900+ radios, 50 boot loaders, and pre-formatting and putting the default files on 50 USB memory sticks, and then assembling the boards in their cases.<br />
<br />
Flashing the radios is fairly simple, and can be done using an FTDI USB serial cable in about a minute. I did those last night while sitting at the computer after the kids went to bed.<br />
<br />
For the boot loaders and OpenWRT firmware, <a href="https://github.com/servalproject/openwrt-packages/tree/MeshExtender2.0/auto-flash">I wrote a simple program</a> today that listens on a serial port for the uboot boot loader, and then directly issues the commands to flash first the boot loader, and then the firmware. When it has finished, it runs vlc and plays an MP3 file to alert me that it is ready for the next board, thus saving lots of dead time between flashing boards. It went smoother with the team in the lab when I stopped playing the <a href="https://www.youtube.com/watch?v=xxhP6vD3unY">music from Wizball</a>, and switched to <a href="https://ia800208.us.archive.org/33/items/ClassicalGuitar-Greensleeves/Greensleeves__Sir_John.mp3">Greensleves</a>.<br />
<br />
Apart from some niggly problems with the USB serial adapter drivers causing the port to stop reading data, the while process takes only a minute or two. This is allowing us to plough through flashing the boards much faster than expected, which is good (I have been interrupted by VLC playing Greensleves at me several times while writing this post).<br />
<br />
Here is my bench setup, as I flash a board. Firmware and boot loader updates are loaded via ethernet from a tftp server on my laptop. Commands are via serial through an adapter on the radio/power connector on the Mesh Extender, and power for the Mesh Extender comes via the micro-USB connector on the Mesh Extender board which exists for that single purpose (it doesn't actually provide an USB data signals):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWhkQmdTi8IoF2n7mzHyeICR-3pJLh5ujLPZiKDuI8mNzi1VrHXbQvS_8ryAvLr-5zEPjiQ8yzTvZ6WkhfAh2IB-OhElIir7pBkuMnsqeGT2yg-WMNNK-9BChe1eD8Y08AGG24woFNCPs/s1600/IMG_20170829_114229.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWhkQmdTi8IoF2n7mzHyeICR-3pJLh5ujLPZiKDuI8mNzi1VrHXbQvS_8ryAvLr-5zEPjiQ8yzTvZ6WkhfAh2IB-OhElIir7pBkuMnsqeGT2yg-WMNNK-9BChe1eD8Y08AGG24woFNCPs/s400/IMG_20170829_114229.jpg" width="400" /></a></div>
<br />
The harder part is the assembly. You can see three of our great students who are helping with the assembly:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0Jt54ejJaLwt1tvU830H_tlNJWdS-R85wXgxGL0puH9vGfzuNwrJ6IGgcy76Et2vXAODoJcZbwj6HiO2wCib8h7EQa7Hy4LqZ-669U-kE6Kp3se4RJX_ITS6hKWGJZ-t7wSV26fV7V14/s1600/IMG_20170829_114243.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0Jt54ejJaLwt1tvU830H_tlNJWdS-R85wXgxGL0puH9vGfzuNwrJ6IGgcy76Et2vXAODoJcZbwj6HiO2wCib8h7EQa7Hy4LqZ-669U-kE6Kp3se4RJX_ITS6hKWGJZ-t7wSV26fV7V14/s400/IMG_20170829_114243.jpg" width="400" /></a></div>
<br />
Ryan is checking that the boards have correctly flashed and boot and begin transmitting via the UHF packet radio.<br />
<br />
Then we have to get the boards into their cases. This is the fun part, because this revision of the PCBs has the wrong D-SUB 25 connector compared to what we wanted, but due to time constraints we are stuck with them for this batch. The problem is that the D-SUB connectors have holes for screws, but are not threaded. This means we have to put washers and nuts on the back of the connectors, which are frustratingly located in rather awkward locations, as shown here marked by the yellow arrows. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaU75qOIeiRlbInSRMsBKhvni0iNfKmM1UOLPoh5Dy9-lSFEXlJUHIhwM1CSpU7gYTrfQ2NLBdABfpEWu2gkhF-1392_DDZgAUiWfOyMVEM0KblUo3v0qTc3LMIgsm6r0i5dc5LkY765Q/s1600/IMG_20170829_130624.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaU75qOIeiRlbInSRMsBKhvni0iNfKmM1UOLPoh5Dy9-lSFEXlJUHIhwM1CSpU7gYTrfQ2NLBdABfpEWu2gkhF-1392_DDZgAUiWfOyMVEM0KblUo3v0qTc3LMIgsm6r0i5dc5LkY765Q/s400/IMG_20170829_130624.jpg" width="400" /></a></div>
<br />
Those locations are right in the bottom of the shell, about 55mm down.
For the one on the left, the super-capacitor is in the way, all making
it rather difficult to get in there to install the washer and nut on
each. <i>Very</i> annoying. Our current least-bad solution is to use some <a href="http://au.element14.com/duratool/cl5045/seizer-straight/dp/1421548?MER=bn_level5_5NP_EngagementRecSingleItem_1">seizers</a>. <br />
<br />
So that's our afternoon today, basically getting enough of these assembled, ready to go, as well as getting everything packed in the big case ready to fly out tomorrow.Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com8tag:blogger.com,1999:blog-9068638994894102968.post-91577248048192405162017-08-22T13:50:00.001-07:002017-08-22T13:50:23.403-07:00Migrating between EFM32 parts in Simplicity Studio One of our projects at the moment involves the EFM32 Pearl Gecko low-power microcontroller. These are really nice, and capable of very low power consumption.<br />
<br />
At the last minute, our supplier could only source us the Jade Gecko instead of Pearl Gecko chips. The only difference is the lack of FPU, so far as I know. Our project doesn't do any floating point, so there wasn't a problem there, or at least, so I thought.<br />
<br />
However, when I tried to change the part in Simplicity Studio I hit endless problems, largely due to a lack of understanding about how Simplicity Studio works.<br />
<br />
Because we were basing our project on a standard kit board, and had made our PCB pin-compatible, we were continuing to use the Board Support Package (BSP) for that kit. The new part, however, was not an option in that kit, so I had to tell Simplicity Studio that I was not using that kit anymore. This was the source of all my pain, but it took me several hours of frustration to figure that out. This is because the error messages that were being generated were not particularly helpful. <br />
<br />
It would complain about strange missing include files, like <span style="font-family: Courier New, Courier, monospace;">redirectserialconfig.h</span> during the build process. This makes sense as I now understand that such files are provided by the BSP -- although the particular magic by which it provides them I still don't fully understand.<br />
<br />
This was because those files are provided from kit-specific include directories.<br />
<br />
This meant I was faced with either: (1) keeping the kit, and having the wrong MCU set, which might or might not work through binary compatibility, or (2) having to build a BSP myself, which I really wanted to avoid, as I still don't have enough low-level understanding of what is going on.<br />
<br />
A bit of digging around and I discovered that the properties dialog of Simplicity studio which lets you choose among the supported parts on a given kit in reality just sets some #defines and include directories. Thus, all I needed was:<br />
<br />
1. Go into Properties -> C/C++ Build -> Settings and for each build target modify GNU ARM C Compiler -> Symbols do delete the <span style="font-family: Courier New, Courier, monospace;">EFM32PG1B200F256GM48=1</span> line, and replace it with <span style="font-family: Courier New, Courier, monospace;">EFM32JG1B200F256GM48=1 </span><span style="font-family: inherit;">(your part names may naturally be different)</span><br />
<br />
2. Go into Properties -> C/C++ Build -> Settings and for each build target modify GNU ARM C Compiler -> Includes and change <span style="font-family: Courier New, Courier, monospace;">"${StudioSdkPath}/Device/SiliconLabs/EFM32PG1B/Include"</span> to <span style="font-family: Courier New, Courier, monospace;">"${StudioSdkPath}/Device/SiliconLabs/EFM32JG1B/Include"</span> (again, adjust part families according to your project).<br />
<br />
Then, finally, I could get the build to succeed again. This is probably all well known to just about everyone, but it tripped me up, so I figured I would document it in case it was useful for others (and also so that I can remember better how to do it in the future).<br />
<br />
For those like myself who are more command-line oriented, here is the diff for my project (which unfortunately I cannot share the complete thing with you):<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">diff --git a/.cproject b/.cproject</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">index 75fdb6d..7ed84db 100644</span><br />
<span style="font-size: xx-small;"><span style="background-color: #cccccc; font-family: Courier New, Courier, monospace;">--- </span><span style="background-color: #999999; font-family: Courier New, Courier, monospace;">a/.cproject</span></span><br />
<span style="font-size: xx-small;"><span style="background-color: #cccccc; font-family: Courier New, Courier, monospace;">+++ </span><span style="background-color: #999999; font-family: Courier New, Courier, monospace;">b/.cproject</span></span><br />
<span style="background-color: #cccccc; font-family: Courier New, Courier, monospace; font-size: xx-small;">@@ -35,7 +35,7 @@</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><option id="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.def.symbols.56412088" name="Defined symbols (-D)" superClass="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.def.symbols" valueType="definedSymbols"></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="DEBUG_EFM=1"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="DEBUG=1"/></span><br />
<span style="background-color: #ea9999; font-family: Courier New, Courier, monospace; font-size: xx-small;">-<span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="EFM32PG1B200F256GM48=1"/></span><br />
<span style="background-color: #b6d7a8; font-family: Courier New, Courier, monospace; font-size: xx-small;">+<span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="EFM32JG1B200F256GM48=1"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span></option></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><option id="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.debug.builtin.59744490" name="Always branch to builtin functions (-fno-builtin)" superClass="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.debug.builtin" value="true" valueType="boolean"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><option id="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.debug.prolog.2095193964" name="Generate debugger-friendly prologs (-mno-sched-prolog)" superClass="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.debug.prolog" value="true" valueType="boolean"/></span><br />
<span style="background-color: #cccccc; font-family: Courier New, Courier, monospace; font-size: xx-small;">@@ -47,7 +47,7 @@</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/emlib/inc&quot;"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/kits/common/drivers&quot;"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/CMSIS/Include&quot;"/></span><br />
<span style="background-color: #ea9999; font-family: Courier New, Courier, monospace; font-size: xx-small;">-<span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/Device/SiliconLabs/EFM32PG1B/Include&quot;"/></span><br />
<span style="background-color: #b6d7a8; font-family: Courier New, Courier, monospace; font-size: xx-small;">+<span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/Device/SiliconLabs/EFM32JG1B/Include&quot;"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span></option></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><inputType id="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.input.1259106684" superClass="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.input"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span></tool></span><br />
<span style="background-color: #cccccc; font-family: Courier New, Courier, monospace; font-size: xx-small;">@@ -121,7 +121,7 @@</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><option id="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.def.symbols.1906527140" name="Defined symbols (-D)" superClass="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.def.symbols" valueType="definedSymbols"></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="DEBUG_EFM=1"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="NDEBUG=1"/></span><br />
<span style="background-color: #ea9999; font-family: Courier New, Courier, monospace; font-size: xx-small;">-<span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="EFM32PG1B200F256GM48=1"/></span><br />
<span style="background-color: #b6d7a8; font-family: Courier New, Courier, monospace; font-size: xx-small;">+<span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="EFM32JG1B200F256GM48=1"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span></option></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><option id="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.debug.builtin.1881027154" name="Always branch to builtin functions (-fno-builtin)" superClass="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.debug.builtin" value="false" valueType="boolean"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><option id="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.debug.prolog.1759032162" name="Generate debugger-friendly prologs (-mno-sched-prolog)" superClass="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.debug.prolog"/></span><br />
<span style="background-color: #cccccc; font-family: Courier New, Courier, monospace; font-size: xx-small;">@@ -133,7 +133,7 @@</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/emlib/inc&quot;"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/kits/common/drivers&quot;"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/CMSIS/Include&quot;"/></span><br />
<span style="background-color: #ea9999; font-family: Courier New, Courier, monospace; font-size: xx-small;">-<span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/Device/SiliconLabs/EFM32PG1B/Include&quot;"/></span><br />
<span style="background-color: #b6d7a8; font-family: Courier New, Courier, monospace; font-size: xx-small;">+<span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/Device/SiliconLabs/EFM32JG1B/Include&quot;"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span></option></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><inputType id="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.input.2098567682" superClass="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.input"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span></tool></span><br />
<span style="background-color: #cccccc; font-family: Courier New, Courier, monospace; font-size: xx-small;">@@ -206,7 +206,7 @@</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><option id="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.def.symbols.1801462641" name="Defined symbols (-D)" superClass="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.def.symbols" valueType="definedSymbols"></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="DEBUG_EFM=1"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="DEBUG=1"/></span><br />
<span style="background-color: #ea9999; font-family: Courier New, Courier, monospace; font-size: xx-small;">-<span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="EFM32PG1B200F256GM48=1"/></span><br />
<span style="background-color: #b6d7a8; font-family: Courier New, Courier, monospace; font-size: xx-small;">+<span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="EFM32JG1B200F256GM48=1"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span></option></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><option id="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.debug.builtin.711105496" name="Always branch to builtin functions (-fno-builtin)" superClass="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.debug.builtin" value="true" valueType="boolean"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><option id="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.debug.prolog.1497848589" name="Generate debugger-friendly prologs (-mno-sched-prolog)" superClass="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.debug.prolog" value="true" valueType="boolean"/></span><br />
<span style="background-color: #cccccc; font-family: Courier New, Courier, monospace; font-size: xx-small;">@@ -218,7 +218,7 @@</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/emlib/inc&quot;"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/kits/common/drivers&quot;"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/CMSIS/Include&quot;"/></span><br />
<span style="background-color: #ea9999; font-family: Courier New, Courier, monospace; font-size: xx-small;">-<span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/Device/SiliconLabs/EFM32PG1B/Include&quot;"/></span><br />
<span style="background-color: #b6d7a8; font-family: Courier New, Courier, monospace; font-size: xx-small;">+<span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/Device/SiliconLabs/EFM32JG1B/Include&quot;"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span></option></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><inputType id="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.input.462471232" superClass="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.input"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span></tool></span><br />
<span style="background-color: #cccccc; font-family: Courier New, Courier, monospace; font-size: xx-small;">@@ -292,7 +292,7 @@</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><option id="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.def.symbols.227214347" name="Defined symbols (-D)" superClass="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.def.symbols" valueType="definedSymbols"></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="DEBUG_EFM=1"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="NDEBUG=1"/></span><br />
<span style="background-color: #ea9999; font-family: Courier New, Courier, monospace; font-size: xx-small;">-<span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="EFM32PG1B200F256GM48=1"/></span><br />
<span style="background-color: #b6d7a8; font-family: Courier New, Courier, monospace; font-size: xx-small;">+<span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="EFM32JG1B200F256GM48=1"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span></option></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><option id="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.debug.builtin.856642787" name="Always branch to builtin functions (-fno-builtin)" superClass="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.debug.builtin" value="false" valueType="boolean"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><option id="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.debug.prolog.2116127698" name="Generate debugger-friendly prologs (-mno-sched-prolog)" superClass="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.debug.prolog"/></span><br />
<span style="background-color: #cccccc; font-family: Courier New, Courier, monospace; font-size: xx-small;">@@ -304,7 +304,7 @@</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/emlib/inc&quot;"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/kits/common/drivers&quot;"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/CMSIS/Include&quot;"/></span><br />
<span style="background-color: #ea9999; font-family: Courier New, Courier, monospace; font-size: xx-small;">-<span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/Device/SiliconLabs/EFM32PG1B/Include&quot;"/></span><br />
<span style="background-color: #b6d7a8; font-family: Courier New, Courier, monospace; font-size: xx-small;">+<span style="white-space: pre;"> </span><listOptionValue builtIn="false" value="&quot;${StudioSdkPath}/Device/SiliconLabs/EFM32JG1B/Include&quot;"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span></option></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span><inputType id="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.input.686138013" superClass="com.silabs.ide.si32.gcc.cdt.managedbuild.tool.gnu.c.compiler.input"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;"> <span style="white-space: pre;"> </span></tool></span><br />
<div>
<br /></div>
Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com0tag:blogger.com,1999:blog-9068638994894102968.post-20442352336904176052017-08-21T20:33:00.001-07:002017-08-21T20:33:18.115-07:00Power/radio cables with over-moulding<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
I thought some folks might be interested to know how we are getting our cables fabricated, so here are some photos of the tool used for the low-pressure injection over-moulding that adds the red rubbery plastic over the connector and circuitry in the cable head.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
First, here you can see a finished cable and the tool, which was designed and machined for us. Unlike high-pressure injection moulding, where you can turn out multiple parts per minute, but the tool cost is tens of thousands of dollars, the complete tool design and manufacture cost here was only about AU$3,000 with our friends at <a href="http://www.innovationengineering.com.au/">Innovation Engineering</a>. The trade-off is that each part takes minutes to produce, instead of seconds. For now at least, that isn't a big problem.</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAqeeuYPWGf3ErXKChxfdVlWQ0yp_apzVvcLOHkfkh7nFkM9DYptM9_gh8MungSI1uDZ1l5ZlDacbugv61M0RedIcPfMdNnmIXwotmyxZaixFKf3cn70JtuqlGfIlHAeK0cVNzDxuqr68/s1600/IMG_20170602_115137-overmoulded-cable-and-tool.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAqeeuYPWGf3ErXKChxfdVlWQ0yp_apzVvcLOHkfkh7nFkM9DYptM9_gh8MungSI1uDZ1l5ZlDacbugv61M0RedIcPfMdNnmIXwotmyxZaixFKf3cn70JtuqlGfIlHAeK0cVNzDxuqr68/s400/IMG_20170602_115137-overmoulded-cable-and-tool.jpg" width="400" /></a>t</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
Looking closer at the tool, you can see the big fat guide pins that hold it in proper alignment when assembled:</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh201HI_3lTYHzW2Ez3kmQNn1IKXQlSIVHH_XQU-JNJsM3lTEkFBaPomPPCLLP-65SsdQwS9vjak7XtvijApsb5JuSA0ysevZlaqGDrn5jT9HgWsCcMs7kGFkqBkkpWwN8I96kfD8KVlWg/s1600/IMG_20170602_115154-overmoulded-cable-and-tool.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh201HI_3lTYHzW2Ez3kmQNn1IKXQlSIVHH_XQU-JNJsM3lTEkFBaPomPPCLLP-65SsdQwS9vjak7XtvijApsb5JuSA0ysevZlaqGDrn5jT9HgWsCcMs7kGFkqBkkpWwN8I96kfD8KVlWg/s400/IMG_20170602_115154-overmoulded-cable-and-tool.jpg" width="400" /></a></div>
<br />
Then on the other half you can see this funny bar with the two brass pins. This is removable, and held in with a couple of magnets. It ensures that there are holes through the moulded body for the thumb-screws. It needs to be removable, as you need to get the cable in and out of the tool.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdRuyV5bgIGymK-MlYKI6p9QD3tfEK2QmCmO93fwLQYJBwqIk2SeJqroRFJmBwvIln_dBD4nxO_tT_90wqRMZYep7SCKUA-JujAoFtbB8PyqkS3VF6-nq0Z4v7mL2D8DOEA2A1aMCTRQo/s1600/IMG_20170602_115159-overmoulded-cable-and-tool.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdRuyV5bgIGymK-MlYKI6p9QD3tfEK2QmCmO93fwLQYJBwqIk2SeJqroRFJmBwvIln_dBD4nxO_tT_90wqRMZYep7SCKUA-JujAoFtbB8PyqkS3VF6-nq0Z4v7mL2D8DOEA2A1aMCTRQo/s400/IMG_20170602_115159-overmoulded-cable-and-tool.jpg" width="400" /></a></div>
<br />
Here is that same tool without it in place:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFYHFuoiH_yAommvmrXonVMXFO40P9-Zx9DCfoNp77L0y30_wAZsvhL-27czDp7PtEAymu0pEjp21d55ifivPExGovjiUQo0Sv4OY8SoKUYhTsUb5Myy22lA0cRrUMyFzwyTRnN7ik808/s1600/IMG_20170602_115206-overmoulded-cable-and-tool.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFYHFuoiH_yAommvmrXonVMXFO40P9-Zx9DCfoNp77L0y30_wAZsvhL-27czDp7PtEAymu0pEjp21d55ifivPExGovjiUQo0Sv4OY8SoKUYhTsUb5Myy22lA0cRrUMyFzwyTRnN7ik808/s400/IMG_20170602_115206-overmoulded-cable-and-tool.jpg" width="400" /></a></div>
<br />
And here is the removable part itself. You can see here that it has the recess that fits snugly around the D-SUB25 connector, so that it doesn't get encased in plastic during moulding, but continues to stick out the end, as it needs to do, in order to be a useful cable:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikjj9M1UbdLqwhQfur6wJNji5be4lZrwfNitnErkw1DNf9sA3lt9A6tDJA63eQUD13zmnBufer6tfKYx1EeSI1jlHq1p4US4n8yB7w8HDENaSP433gAMz9fsXi11vjibje3b6M9F5CkUc/s1600/IMG_20170602_115238-overmoulded-cable-and-tool.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikjj9M1UbdLqwhQfur6wJNji5be4lZrwfNitnErkw1DNf9sA3lt9A6tDJA63eQUD13zmnBufer6tfKYx1EeSI1jlHq1p4US4n8yB7w8HDENaSP433gAMz9fsXi11vjibje3b6M9F5CkUc/s400/IMG_20170602_115238-overmoulded-cable-and-tool.jpg" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
Then finally, here it is all assembled (but without a cable inside):<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEia3XlXkci8U1alD9xyi8NIHptQpGY7uWi0HaM1x5j-SruMyFisc3pwaGjFZu-Nxa8KFfG86DEw4kH-5QR-AwXG3UrC8CFAh6PAmNT5IK2LEah0GQaxn0zYm9O8zm8GNR503qsHNbqZ_lc/s1600/IMG_20170602_120856-overmoulded-cable-and-tool.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEia3XlXkci8U1alD9xyi8NIHptQpGY7uWi0HaM1x5j-SruMyFisc3pwaGjFZu-Nxa8KFfG86DEw4kH-5QR-AwXG3UrC8CFAh6PAmNT5IK2LEah0GQaxn0zYm9O8zm8GNR503qsHNbqZ_lc/s400/IMG_20170602_120856-overmoulded-cable-and-tool.jpg" width="300" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
As I have said before, it is so much easier to work with a company who is only a few minutes away, so that if there are problems, they can be quickly inspected and dealt with, and so that communications can be free flowing, so that everything goes well.</div>
<br />
Now we are just waiting on the delivery of our next 45 of these cables, ready for our third trip to Vanuatu.Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com0tag:blogger.com,1999:blog-9068638994894102968.post-68503201757037226722017-08-14T17:30:00.001-07:002017-08-14T17:30:07.546-07:00Assembling injection moulded Mesh Extenders<div class="separator" style="clear: both; text-align: left;">
After various delays on various fronts, we now have in our possession enough components to assemble 40 Mesh Extenders, sufficient for the remaining activities for the Vanuatu Pilot.</div>
<br />
Yesterday, the RFD900+ radios, antennae and Mesh Extender PCBs arrived: <br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDu7DxDtg6AUFFrXXRnp7pFGOLTT4m7lwzBJFlDUkZZoFq1lpqytqj37PqnNQLNxwKxuxCjJpTTKTj4xQUyd1KaJorFYqYEH-ZWfUsq9T5cgnZSJBz39dtgNFvRtmkh40suUQkVTbI1es/s1600/IMG_20170814_115137.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDu7DxDtg6AUFFrXXRnp7pFGOLTT4m7lwzBJFlDUkZZoFq1lpqytqj37PqnNQLNxwKxuxCjJpTTKTj4xQUyd1KaJorFYqYEH-ZWfUsq9T5cgnZSJBz39dtgNFvRtmkh40suUQkVTbI1es/s400/IMG_20170814_115137.jpg" width="400" /></a></div>
<br />
We already had the injection-moulded housings on hand (in the boxes behind the radios and PCBs):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0HT1c3IMHFAmKCpEYgg7PTvff1g1L-5KMburJm_bX52hDrIwe86Zx0exJ0IxuYShpDhheZTizb1fsPjqhA04IQ23ABuhkLcuGSmU2Ac9f3yXchPRGpXIhqjlYrT6y-94tq7-bj935UFU/s1600/IMG_20170814_115440.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0HT1c3IMHFAmKCpEYgg7PTvff1g1L-5KMburJm_bX52hDrIwe86Zx0exJ0IxuYShpDhheZTizb1fsPjqhA04IQ23ABuhkLcuGSmU2Ac9f3yXchPRGpXIhqjlYrT6y-94tq7-bj935UFU/s400/IMG_20170814_115440.jpg" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
First step of assembly is to fit the reverse SMA bulk-head connectors to the cases, and also install the o-rings. While not particularly glamorous, this represents some number of hours of work to do. Karthik, a work placement student, has been placed with us over the next few months, and gets to be the lucky one to do this task:</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgF1XP0NGkRwUdr5B-BC2cP8l9yDA-i_kXzQ3Xd0zMKdGXFcqrIAIl_P9qz8cbut_zO2ruiFgdB4pCxm21SQ-Y_YK2zR-R5vGedGvr0dy2BEQjAL04VdIrBTcApQY6YVSqLWBvXFHPsaCg/s1600/IMG_20170814_170023.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgF1XP0NGkRwUdr5B-BC2cP8l9yDA-i_kXzQ3Xd0zMKdGXFcqrIAIl_P9qz8cbut_zO2ruiFgdB4pCxm21SQ-Y_YK2zR-R5vGedGvr0dy2BEQjAL04VdIrBTcApQY6YVSqLWBvXFHPsaCg/s400/IMG_20170814_170023.jpg" width="400" /></a></div>
<br />
The first afternoon's work, we have 16 units with seals and 2 of the 3 RSMA leads in place:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1fIppyPNeazaEGh1qCLKyzIOhOEeeZwzt-T7xn45RVKjRpvcNrl7B0KEWjbfbVeI4kTWnJgsdl2zcc0hoLfpYgZDl0AMR2J7qCAMXCKUoZhLLbvv9F2XE7z0Id3mvcPXScdk-0NnEric/s1600/IMG_20170814_170136.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1fIppyPNeazaEGh1qCLKyzIOhOEeeZwzt-T7xn45RVKjRpvcNrl7B0KEWjbfbVeI4kTWnJgsdl2zcc0hoLfpYgZDl0AMR2J7qCAMXCKUoZhLLbvv9F2XE7z0Id3mvcPXScdk-0NnEric/s400/IMG_20170814_170136.jpg" width="300" /></a></div>
<br />
After these have been all prepared, we will then proceed with getting the firmware on the PCBs, and radios and bulk storage fitted.<br />
<br />
Our original plan was to use microSD cards, as they are lower power consumption than USB memory sticks, and probably handle power loss better than memory sticks. However, there is a problem with the kernel driver for the microSD card interface, which we have yet to resolve, so we are probably going to stick with USB memory sticks for now. <br />
<br />
Fortunately we were able to get the USB port working in the Mesh Extenders, after a bunch of earlier problems with signal integrity of the USB data traces.<br />
<br />
The only side effect is that we probably won't be able to reliably run these units on solar without a battery connected -- we'll find out for sure as we proceed with testing.Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com0tag:blogger.com,1999:blog-9068638994894102968.post-338347806610161882017-08-03T18:33:00.001-07:002017-08-03T18:33:42.507-07:00Education in Emergencies challengeJust a quick post to say that we have been short-listed in this challenge to find solutions for sustaining education during emergencies:<br />
<br />
<a href="https://challenges.openideo.com/challenge/education-emergencies/refinement/interactive-and-integrated-on-line-education-off-the-grid">https://challenges.openideo.com/challenge/education-emergencies/refinement/interactive-and-integrated-on-line-education-off-the-grid</a><br />
<br />
We feel that Serval is an ideal match for this use-case, especially if an online education system, like Moodle, were extended to support mesh delivery and interaction.<br />
<br />
Please feel free to take a look at our entry, and hit the "love it" button, to help raise the profile of our entry.Paul Gardner-Stephenhttp://www.blogger.com/profile/10150903760695355706noreply@blogger.com2