Getting rid of the Offers4U ad-ware track-ware on Mac

Offers4U is an annoying ad-inserting thing for browsers that pops up large annoying "offer suggestions" and other junk when browsing the web (which also implies that it is taking a bit too close a look at your browsing activities..).

There is plenty of information about how to get rid of it on Windows, however there wasn't any clear information on how to get rid of it from Chrome on a Mac.

After a little exploration I found that it is part of the seemingly-unrelated "SafeFrom.Net helper" that claims only to be a tool to make it easier to download videos from youtube and other streaming sites.

To get rid of it, open chrome://extensions/ in chrome, and then disable the SaveFrom.net helper extension.  I guess if you still want to be able download from youtube and other places (e.g., if you lost the original of a video you uploaded some time ago), then you can turn the extension on, open a new tab, and then disable it again after completing the download.

Working on Mesh Extender firmware

I've been working on various aspects of the Mesh Extender firmware lately.

There are now a bunch of configuration options you can set by putting text files on the FAT partition of the USB memory stick.  This makes it fairly easy to set the configuration in the field, but putting the USB stick into almost any computer.  Gory details for them are on github, but I'll introduce a couple of fun ones now.

otabid.txt lets you specify a Rhizome Bundle ID (BID) that will be assumed to contain over-the-air updates.  Together with the over-the-air-updates/bootstrap program in the mesh-extender-builder repository linked above, this creates a nice easy way to update the main Serval binaries and configuration files on a Mesh Extender.  While this facility was designed to help with the development process, it will also come in handy for general software updates.  By setting the BID to a BID of your own choice, it is possible to have separate groups of Mesh Extenders that get updated separately, so that you can, for example, maintain separate development and production devices.

The second file is monitor.sid, and works with the above so that you can get a Mesh Extender to send a MeshMS message to someone whenever they apply an over-the-air update.  The message contains the GIT commit of the mesh-extender-builder repository that built the over-the-air update, thus providing you with a nice easy way to keep track of which Mesh Extenders have which version of what.

Sending the MeshMS was pretty easy, as we have a nice command line interface for doing so. The only real magic is getting the SID of the Mesh Extender, which we do with a nested shell command to get the SID out of Serval's keyring structure.  Here is the complete bit of code that sends the MeshMS, if monitor.sid exists.

# Now send a MeshMS to whoever cares about this mesh extender

if [ -e /dos/monitor.sid ]; then

  RECIPIENT=`cat /dos/monitor.sid`

  COMMIT="GITCOMMITSTUFF"

  /serval/servald meshms send message `/serval/servald keyring list | tail -1 | cut -f1 -d:` ${RECIPIENT} "Over-the-air-update installed on this mesh extender: git ${COMMIT}"

fi

What was really nice was that this worked first time.  Once I got the SID out of my phone, and put it on the Mesh Extender, and pushed out an OTA update, I had a MeshMS within a minute or so that looked like this:

Note that the timestamp is the creation time of the OTA update, not the time it was applied.

Also, note that these OTA updates are pushed out over the Serval Mesh itself -- the Mesh Extender has no internet access.  In other words, we now have a nice facility that allows us to update Mesh Extenders in the field, without any supporting infrastructure, and with enough feedback so that we know which devices have been updated to what version.

RFD900+ now supported by flash-rfd900 utility

We have a bunch of the newer RFD900+ radios, which have 1 - 2 dB better receive sensitivity than the older RFD900/900a (no plus) versions of the radio.

However, we were previously unable to use these in Mesh Extenders, because we couldn't flash them using our little C utility that we wrote for flashing them on Mesh Extenders.

Using the officially supported flash utility was not an attractive option for us for a couple of reasons: (1) it requires all sorts of dependencies, like python, that ended up adding something like 200MB to the Mesh Extender image, which is normally ~50MB; and (2) it wasn't as fast as we would like.

Our flash-rfd900 utility (https://github.com/servalproject/flash-rfd900) worked fine on the old RFD900 / RFD900a radios, but seemed to brick the 900+ radios.  Some very helpful insights from the folks at RFDesign revealed that the new radio uses 24-bit addresses in the bootloader, instead of 16-bit ones.   The intel hex program file can also contain type-04 records in addition to the usual type-00 records that contain the actual data. Type-04 records indicate which 64KB bank is being addressed, and are necessary for the RFD900+ which has an upgraded micro-controller that has 128KB of program flash, and twice the RAM resources of the older radios.

Adding support for these two features was fairly simple, and after that flash-rfd900 is now working for the RFD900+ radios, so we can finish getting the Mesh Extender firmware image to work with that model of radio.

Debugging RFD900 Ad-Hoc UHF behaviour

So I am getting closer to having working ad-hoc UHF communications between Mesh Extenders with RFD900 family radios running our custom CSMA firmware (not that for now you need a spectrum license from your local regulator to do this at >3mW in most countries).

Now I am debugging why MeshMS messages are not being delivered in reasonable time. It seems that lbard is not making the most sensible choices in terms of which piece of which bundle to transmit.

However, I need a nice way to see what it thinks it is transmitting, without having to stop lbard and run it in debug mode, which of course loses the accumulated state in lbard, and thus might cause the bug to disappear.

To address this, I am implementing a function to dump the current state of lbard in HTML so that it can be read wirelessly using the web server on the Mesh Extender to serve it up.  This will show the calculated priority of the bundles on each node, as well as their information about neighbours and in-progress transfers from neighbours.  Hopefully with that I will be able to track down the remaining priority bugs quite quickly.

Shaking the bugs down with ad-hoc communications via RFD900

Last week I was up at Arkaroola again working on getting the Mesh Extenders to communicate via the RFD900 radios running my new CSMA firmware using our new spectrum license that allows us to avoid frequency hopping at (relatively) high power levels.

The CSMA firmware itself had very few problems to be solved.  The main change I made was introduce a couple of escape sequences to allow setting the power level of each packet to either low (requiring no spectrum license) or high (requiring a spectrum license).

This was part of a general effort to make the power level easily controllable so that I could set the Mesh Extenders to either "normal" or "Arkaroola" mode, without having to reflash them.

In the end, I implemented two tests that the Mesh Extenders perform before sending a packet at high power: First, the file "HIPOWER.EN" must exist on the USB memory stick on the FAT32 partition, and second, the slider switch on the MR3020 PCB must be set to the middle position. If either of those are not set correctly, then packets will be transmitted at low power.  Thus I can either replace the contents of the USB memory stick, or slide the switch to disable hi-power mode when I return from Arkaroola.  Ideally I should modify the lid design of the 3D-printed case so that I can access the slider switch without unscrewing it, but that's really just a matter of convenience.

So after fixing that, and having a frustrating day while I realised I was using the wrong binary for the RFD900 radios, I managed to send a MeshMS message between a pair of Mesh Extenders operating in this way.  There are still some scheduling bugs in LBARD that prevented the confirmation reply from getting back to the sending phone. I also have a list of other relatively minor issues to work through to get it all operating stably, all of which I hope to work through this week ready for the next trip to Arkaroola, where I hope to be able to operate 8 or more Mesh Extenders together using the RFD900 in this ad-hoc mode, and to be able to send and receive MeshMS messages over a fairly large area.  This will be very satisfying and a significant step forward if I can achieve it.

Simple CSMA firmware for RFD900 radios

I have been continuing to work on getting multiple Mesh Extenders communicating nicely via the RFD900 radio.  

As I have previously mentioned, the challenge with this is that the frequency hopping firmware that they normally run cannot work with multiple radios without a defined network controller.  This just isn't possible with Mesh Extenders, where we expect nodes to wander in and out of coverage on a regular basis.

Fortunately, it is allowable to use the ISM915 band in Australia, the USA, New Zealand and a few others without frequency hopping.  To do this, you must however use very wide channels, or limit your transmit power to not more than 3mW EIRP.  Fortunately, as described in my last post, the RFD900 can still transmit a long way on a couple of milli-Watts EIRP.  While at Arkaroola, we can of course transmit at upto 1W EIRP under our scientific spectrum license.  In the longer term, the next model of radio from RFDesign will allow the use of very wide channels, so the scientific license is really just an interim solution. Anyway, enough of the spectrum regulation side of things ...

The normal RFD900 firmware's frequency hopper uses a Time Division Multiplexing (TDM) scheme, which is great for a pair of radios, or a small group with a defined controller, who can control who has what slice of time.  But as mentioned, that doesn't make sense for Mesh Extenders.

Instead, we need a simple CSMA (Carrier Sense Multiple Access) or similar scheme that allows disorganised radios to communicate with each other on a transient and dynamic basis. The downside to this is that the effective bandwidth will be probably less than 1/3 of what can be achieved using TDM (which is part of why the original firmware uses TDM).

Because we can now also have potentially a lot of radios talking, we need a clear way to mark packet boundaries in the serial stream -- for both transmit and receive -- so that we can reliably send and receive whole packets.

To this end, I have culled all the TDM code out of the RFD900 firmware, and implemented some simple packet framing.

The framing on the receive side we had already implemented for the Mesh Extenders. This just puts a simple prefix on each received packet in the serial buffer. That prefix contains a magic value, as well as some useful statistics, mostly signal strength.

For the transmit side, I have changed the firmware so that it holds onto bytes in the TX buffer until it sees "!!".  To include a ! in the packet it must be escaped as "!.".

After a bit of fiddling around, I was able to get all this working fairly nicely.

The only other piece was to add a simple Listen Before Talk (LBT) function that holds off sending a packet until the ambient noise drops below a threshold.  This is to stop us talking over the top of other stations.  This is vital now that we no longer have TDM to keep the radios from talking over the top of each other.  For now I have a hard-wired threshold, but I will make this configurable soon, and if I can think of a good approach, I will make it dynamic based on the background noise on the channel, so that it doesn't stop transmission in busy urban areas, but still stops talking over the top of distant stations when in low-noise environments.

(In fact, this whole effort was triggered by problems using the existing LBT code in version 1.6 of the RFD900 firmware, on which our Mesh Extender version is based. Basically if you set the LBT register on that firmware, the radio stops responding.  This may well have been fixed in later versions of the firmware.)

Anyway, while there is more work to be done, you can see the work so far at https://github.com/gardners/SiK/tree/csma-packetised.

Testing RFD900 radio at Arkaroola

Last week I was up at Arkaroola for Science Week, and took the opportunity to test the RFD900 radios up there in preparation for some upcoming Mesh Extender testing there.

The idea of the tests was to try the RFD900s out in single-channel (non-hopping) mode, now that we have an experimental spectrum license issued for Arkaroola.  This means that we can run the RFD900 at full power, without frequency hopping enabled, as a precursor to the new radio the RFDesign folks are working on, that can be operated in an equivalent mode without the special spectrum license.

The benefit of all this is that we can completely avoid the synchronisation issue between Mesh Extenders, and thus allow more than just pairs of Mesh Extenders to communicate. This is a big step forward, and one that has been a long time coming!

So, I configured the radios down here in Adelaide, but set to 1mW, so that we would not be in breach of the LIPD Class License under which the RFD900 is typically operated.  This let me confirm that I had them setup properly.

Then, up at Arkaroola, I put one radio in the window of one of the buildings in the Arkaroola Village, and drove up towards Coulthard's Lookout, about 4 - 5km away, to confirm that we could get that kind of range.  We were not expecting any problems, but just wanted to make sure that they worked in the rocky terrain that is full of iron and odd shaped rocks on the ground that can mess with radio propagation.

A little way before Coulthard's Lookout there is a ridge that has excellent line of sight back to the village, and is almost 4km from the village.  So we stopped there to test performance, which confirmed that we were losing only a small percentage of packets.  We did also discover that horizontal polarisation was pretty much useless, and that it was the vertical polarisation that was doing all the work.

Here I am in my "other" office. The village is just visible in the distance.

We then went up to Coulthard's Lookout, and were receiving only perhaps 50% of packets, which rather surprised us, as we had expected better performance, however, it was adequate.  It was only the next morning driving home that I realised that I had left the radio set to 1mW !  So the RFD900 was still getting 50% of packets through at 1/500th the TX power that we could use.

Given that every 4x increase should double the range, this means that we should expect 50% packet throughput at a range more like 85km -- and this is with both radios on the ground. Previous tests indicating ~80km range of the RFD900 has involved one radio being in the air on a UAV.

Part of the reason for the great performance (apart from the great design and implementation of the RFD900) is the REALLY low level of background noise at Arkaroola.  Based on the RSSI indications of the RFD900 it is probably 10 - 20dB lower than in Adelaide.  But this isn't news to us, as we know from previous work that the noise floor at 2.4GHz - 5GHz is around -100dBm, compared with around -80dBm here in Adelaide.

Apparently it was family day in the office.

Working on the Mesh Extender UHF Radios and other things

It's been a while since the last post, but we have been busy in the background on some exciting things:

With the help of the NLnet Foundation, we are making the first steps towards an iOS port of the Serval Mesh.

With the help of USAID, we are working on "slow-media transports" for Android, in particular using Bluetooth device names, and hopefully, Wi-Fi Direct service discovery information, to carry mesh traffic. This is really significant, because it allows carriage of ad-hoc communications on non-rooted devices.  The trade-off is that it is much slower than ad-hoc Wi-Fi.

The third thing we are working on at the moment is making Mesh Extenders work in groups of more than two units.

Historically we have been stuck with pairs because the radios we are using are required to frequency hop when using the ISM 915MHz band available here in Australia, New Zealand and the USA.

We are now actively working to remove this limitation, using the new radios that RFDesign are creating that allow broader spectrum occupancy that will allow them to qualify as spread spectrum devices under the relevant FCC and ACMA rules (LIPD class 45 for the ACMA rules).  This means that we can drop the frequency hopping that causes all the synchronisation issues for ad-hoc and transient groups of Mesh Extenders that currently exist.

In other words, this will allow people to carry a Mesh Extender, and for it to automatically mesh with any other Mesh Extenders that come into range from time to time.  The practical limit will probably be 5 - 25 Mesh Extenders in range at any one time -- something I hope to quantify better in the near future.

Now, the only trouble is that those radios are not yet finished and available.

So we have embarked on an interim solution, by obtaining a spectrum license from ACMA that will allow us to operate the RFD900 radios without frequency hopping at upto 1W EIRP in our favourite piece of Outback, the Arkaroola Wilderness Sanctuary.

Our scientific spectrum license which was granted earlier in the week allows us use of 915MHz - 928MHz at upto 1W anywhere within 130km of 29'52"35S, 138'52"25E, which is great because this affords us great flexibility for testing in a variety of settings.  

The license stands for the next 12 months, which will hopefully be time enough for us to transition to the new RFDesign radios.

We are also open to making use of this license open to other people who would like to experiment in similar ways in that area (but note that it is really quite remote, 400km from the nearest large town, and centred about 130km from the nearest settlements with any facilities).  We would also need to make sure that ACMA and the University are comfortable with such an arrangement, but I am happy to explore this if anyone is interested.

Meanwhile, I hope to modify the configuration of some radios, and put the latest servald on a couple of Mesh Extenders and, all things going well, to perform some rudimentary testing at Arkaroola this coming week.

Update: This map shows the approximate area of the license.

Mapping assistance for Vanuatu Cyclone Response

We have had a request from the head of the Disaster Research Centre here at Flinders University asking for volunteers to help re-map Vanuatu following the category 5 cyclone that hit there in the last few days.

Here is the call for assistance:

“Dear MicroMappers,

The United Nations has just activated MicroMappers to support their rapid damage assessments in Vanatu following the Category 5 Cyclone. You can accelerate the UN's efforts by donating a few minutes of your time. Simply click here: 

http://MicroMappers.org

No prior experience necessary. If you can click "Like" on a Facebook picture or video, you can be a Digital Humanitarian.

Thank you!"

OSX Yosemite Printer keeps pausing, and gets stuck on "Ready to print"

If you are printing to a PaperCut server, apparently printing can have trouble from a mac running Yosemite, due to the improved security in CUPS 2.0, which is part of Yosemite.

The interim solution is to relax the sandboxing policy in CUPS 2.0, so that it behaves like it used to.  Of course, it would be better if PaperCut fixed the underlying problem in their software.

The short version of what to do is:

sudo sh -c 'echo "Sandboxing Relaxed" >> /etc/cups/cups-files.conf'
sudo launchctl stop org.cups.cupsd

The long version can be found at:

http://www.papercut.com/kb/Main/MacOS1010YosemiteKnownIssues

MDP, Rhizome, MeshMS and VoMP over Bluetooth

Under a grant from US-AID we are in the process of adding some low-bandwidth transports to the Serval Mesh.  Ultimately, this will hopefully include Wi-Fi Direct service directory, Bluetooth Device Names, as well as using those transports directly.

What Jeremy has achieved so far is to use Bluetooth with an automated peer discovery.  That is, if two phones running Serval Mesh have bluetooth turned on, and at least one of them is marked discoverable, then Serval Mesh on the phones will automatically create a bluetooth serial connection, and start transferring data as though it were another wireless interface, which, after all, it is.

The following photos show Andrew and Jeremy making an authenticated, encrypted VoMP call over Bluetooth:

The in-call latency is probably around 800ms, so quite usable.

And here we have an upside-down MeshMS conversation.  MeshMS messages deliver in just a couple of seconds if there is no other Rhizome traffic.

More posts as we make further progress.

Mac OSX "Hold for authentication" when trying to print

Today I tried to get printing to work from my mac (running Yosemite) to the printers here at work.  The printers are accessed using SMB.

I could easily add the printers, but when I tried to print, it would accept my username and password for the printer, but then revert straight back to saying "Hold for authentication".

Deleting the printers and recreating them didn't fix it.
Nor did deleting the associated entries from the OSX keyring help.

Thus the usually recommended steps failed to fix the problem.

It took a couple of hours of research and experimentation to discover that this page had most of the information needed to solve the problem.

The problem in my case was caused by the print system on the mac from failing to properly negotiate authentication in some way. But here is how I fixed it:

Step 1: Try to print something to this printer.  Then open the print queue for the printer, and click on the refresh (curly-arrow) icon to the right of the job name.  It should prompt you for your username and password for the print server, and then revert to "Hold for authentication".

This is really important. If you don't try to send a print job and have it stuck in "Hold for authentication" first, strange things can still happen after following the procedure below.  For me, I found that the print job would try to print, but the print queue would then immediately become paused, with the print job marked "Ready to print".  Very frustrating.

Step 2: Open a terminal window.  Everything we do from here will be done from in the terminal window.

Step 3: Find out the name of the printer on the command line by typing the following and pressing return:

lpstat -s

Step 4: Find the printer in question in the output. You will see that the names of the print queues have any fancy characters replaced with an underscore.  So your print queue name might have lots of underscores in it.  For example, my printer called "Tonsley : mono" appears in the output of lpstat -s below as "Tonsley___mono" (highlighted below):

$ lpstat -s

system default destination: Tonsley___colour__A3_in_tray_2_

device for GenericScanner: usb://00000000-0000-0000-FA13-000000000000

device for LaserJet_6MP__Jet_Direct_: socket://192.168.1.200/

device for Lexmark_6200_Series: usb://Lexmark/6200%20Series?serial=21B1280030008E5

device for Loopback: socket://127.0.0.1/

device for Samsung_CLP_310_Series: usb://Samsung/CLP-310%20Series?serial=149RBAFZ400869Z

device for Samsung_CLP_315_Series___alfred: dnssd://Samsung%20CLP-315%20Series%20%40%20alfred._ipp._tcp.local.

device for Samsung_ML_1640_Series: usb://Samsung/ML-1640%20Series?serial=3511BAFS501610N.

device for Tonsley___colour__A3_in_tray_2_: smb://tonsprint.isd.ad.flinders.edu.au/mdf-colour

device for Tonsley___mono: smb://tonsprint.isd.ad.flinders.edu.au/mfd-bw

$

Step 5: Force the mac to use username,password authentication for this printer by typing the following command (but don't forget to change the printer name from Tonsley___mono to the name of your print queue that you obtained in the previous step).  

sudo lpadmin -p Tonlsey___mono -o auth-info-required=username,password

When you hit return after typing this command you will be asked for your mac password.  Type it in.

Step 6: Attempt to print using the printer.  It should now work.  If it doesn't try deleting the printer and following the sequence again.  If the print queue keeps pausing itself and saying the print job is "ready to print", read the important note on step 1.

If this information helps you, please consider donating to http://servalproject.org.

Bleeding Edge Mesh Extender kits finally ready for shipping

It has taken us much longer than we had hoped, but we finally have the Mesh Extender kits ready to post out to our generous supporters from our Indiegogo campaign.  As a token of appreciation to our long suffering supporters, we have included a pair of Serval test phones into the kits so that everything they need is there and setup for when it arrives.  Here is the contents of each kit:

I took a couple of shots during the testing of the last pair I packed this morning here at the University's new building at the Tonsley site.  Here is a Mesh Extender and test phone up on our 4th floor lab:

Then took another down to the ground floor to make sure the UHF radios still had link:

Finally, all the units were packed.  Once I have reconfirmed everyone's postal address, we will send these out.