I'm playing around with Electron lately.

One of the things I wanted to do was build the Electron Application for Windows on my Linux development box, which runs Debian.

I used electron-builder for building the app.

In this post I'll focus on making electron-builder build the app for windows and will ignore setting up electron, electron-builder and making the electron app.

TL;DR

You need Wine 1.8+. Debian will install Wine 1.6.2.

To install Wine 1.8+ on Debian:

• Add Jessie Backports to /etc/apt/sources.list
• See: Installation from Jessie Backports

Configuration

To make electron-builder build for windows, the following is needed in package.json:

  "build": {
    "appId": "com.aaronlenoir.gnucash-reporter",
    "win": {
      "target": [
        "zip"
      ]
    }
  },

For convenience, I have the following scripts too:

  "scripts": {
    ...
    "pack": "build --dir",
    "dist": "build",
    "pack-win": "build --dir --win",
    "dist-win": "build --win"
  },

That way, I can run npm run pack-win to build for windows.

Building

Electron-builder needs Wine in order to build for windows. So I had installed it using apt:

apt-get install wine

Problem 1: Wine version

The first run, I got the following error:

$ npm run pack-win

> gnucash-reporter@0.0.1-alpha pack-win /home/user/electron/gnucash-reporter
> build --dir --win

Skip app dependencies rebuild because platform is different
TypeError: Invalid Version: it
    at new SemVer (/home/user/electron/gnucash-reporter/node_modules/semver/semver.js:293:11)
    at compare (/home/user/electron/gnucash-reporter/node_modules/semver/semver.js:566:10)
    at lt (/home/user/electron/gnucash-reporter/node_modules/semver/semver.js:600:10)
    at /home/user/electron/gnucash-reporter/node_modules/electron-builder/src/packager.ts:473:7
...

Note the strange message: TypeError: Invalid Version: it

Looking at the stack trace, it looked like it was checking the version of Wine. But apparently the version was not a valid Semver version number: "it".

The mystery was solved however, asking wine for its version:

$ wine --version
it looks like multiarch needs to be enabled.  as root, please
execute "dpkg --add-architecture i386 && apt-get update &&
apt-get install wine32"
wine-1.6.2

Now it was clear where "it" came from. So I followed the instructions and ran:

dpkg --add-architecture i386 && apt-get update && apt-get install wine32

After that, wine had the following version:

$ wine --version
wine-1.6.2

So I ran npm run pack-win again:

And received the error:

Error: wine 1.8+ is required, but your version is 1.6.2, please see https://github.com/electron-userland/electron-builder/wiki/Multi-Platform-Build#linux

Problem 2: Getting Wine 1.8

I'm not very familiar with debian, but I always assumed apt would give me the latest stable version of applications.

I followed the instructions on: https://github.com/electron-userland/electron-builder/wiki/Multi-Platform-Build#linux

But they don't work well on Debian, since it can't find the appropriate packages.

I found these instructions on the debian wiki, which worked:

• Add this line line to /etc/apt/sources.list

deb http://httpredir.debian.org/debian jessie-backports main

• Update the package lists

sudo apt-get update

• Then, for 64-bit systems:

sudo apt install \
      wine/jessie-backports \
      wine32/jessie-backports \
      wine64/jessie-backports \
      libwine/jessie-backports \
      libwine:i386/jessie-backports \
      fonts-wine/jessie-backports

After this:

$ wine --version
wine-1.8.6 (Debian 1.8.6-5~bpo8+1)

Results

After this the build succeeded:

user@debian:~/electron/gnucash-reporter$ npm run pack-win

> gnucash-reporter@0.0.1-alpha pack-win /home/user/electron/gnucash-reporter
> build --dir --win

Skip app dependencies rebuild because platform is different
Packaging for win32 x64 using electron 1.4.15 to dist/win-unpacked
user@debian:~/electron/gnucash-reporter$ ls
build  dist  install-builder-deps.sh  LICENSE  node_modules  package.json  README.md  src  test
user@debian:~/electron/gnucash-reporter$ cd dist/
user@debian:~/electron/gnucash-reporter/dist$ ls
linux-unpacked  win-unpacked

win-unpacked, in windows, looks like this:

And running the executable ran the application without any issues.

.

Conclusion

You need:

• Electron
• Electron-Builder
• Wine 1.8+

Debian runs Wine 1.6.2 standard, and you need to add the "jessie backports" package source to your system in order to install Wine 1.8+.

For reference, the practice application I'm building in electron is on github: https://github.com/AaronLenoir/gnucash-reporter.

This month I was able to get https://blog.aaronlenoir.com up and running.

This means I got:

• a domain name (aaronlenoir.com);
• an easier URL for my blog (it used to be https://blog-aaronlenoir.rhcloud.com);
• an SSL Certificate

My main motivation was that if I need to move my blog to some other hosting, the old URL's will still work. I'd just point the domain to another host.

With the rhcloud.com URL, this would not be possible.

I'm still hosting this blog on RedHat's OpenShift using Ghost. Setting it all up was interesting. So here's what needed to be done.

Getting a domain name

I never bought a domain name, so I was unsure what would be involved.

I really wanted aaronlenoir.be but that was already taken.

At first I looked at dnsimple, because I heard a lot of good thing about them. But I'd probably not need most of their services. They would be a better fit if you need to manage more domains. I just needed one domain name.

Then I looked at both GoDaddy and Namecheap. In general, I read more postive things about Namecheap than GoDaddy. From what I understand, GoDaddy is really cheap but they charge extra for almost any additional feature.

I went with Namecheap and bought aaronlenoir.com for $20 for two years. This includes a 1 year WhoisGuard subscription. WhoisGuard ensure that when you look up my domain information, you don't see my personal details.

That was really all there was to it.

Point the Domain to the blog

When you get a domain, you can create a number of sub-domains (for example blog.aaronlenoir.com).

Since I already had hosting, it was a matter of pointing blog.aaronlenoir.com to blog-aaronlenoir.rhcloud.com).

Then in the OpenShift Web Console, open the app, click on "change alias" and add the domain.

It takes a while for this change to have effect. After about half an hour I was able to visit my blog from the domain.

Fixing Ghost redirect

When I visited: http://blog.aaronlenoir.com I was always redirected to https://blog-aaronlenoir.rhcloud.com/

The reason for this was the configuration of the blog (Ghost). In the config file, I had to hard-code the domain name, instead of using the OpenShift environment variable. In config.js change

url: 'https://'+process.env.OPENSHIFT_APP_DNS,

to

url: 'https://blog.aaronlenoir.com',

I could've added some environment variables, but it's fine.

Applying Https

So that is step 1, and it works when I use "http". But I want to go for HTTPS.

This used to be a pain, but now this should be quite easy to do, for no money.

Getting a Certificate

Let's Encrypt!

The first thing to do is getting a certificate. Traditionally, you'd had to purchase one from a Certificate Authority.

Luckily, now there is "Let's Encrypt" (support them!). They provide free certificates and potentially automatically renew them.

The main concept is that if you can prove you own the domain then you get a certificate. This can be proven by uploading a specific file to the domain. Let's encrypt the checks from their end that the file is available and then provides you with a certificate.

If you have root or adminstrator access to the server, this can be fully automated. In my case, on OpenShift, I do not have this and I cannot run the automatic tools.

So I had to perform this manually.

Getting a Certificate on OpenShift Manually

To use Let's Encrypt I must be able to upload a specific file to the following location:

• http://blog.aaronlenoir.com/.well-known/acme-challenge

There are some problems:

• I need that folder somewhere in my OpenShift Gear
• By default, Ghost won't serve files from that directory.

Getting the acme-challenge folder on OpenShift

Cloning my gear repo from OpenShift, the file .openshift/action_hooks/deploy can be edited to ensure the required folders are present. The following needs to be added to the script:

if [ ! -d "$OPENSHIFT_DATA_DIR/.well-known" ]; then
        mkdir -p $OPENSHIFT_DATA_DIR/.well-known
fi
if [ ! -d "$OPENSHIFT_DATA_DIR/.well-known/acme-challenge" ]; then
        mkdir -p "$OPENSHIFT_DATA_DIR/.well-known/acme-challenge"
fi
echo "Symlinking Lets Encrypt dir."
rm -rf $OPENSHIFT_REPO_DIR/.well-known
ln -sf $OPENSHIFT_DATA_DIR/.well-known $OPENSHIFT_REPO_DIR/.well-known

Now this script will run every time I deploy an update to the Gear.

Uploading files to the Data Dir

This can be done through an scp command. For example:

scp SomeFileName.txt 337a20c29d20405197d0fac456d60dc4@blog-aaronlenoir.rhcloud.com:/var/lib/openshift/337a20c29d20405197d0fac456d60dc4/app-root/data/.well-known/acme-challenge

Making Ghost host the acme-challenge folder

For this, I had to do a dirty trick. Hopefully there is a solution in later versions of Ghost.

First I added a setting to the .config file, to the "paths":

paths: {
    contentPath: path.join(__dirname, '/content/'),
    acmeChallenge: path.join(__dirname, '/.well-known/acme-challenge')
}

I added the acmeChallenge setting.

Secondly, I have to tell Ghost it's ok to serve this folder as static files. This is currently done in the Ghost code itself. This is really annoying, because updating Ghost will probably break this.

In node_modules/ghost/core/server/middleware/index.js add a line:

blogApp.use('/.well-known/acme-challenge', express['static'](config.paths.acmeChallenge, {maxAge: utils.ONE_YEAR_MS}));

Running the Let's Encrypt client

In my case, I can't run the Let's Encrypt client on my actual webserver. So I'll have to run it on a local machine. Apart from that, it's very straight forward:

• git clone https://github.com/letsencrypt/letsencrypt
• cd letsencrypt
• ./letsencrypt-auto -a manual -d blog.aaronlenoir.com

From that point, it's a matter of following the instructions on the screen, until it asks you to upload the file to the server.

Then, the requested file can be uploaded using the command shown before (but then with the correct file):

scp SomeFileName.txt 337a20c29d20405197d0fac456d60dc4@blog-aaronlenoir.rhcloud.com:/var/lib/openshift/337a20c29d20405197d0fac456d60dc4/app-root/data/.well-known/acme-challenge

After ensuring the file is accesible the following appeared:

So the certificates are ready in /etc/letsencrypt/live/aaronlenoir.com

They are valid for three months. So every three months (or a bit earlier), I'll have to redo this process.

Again, this could be fully automated in many cases.

Installing the Certificate

In OpenShift's web management console, you go to the app and choose "change alias" again and upload a certificate:

• fullchain.pem for "SSL Certificate"
• privkey.pem for "Certificate Private Key"

And click Save.

The result is that I was able to visit https://blog.aaronlenoir.com and not get certificate warnings.

SSL Labs Results

Running this through SSL Labs was encouraging:

Renewing the Certificate

So after three months, the certificate expires. Because OpenShift doesn't support renewing the certificates automatically, I have to do this myself every time.

It is the same:

• Run Let's Encrypt: ./letsencrypt-auto -a manual certonly -d blog.aaronlenoir.com
• Upload the requested file using scp
• Upload fullchain.pem as "SSL Certificate" and privkey.pem as "Certificate Private Key"

Since last time, the letsenrypt client does complain if you don't pass "certonly" too.

Conclusion

Because I choose to self-host my blog, I'm forced to do some fiddling. But in general, the process was a lot less painful and cheaper than I imagined. Even if I have to get the Let's Encrypt certificates manually.

Now I'm set for the future, if I ever choose to move my hosting elsewhere.

I must urgently update to a higher version of Ghost, but that's going to be a lot more painful I think.

Resharper tells us to convert for(each) loops to Linq. An argument I read was that Linq queries offer options for parallelization.

Iterate over each item in parallel, making it faster on hardware that offers more than one processor.

To put this to the test I made a quick experiment.

The Problem

To simulate a problem, I want to compare two byte arrays and count how many items are the same.

int Compare(byte[] dataA, byte[] dataB)

Implementations

First, the naive approach:

static int CompareNaive(byte[] dataA, byte[] dataB)
{
    var result = 0;

    for(var i = 0; i < dataA.Length; i++)
    {
        if (dataA[i] == dataB[i]) { result += 1; }
    }

    return result;
}

Then using Linq:

static int CompareLinq(byte[] dataA, byte[] dataB)
{
    return dataA
            .Select((data, index) => dataB[index] == data ? true : false)
            .Count((result) => result);
}

I suspect there are multiple ways to do this in Linq, and I might not have taken the most efficient route.

Now, with Linq and some parallelization magic:

static int CompareLinqParallell(byte[] dataA, byte[] dataB)
{
    return dataA
            .AsParallel()
            .Select((data, index) => dataB[index] == data ? true : false)
            .Count((result) => result);
}

The AsParallel expression enables parallelization before the Select.

Results

For a 1000 iterations of each function, comparing two 1 MB (1024 * 1024 bytes) arrays.

CompareNaive: 4153
CompareLinq: 4153
CompareLinqParallell: 4153
To begin press return.

Starting 'CompareNaive' ...
Ran 'CompareNaive' 1000 times in 00:00:00.8461644 (average of 0,8461644 ms).
To begin press return.

Starting 'CompareLinq' ...
Ran 'CompareLinq' 1000 times in 00:00:14.7678043 (average of 14,7678043 ms).
To begin press return.

Starting 'CompareLinqParallell' ...
Ran 'CompareLinqParallell' 1000 times in 00:00:03.7464192 (average of 3,7464192 ms).

CPU Usage

The naive approach (1) causes a little spike (around 40%) in some of the processors.

The regular Linq approach (2) uses quite a lot of processors for quite a long time. But not all of them.

The parallell Linq approach (3) takes all processors to 100%.

Note: this is on a 4 core machine with 8 logical processors.

Caution

The AsParallel expression can't just be put anywhere. Consider these two queries:

dataA
    .AsParallel()
    .Select((data, index) => dataB[index] == data ? true : false)
    .Count((result) => result);

dataA
    .AsParallel()
    .Select((data, index) => dataB[index] == data ? true : false)
    .AsParallel()
    .Count((result) => result);

The latter is considerably slower. It takes a few seconds to execute it once for the 1 MB byte array.

Additionally, if the data set is really small, the overhead of AsParellel might not be worth it at all.

Measuring Execution Time

To be complete, here's the crude way I measure the execution times:

static void Measure(string name, Action action, int iterations)
{
    var sw = new Stopwatch();

    Console.WriteLine($"Starting '{name}' ...");

    sw.Start();
    for (int i = 0; i < iterations; i++)
    {
        action();
    }
    sw.Stop();

    var averageElapsed = sw.Elapsed.TotalMilliseconds / iterations;

    Console.WriteLine($"Ran '{name}' {iterations} times in {sw.Elapsed} (average of {averageElapsed} ms).");
}

Conclusion

For this problem, the AsParallel expression will speed things up compared to a non-parallel Linq query.

The naive approach is the fastest and the least resource intensive. In my opinion, it's also the easiest to read.

Of course, it depends a lot on what you're doing in the loop. Linq is really powerful and in some cases makes your code easier to read and maintain.

Additionally, if the query is not executed frequently or on small data sets, there's probably no point in worrying about this.

In Het Nieuwsblad van Zaterdag 29 oktober 2016 stond een interessant artikel over privacy, met name privacy online.

Als onderdeel daarvan stonden er een aantal goede tips om je privacy online te beschermen.

Jammer genoeg stond er niet bij hoe je die praktisch kan toepassen. Ik wil dit hier even aanvullen met praktische tips.

Het volledige artikel lees je hier (als je een abonnement op Het Nieuwsblad hebt ...)

Wachtwoorden

"Gebruik krachtige wachtwoorden."

Het belangrijkste bij een wachtwoord is de lengte. Hoe langer hoe veiliger. Elk extra karakter verhoogt de veiligheid exponentieel. Voor je belangrijkste accounts maak je best een lang wachtwoord, met ongebruikelijke woorden en enkele getallen of speciale tekens.

Voor een lang wachtwoord te maken dat ook makkelijk te onthouden is begin je best met een zin. Liefst een zin die niet echt logisch is, maar wel te onthouden.

• Bijvoorbeeld: Vrolijke microgolf danst in 1980 in Vlaanderen?

Gebruik, zoals ook in het artikel vermeld, nooit op verschillende sites hetzelfde wachtwoord. Makkelijker gezegd dan gedaan natuurlijk.

Eén manier is complexe wachtwoorden bij houden in een boekje. Dat is relatief veilig als je het boekje niet overal laat rond slingeren. Immers moet je al fysiek aanwezig zijn om jouw wachtwoorden te onderscheppen.

Iets praktischer is een "Password Manager". Dat is een programma waar je in een beveiligd bestand bij houdt welke gebruikersnaam en wachtwoord je voor welke site gebruikt.

Het bestand is beveiligd met één sterk wachtwoord. Je moet dus maar één moeilijk wachtwoord onthouden, terwijl je toch moeilijke én unieke wachtwoorden op elke site kunt gebruiken.

Zelf hou ik dit moeilijker wachtwoord in mijn hoofd. Voor enkele andere belangrijke accounts hou ik deze op papier bij. Alle andere wachtwoorden zitten in een password manager.

Die applicaties gaan ook voor jou willekeurige complexe wachtwoorden genereren.

De meest eenvoudige in gebruik is "LastPass". Deze kan je op verschillende toestellen gebruiken en integreert in jouw browsers zoals Chrome en Firefox.

Eén opmerking, LastPass bewaart jouw beveiligd bestand op hun systeem. Dat is een vorm van vertrouwen in LastPass. Het is bovendien niet altijd een gratis service.

Een gratis alternatief is KeePass. Dit is een programma dat het beveiligde bestand lokaal op je PC bij houdt. Het is iets minder eenvoudig in gebruik als LastPass.

Een ander uitstekend alternatief is 1Password. Maar dat is iets moeilijker in gebruik.

Updates

Als je applicaties hebt die zelf vragen om te updaten, dan doe je dit best. Doe dit NOOIT als je op een website komt en die vraag krijgt vanuit de websites.

Dit geldt ook voor mobiele apps op je tablet of smartphone.

Schakel Windows Updates in, als je Windows gebruikt. Dat is soms vervelend maar erg belangrijk.

Dit geldt ook voor Apple en hun mobiele platformen.

Ook voor Android, maar daar zijn updates verre van evident, en dat is erg jammer.

Maak Geregeld een Backup

Update oktober 2017: CrashPlan heeft besloten niet langer particuliere klanten te bedienen. Ik ben overgeschakeld op Arq Backup in combinatie met B2 (Backblaze) cloud storage. De bestanden worden ook hier lokaal versleuteld.

Ik weet niet direct hoe dat je privacy ten goede komt, maar het is zeker een goede tip.

Een populaire vorm van cybercriminaliteit is je bestanden versleutelen en dan losgeld vragen. Dit heet "Ransom Ware".

Bij de meeste Ransom Ware heb je zonder backups weinig keuze: betalen of je bestanden kwijt zijn.

Met backups heb je de kans een kopie van de oude bestanden terug te halen.

Klinkt simpel, kopieer af en toe alle bestanden naar een externe schijf. Maar dat is snel erg vervelend. Bovendien kan je per ongeluk de oude backup met een reeds versleutelde kopie overschrijven voor je beseft dat je slachtoffer bent van deze Ransom Ware. Dan zijn je backups waardeloos.

Je moet dus verschillende backups voor verschillende tijdstippen hebben.

Er zijn services die dit voor jou kunnen doen.

Ikzelf heb erg goede ervaringen met "CrashPlan". Deze applicatie gaat jouw bestanden online bij houden. Bovendien houdt het meerdere kopies doorheen de tijd bij. Dus zelfs als je laatste kopie versleuteld is, kan je verder terug in de tijd gaan.

Je kan daarenboven de bestanden eerst zelf lokaal versleutelen met een wachtwoord naar keuze. Zo worden jouw bestanden alleen versleuteld opgeslagen bij CrashPlan.

Opgelet, dit vereist vertrouwen in de backup service van CrashPlan. Bovendien is het niet gratis. Maar voor € 50 / jaar krijg je onbeperkte backup ruimte.

Er zijn zeker nog andere services (bij Apple bijvoorbeeld). Maar CrashPlan was de enige die ik had gevonden die je toelaat je bestanden lokaal te versleutelen.

Openbare WiFi Netwerken

Handig als je ergens onderweg zomaar op een WiFi netwerk kan.

Maar opgelet, deze netwerken zijn niet altijd te vertrouwen. Zelfs als de eigenaars ter goeder trouw zijn kan het geïnfecteerd zijn met Spyware, waardoor jouw gegevens kunnen worden gestolen.

Het artikel vermeldt dat je er moet op letten dat de websites die je bezoekt via https gaan, zeker als je daar persoonlijke gegevens in deelt (zoals wachtwoorden).

Dit zie je in de adresbalk van de browser, aan het "groene slotje". Hier nog even een overzicht van hoe dit eruit ziet in de verschillende browsers.

Firefox

Chrome

Microsoft Internet Explorer

Microsoft Edge

Brave

Opera

Extra: 2-Factor Authentication

Dit stond niet vermeldt in het artikel, maar veel grote sites bieden je de mogelijkheid tot 2 factor authentication.

Dat betekent dat je naast je username en wachtwoord ook nog een code moet ingeven die wordt gegenereerd op basis van een geheime code die je bezit of die je via SMS wordt doorgestuurd op het moment dat je je wilt aanmelden. Bijvoorbeeld via een app zoals "Google Authenticator".

Op die manier kan men niet op jouw account inloggen zelfs al hebben ze jouw wachtwoord.

Extra: Hoe weten dat je wachtwoord is gestolen?

Als een website een datalek heeft dat publiek wordt gemaakt, is er een service "Have I Been Pwnd". Als je je daar inschrijft krijg je een e-mail als jouw e-mail (en meestal ook wachtwoord) in zo'n datalek optreedt.

Extra: Privacy Instellingen

Of je nu Windows, Android, iOS of iets anders gebruikt: controleer eens bij de instellingen welke informatie er zoals wordt gedeeld met de verschillende apps die je gebruikt.

Vaak wordt informatie gedeeld en opgeslagen over jou, terwijl dat helemaal niet nodig is.

Opgelet: Vertrouwen

In alle gevallen moet je software, websites of bedrijven vertrouwen. Stel je altijd de vraag hoeveel vertrouwen je in iets wilt stellen.

Bijvoorbeeld, zelfs met 2 factor authenticatie: kan je de website die het aanbiedt vertrouwen om dit correct te doen, zodat het niet omzeilt kan worden? Het antwoord is "niet 100%".

Wees, zoals het artikel ook zegt, altijd zuinig met je gegevens, zowel online als in het dagelijkse leven. Zij worden heel vaak opgeslagen en doorverkocht. Zelfs als ze niet worden verkocht is het vrij waarschijnlijk dat ze vroeg of laat gelekt worden.

Conclusie

Het artikel had enkele goede tips maar gaf niet veel hulp om deze ook effectief toe te passen.

Hopelijk geeft deze post nuttige info bij de goede tips uit het artikel van Het Nieuwsblad.

Ikzelf gebruik altijd een ad-blocker (ublock origin) op het Internet. Dit zorgt voor sneller ladende websites, minder afleiding, meer privacy en minder kans op "virussen".

Een redelijk argument tegen ad-blockers is dat sommige websites hierdoor inkomsten mislopen. Zij bieden gratis diensten aan maar krijgen geen reclameinkomsten.

Er is momenteel een experimentele browser (zoals Firefox, Chrome of Internet Explorer) aan het proberen een alternatief model aan te bieden: Brave.

Micro Payments

Kort gezegd: de browser ziet welke websites je bezoekt en verdeelt een bepaald bedrag over die websites. Websites die je vaak bezoekt zullen een groter deel van de koek krijgen.

Je kan zelf je budget instellen dat je wilt verdelen, per maand.

De betalingen zouden redelijk anoniem zijn.

Er is geen expliciete deelname nodig van de bezochte websites. Wanneer Brave ziet dat een bepaald bedrag naar een website zou kunnen gaan en er is nog geen betaalplan, nemen zij contact op met die website zodat ze het geld kunnen over maken.

Opgelet, dit alles is nog in test fase.
Ik wou dit graag eens uittesten. In deze post het relaas.

Stap 1: Brave Installeren

Brave is gelijkaardig aan andere browsers. Het verschil zit hem vooral dat ad block, anti-tracking e.d. functionaliteiten ingebouwd zijn. Zo hoef je geen bijkomende extensies te installeren.

Brave is nog niet even aangenaam werken als Firefox en Chrome, moet ik wel zeggen. Maar op het moment van schrijven zaten ze ook pas aan versie 0.12.1.

Bij de instellingen zijn de details te vinden om het betaalsysteem in te schakelen. Standaard staat het uitgeschakeld.

Stap 2: Brave Payments inschakelen

Brave Payments werkt via "Bitcoin". Dat is een vorm van elektronisch geld, gebasseerd op wiskunde en enryptie. De werking uitleggen gaat me wat ver. De basis is dat je een lokale portefeuille ("wallet") hebt. In de praktijk is dit gewoon een bestandje op je computer.

Zo'n - lege - portfeuille wordt aangemaakt als je Brave Payments inschakelt.

De eerste stap is om Bitcoin in de portefeuille te krijgen.

Stap 3: Bitcoin aankopen

Er zijn verschillende diensten waarmee je Bitcoin kan aankopen. Hou er rekening mee dat Bitcoin een koers heeft, net zoals elke ander munt.

Brave biedt je de keuze tussen Bitcoin aankopen met een bancontact kaart of credit kaart. Alternatief kan je ook van een bestaande Bitcoin portefeuille geld opladen.

Ikzelf heb geen Bitcoin portfeuille, dus zal het moeten overschrijven. Brave biedt je de kans via de dienst Coinbase. Deze ontvangt je betaling en zet dit om naar Bitcoin.

Opgelet: hoewel Bitcoin niet meer zo nieuw is, blijft het toch een eigenaardige munteenheid en je moet een beetje vertrouwen leggen in de diensten die jouw geld naar Bitcoin zullen omzetten.

Wat me opvalt is dat Brave momenteel sowieso USD (US Dollars) lijkt te verwachten. Dat wil zeggen dat we met twee koersen rekening moeten houden: Euro naar USD en USD naar Bitcoin.

Op het moment van schrijven kwam € 1 overeen met $ 1.12 en 1 Bitcoin kwam overeen met 609.9 USD.

Obstakel 1

Jammer genoeg kon ik in België via Coinbase géén Bitcoin aankopen.

Dit is eigenaardig want op de website van Coinbase kan je lezen dat dit in België wel mogelijk moet zijn.

Om hierrond te werken zal ik eerst een Bitcoin portefeuille moeten aanmaken en vanaf daar geld overschrijven naar mijn Brave portefeuille.

Stap 3.1 Bitcoin Portefeuille Aanmaken via Coinbase

Opgelet: Waarschijnlijk kan je rechtstreeks naar je Brave portefeuille storten en hoef je helemaal geen Coinbase portefeuille aan te maken.

Coinbase gaat voor je een Bitcoin portefeuille aanmaken en beheren. Een beetje zoals Paypal kan je dan geld opladen naar de portefeuille. Deze blijft wel op de site van Coinbase staan.

De eerste stap is je inschrijven. Coinbase vraagt je je GSM nummer te koppelen, maar dat sla ik maar over.

Hierna krijg je een overzicht van een lege portefeuille.

Via "Settings" kan je een "Payment Method" toevoegen op Coinbase:

Hiervoor moet je je identificeren, met onder andere een foto van je identiteitskaart. Daar heb ik weinig interesse in.

Stap 3.1.1 Bitcoins aankopen zonder Coinbase

Via Coinbase heb ik nu een "portefeuille". Ik wil via een andere service geld storten op mijn portefeuille van Coinbase.

Via Coinbase kan ik het "adres" van mijn portefeuille opvragen.

Het is ondertussen al duidelijk dat Brave's gebruik van Bitcoin voor mensen zonder ervaring hiermee nogal een grote drempel wordt. Ik kan me voorstellen dat het in sommige landen eenvoudiger is dan in andere.

Via een simpele google kom ik terecht bij Bitonic.nl. Hier kan je met Mister Cash Bitcoin aankopen. Dit zou ook in België moeten werken.

• Ik geef 10 euro in om aan te kopen
• Ik krijg een waarschuwing dat grote bedragen voordeliger zijn wegens transactie kosten. Ongewtijfeld. Ik vermoed dat hier betere alternatieven voor zijn.
• Hierna selecteer is Mister Cash als betaalwijze
• Ik geef mijn BitCoin adres in, zoals verkregen op Coinbase
• Daarna geef ik mijn IBAN rekeningnummer in
• Na afrekenen krijg ik een betaalformulier te zien, gehost door "bancontact.pay.be"
• Verder verloopt de betaling zoals een gewone Internet betaling met bancontact
• Ik krijg de melding dat de betaling gelukt is en dat mijn "bitcoins worden verstuurd binnen ca. 1 minuut"

Een minuutje later zie ik bij Coinbase een melding van de transactie, met status "Pending" (wachtend) en een bedrag van 0.01703405 Bitcoin, wat op dit moment overeenkomt met €9.23. De transactie heeft me dus €0.77 gekost.

De transactie wacht op het Bitcoin netwerk om de transactie te verifiëren. Zover ik Bitcoin begrijp komt het erop neer dat iedereen die eraan deelneemt dezelfde lijst van transacties deelt. Er wordt onderling (automatisch) overeengekomen welke transacties legitiem zijn.

Na een half uurtje heb ik 2 bevestigingen, dus het is een kwestie van tijd. Blijkbaar vereist Coinbase 3 confirmations vooraleer de overschrijving bevestigd is.

Na een uurtje heb ik deze confirmations. Blijkbaar zou dit sneller gaan voor grotere bedragen.

Stap 3.2 Bitcoin naar Brave storten

Nu er 0.01703405 Bitcoin in mijn portefeuille zitten moet ik die kunnen overschrijven naar mijn Brave portfeuille. Terug in Brave kies ik nu voor de optie om vanaf een bestaande Bitcoin portefeuille Bitcoin over te zetten (Send / Receive).

Met een Bitcoin app zou ik een QR code kunnen scannen, maar dat heb ik nog niet. Ik kan wel een overschrijving doen naar een Bitcoin adres:

Via Coinbase kan je bij "Send/Receive" een bitcoin adres ingeven. Je kan dan een bedrag ingeven en de overschrijving doen.

Opnieuw is het dan wachten tot de overschrijving compleet is.

Het lijkt dat je hiervoor je ID niet moet verifiëren door het uploaden van je identiteitskaart! Dus zeker niet doen.

Na twee bevestigingen zie ik in Brave mijn bedrag ($5.66) staan.

Stap 4 Surfen

Je kan je maandelijks budget instellen. Elke 30 dagen wordt dat budget verdeelt over de websites die je hebt bezocht.

Als er websites zijn waarvan je niet wilt dat ze deelnemen (je betaalt ze bijvoorbeeld al) kan je deze uitsluiten:

Eén opmerking na de payment, het is niet mogelijk om te zien hoeveel van je bijdrage er exact naar welke sites gaat.

Ik zou dit graag wat beter kunnen opvolgen.

Conclusie

Het concept van Brave is erg interessant. Het gebruik van Bitcoin hiervoor zorgt ongetwijfeld voor een hoge drempel, zeker voor minder technische gebruikers of gebruikers zonder ervaring met Bitcoin.

Hopelijk krijgt het idee tractie en kan je later ook op meer praktische manieren een potje geld uitdelen in ruil voor een reclame-vrij Internet. Als genoeg mensen dit systeem willen gebruiken gaan websites hopelijk minder geneigd zijn storende ads te verspreiden en toch nog eerlijk betaald worden.

De vraag is of genoeg mensen genoeg bewust (!) hinder ondervinden om dit aantrekkelijk te maken. Het zal ook wel eenvoudiger moeten worden dan Bitcoin om het draagvlak te vergroten.

Some time ago I started work on a small library to read FLAC file metadata. Nice to know publishing your niche personal projects can help people.

I was going to use it to manage my small collection of music.

At first, it was all about reading the track length. I wrote that on the train during my commute.

Eventually, I ended up supporting all metadata of the FLAC format.

I wrote it for .NET, which did not seem to have a managed library for that at the time.

I added support for writing metadata to FLAC files, even though I saw no use case for it myself. It was more an exercise in discipline.

For myself, I felt that I had to "finish" it completely. Meaning, support all metadata in the spec and publish the library. Not with the assumption anyone would need it, but just to finish one of my side-projects for once. I have many side-projects that equate to "half a page of scribbled lines".

So I published FlacLibSharp on NuGet and put the source on BitBucket and later on GitHub.

I never considered anyone would have a use case for this. Supporting only FLAC metadata for .NET seems really specific.

Nothing much did happen.

Downloads on NuGet seemed to come, for the most part, from bots downloading the packages.

Googling the library only gave me results related to NuGet and GitHub itself.

It did get 6 stars on GitHub after a while.

Then, somebody opened an issue on GitHub. Noticing it was lacking support for something encouraged in the VORBIS comment specification.

If it was just for me, I would not have touched the library anymore. But it turns out someone was actually using it.

So I was more than happy to address the issue.

Thoughts

So it turns out just putting your stuff online can help somebody. Even though you don't expect it.

If it wasn't for GitHub, I would've not known about it. Places like GitHub make it easy to open issues and even suggest changes in the code. It was possible before, but never this easy.

It's also kind of scary. You put something online, perhaps badly tested, and others start to use it.

What happens if they use it for something important, and it totally screws it up?

What happens if your project becomes popular?

Are you prepared? Are you responsible? Are you accountable? Do you owe the users anything?

Conclusion

I'll put my stuff out there. If it's untested or otherwise risky, I'll try to mention that. And I'll take the risk of becoming popular, or helping someone.

Also, if you find a project like this useful, consider informing the creators about it. It might make their day ;-)

In this post, I walk through the creation of a single Actor for Akka.NET. It will download files and report its progress to other Actors.

I'll go through all details but might skim over some of the basics of Akka.NET.

Actor Model

Here's a short introduction.

Each "actor" in the system has a clear responsibility. Actors communicate by sending and receiving immutable messages.

There can be many instances of a certain Actor. An Actor is cheap. They have an internal state, but do not share state with other Actors, or anything else.

This allows Akka.NET to run your code in parallell. At the same time the code you write in Actors runs sequentially, so you never have to worry about locks of some sort. Since the only state you change is internal and the code inside a single actor is synchronous, you will be fine.

The Download Actor

To put it in practice, we'll try to make an Actor that has one simple task: to download something from a URL.

It will:

• Accept multiple download requests (executed sequentially)
• Run a single download at a time
• Report start, end and progress in the form of a Message to the requestor

The basics

First, we create an Actor called FileDownloadActor

public class FileDownloadActor : ReceiveActor
{
}

Then we can define the messages. We'll define one incoming message RequestDownload and three outgoing messages DownloadStarted, DownloadCompleted and DownloadProgressed:

public class FileDownloadActor : ReceiveActor
{
    #region Messages

    public class RequestDownload
    {
        public Uri Uri { get; private set; }
        public RequestDownload(Uri uri) { Uri = uri; }
    }

    public class DownloadStarted
    {
        public Uri Uri { get; private set; }
        public DownloadStarted(Uri uri) { Uri = uri; }
    }

    public class DownloadCompleted
    {
        public Uri Uri { get; private set; }
        public string Path { get; private set; }
        public DownloadCompleted(Uri uri, string path)
        {
            Uri = uri;
            Path = path;
        }
    }

    public class DownloadProgressed
    {
        public Uri Uri { get; private set; }
        public double Progress { get; private set; }
        public DownloadProgressed(Uri uri, double progress)
        {
            Uri = uri;
            Progress = progress;
        }
    }

    #endregion
}

Note that we put the message classes inside the Actor class. This is not a requierment but I find it adds clarity.

So everything starts with a RequestDownload, providing only a Uri.

When the downloading starts the requestor will be informed of this. The DownloadProgressed messages also includes the progress.

When the download completes, the path of the downloaded file is also provided. This is actually shared state between two Actors (the DownloadActor and the Sender). It's assumed the two actors can both access that path. But sending a large byte array in a message is bad for performance.

Keeping State

The DownloadActor will have to keep some state:

• The Uri that is currently being downloaded
• The path of the file that is being written
• A reference to the requester of the current download, which is another Actor

There's some other stuff we'll need, but we can introduce that later. So we'll need a class member field for each of these items.

public class FileDownloadActor : ReceiveActor
{
    #region State

    private Uri _currentUri;
    private string _currentTargetPath;
    private IActorRef _currentDownloadRequestor;

    #endregion

    #region Messages
    ...
    #endregion
}

Initialization

Actors are always in a certain state. The state defines . In our case the DownloadActor can be ready to accept new download requests, or can be downloading.

Here's the states defined for the DownloadActor:

public class FileDownloadActor : ReceiveActor
{
    #region State

    ...

    #endregion

    #region Initialization

    public FileDownloadActor()
    {
        Become(Ready);
    }

    #endregion

    #region Messages

    ...

    #endregion

    #region States

    public void Ready()
    {
        Receive<RequestDownload>(message =>
        {
            HandleDownloadRequest(message);
            Become(Downloading);
        });
    }

    public void Downloading()
    {
        // TODO: Handle incoming Download Requests while downloading
        // TODO: Go back to Ready state when a download completes
    }

    #endregion
}

I will come back to the TODO's shortly, but first let's implement the HandleDownloadRequest method.

public class FileDownloadActor : ReceiveActor
{
    #region State

    ...

    #endregion

    #region Messages

    ...

    #endregion

    #region States

    ...

    #endregion

    #region Handlers

    private void HandleDownloadRequest(RequestDownload message)
    {
        _currentUri = message.Uri;
        _currentTargetPath = Path.GetTempFileName();
        _currentDownloadRequestor = Sender;

        _currentDownloadRequestor.Tell(new DownloadStarted(_currentUri));
    }

    #endregion
}

Note, in the message handler, we have access to the "Sender" property.

Doing the Download

A naive implementation could be the following. Careful though, in the next steps we'll be improving a lot of things.

public class FileDownloadActor : ReceiveActor
{
    #region State

    ...

    #endregion

    #region Messages

    ...

    #endregion

    #region States

    ...

    #endregion

    #region Handlers

    private void HandleDownloadRequest(RequestDownload message)
    {
        ...

        StartDownload();
    }

    #endregion

    #region Helper Functions

    private void StartDownload()
    {
        var client = new WebClient();

        // Careful, this is not the best way to do this!
        client.DownloadFile(_currentUri, _currentTargetPath);
        _currentDownloadRequestor.Tell(new DownloadCompleted(_currentUri, _currentTargetPath));
        Become(Ready);
    }

    #endregion
}

Testing

Currently, we have a basic Download Actor, with a lot of problems still, but we have the Actor, so we could start testing it.

Due to the parallell nature of Akka.NET, testing in Unit Test frameworks can be a bit tricky. It's not hard, but it's a little different and I won't go into it here. I'll give you a hint: use Akka.Test.

For now, we'll just test it in a simple Console application.

class Program
{
    static void Main(string[] args)
    {
        var system = ActorSystem.Create("downloader");

        var props = Props.Create(() => new FileDownloadActor());
        var actor = system.ActorOf(props);

        var requestDownload = new FileDownloadActor.RequestDownload(new Uri("http://getakka.net/images/akkalogo.png"));
        actor.Tell(requestDownload);

        Console.ReadLine();
    }
}

Issue with the current implementation

As said, there are quite some issues here:

• How do we know when the download is completed and where the file is. Who handles the started, progress and completed messages?
• The download action is a long-running synchronous operation. That's not a good thing in Akka. The action should be asynchronous.
• While the actor is downloading, it ignores other incoming requests

Creating a TestActor

To solve the first issue, we can create a simple test actor that writes something on the console for each received message. The test actor will have a reference to a DownloadActor:

public class TestActor : ReceiveActor
{
    private readonly IActorRef _downloadActor;
    private readonly Uri _testUri;

    public TestActor(IActorRef downloadActor, Uri testUri)
    {
        _downloadActor = downloadActor;
        _testUri = testUri;

        Become(Ready);
    }

    public class StartTest { };

    public void Ready()
    {
        Receive<StartTest>(message => HandleStartTest(message));
        Receive<FileDownloadActor.DownloadStarted>(message => HandleDownloadStarted(message));
        Receive<FileDownloadActor.DownloadProgressed>(message => HandleDownloadProgressed(message));
        Receive<FileDownloadActor.DownloadCompleted>(message => HandleDownloadCompleted(message));
    }

    private void HandleStartTest(StartTest message)
    {
        Console.WriteLine($"Starting test for Uri '{_testUri}'");
        _downloadActor.Tell(new FileDownloadActor.RequestDownload(_testUri));
    }

    private void HandleDownloadStarted(FileDownloadActor.DownloadStarted message)
    {
        Console.WriteLine($"Download started Uri '{_testUri}'");
    }

    private void HandleDownloadProgressed(FileDownloadActor.DownloadProgressed message)
    {
        Console.WriteLine($"Download progressed for Uri '{_testUri}': {message.Progress}");
    }

    private void HandleDownloadCompleted(FileDownloadActor.DownloadCompleted message)
    {
        Console.WriteLine($"Download completed for Uri '{_testUri}'.");
    }
}

Usage is like so:

class Program
{
    static void Main(string[] args)
    {
        var system = ActorSystem.Create("downloader");

        var props = Props.Create(() => new FileDownloadActor());
        var actor = system.ActorOf(props);
        var testProps = Props.Create(() => new TestActor(actor, new Uri("http://getakka.net/images/akkalogo.png")));
        var testActor = system.ActorOf(testProps);
        testActor.Tell(new TestActor.StartTest());

        Console.ReadLine();
    }
}

The output of this application should now be:

Starting test for Uri 'http://getakka.net/images/akkalogo.png'.
Download started Uri 'http://getakka.net/images/akkalogo.png'.
Download completed for Uri 'http://getakka.net/images/akkalogo.png'.

Handle Asynchronous Downloads

There are several ways to make the WebClient do asynchornous downloads:

• WebClient.DownloadFileAsync: uses callbacks
• WebClient.DownloadFileTaskAsync: use the Task Parallell Library

Here, we'll use the DownloadFileAsync. For an example of using Task inside Actors, check out this example: Akka.NET PipeTo Sample.

The problem with the PipeTo approach is that our async process gives feedback when it's not done yet too. Getting progress reporting to work then seemed quite hard.

With callbacks, it's easier. We just handle the two callbacks (progress and success).

A naive approach - that actually works - is the following:

    private void StartDownload()
    {
        var client = new WebClient();

        client.DownloadProgressChanged += HandleWebClientDownloadProgressChanged;
        client.DownloadFileCompleted += HandleWebClientDownloadCompleted;
        client.DownloadFileAsync(_currentUri, _currentTargetPath);
        Become(Ready);
    }

    private void HandleWebClientDownloadProgressChanged(object sender, DownloadProgressChangedEventArgs e)
    {
        _currentDownloadRequestor.Tell(new DownloadProgressed(_currentUri, e.ProgressPercentage));
    }

    private void HandleWebClientDownloadCompleted(object sender, AsyncCompletedEventArgs e)
    {
        _currentDownloadRequestor.Tell(new DownloadCompleted(_currentUri, _currentTargetPath));
    }

The event handlers run outside the context of the actor. Using the state could be an issue. The best implementation would be to make the handlers send a simple message to the actor itself and access the state in those message handlers.

The output should now be something like this:

Starting test for Uri 'http://getakka.net/images/akkalogo.png'.
Download started Uri 'http://getakka.net/images/akkalogo.png'.
Download progressed for Uri 'http://getakka.net/images/akkalogo.png': 100.
Download progressed for Uri 'http://getakka.net/images/akkalogo.png': 100.
Download completed for Uri 'http://getakka.net/images/akkalogo.png'.

It seems like WebClient fires the progress event twice. But that's not an issue.

Queueing Requests

In the current implementation requests that come in when the Actor is already downloading are ignored.

Luckily, Akka.NET has a way to "stash" these messages for later use. We have to do a few steps to make this work:

• Implement the IWithUnboundedStash in the Actor
• While downloading, push incoming RequestDownload messages on the stash
• When the download is completed make the stash send the queued messages

Implement the IWithUnboundedStash in the Actor

public class FileDownloadActor : ReceiveActor, IWithUnboundedStash
{
    ...
    public IStash Stash { get; set; }
    ...
}

While downloading, push incoming RequestDownload messages on the stash

public class FileDownloadActor : ReceiveActor, IWithUnboundedStash
{
    ...

    #region States

    ...

    public void Downloading()
    {
        Receive<RequestDownload>(message => Stash.Stash());
        // TODO: Go back to Ready state when a download completes
    }

    #endregion

    ...
}

When the download is completed make the stash send the queued messages

public class FileDownloadActor : ReceiveActor, IWithUnboundedStash
{
    ...

    #region States

    ...

    public void Downloading()
    {
        Receive<RequestDownload>(message => Stash.Stash());
        Receive<DownloadCompleted>(message => {
            Become(Ready);
            Stash.UnstashAll();
        });
    }

    #endregion

    ...
}

Now, the output is something like the following, which shows the final problem:

Starting test for Uri 'http://getakka.net/images/akkalogo.png'.
Download started Uri 'http://getakka.net/images/akkalogo.png'.
Download progressed for Uri 'http://getakka.net/images/akkalogo.png': 100.
Download progressed for Uri 'http://getakka.net/images/akkalogo.png': 100.
Download completed for Uri 'http://getakka.net/images/akkalogo.png'.

The Actor itself doesn't know yet when to come out of the "Downloading" state.

To do that, it has to send itself a DownloadCompleted message. A naive way - that doesn't work this time - could be the following:

public class FileDownloadActor : ReceiveActor, IWithUnboundedStash
{
    ...

    #region Helper Functions

    ...

    private void HandleWebClientDownloadCompleted(object sender, AsyncCompletedEventArgs e)
    {
        _currentDownloadRequestor.Tell(new DownloadCompleted(_currentUri, _currentTargetPath));
        Self.Tell(new DownloadCompleted(_currentUri, _currentTargetPath));
    }

    #endregion
}

Running this produces the following exception:

An unhandled exception of type 'System.NotSupportedException' occurred in Akka.dll

Additional information: There is no active ActorContext, this is most likely due to use of async operations from within this actor.

We cannot use the Self property, but we have to send ourselves a message somehow.

However we can create a field _self and assign it the Self property.

public class FileDownloadActor : ReceiveActor, IWithUnboundedStash
{
    #region State

    ...

    private IActorRef _self;

    ...

    #endregion

    #region Initialization

    public FileDownloadActor()
    {
        _self = Self;
        Become(Ready);
    }

    #endregion

    ...

    #region Helper Functions

    ...

    private void HandleWebClientDownloadCompleted(object sender, AsyncCompletedEventArgs e)
    {
        _currentDownloadRequestor.Tell(new DownloadCompleted(_currentUri, _currentTargetPath));
        _self.Tell(new DownloadCompleted(_currentUri, _currentTargetPath));
    }

    #endregion
}

This will make the Actor able to handle multiple requests, sequentially.

Taking Advantage of Actor's single threaded nature

Because we know the Actor will do one download at a time, we can re-use a single instance of the WebClient class.

public class FileDownloadActor : ReceiveActor, IWithUnboundedStash
{
    #region State

    ...

    private readonly WebClient _client = new WebClient();

    ...

    #endregion

    #region Initialization

    public FileDownloadActor()
    {
        _self = Self;
        _client.DownloadProgressChanged += HandleWebClientDownloadProgressChanged;
        _client.DownloadFileCompleted += HandleWebClientDownloadCompleted;
        Become(Ready);
    }

    #endregion

    ...

    #region Helper Functions

    private void StartDownload()
    {
        _client.DownloadFileAsync(_currentUri, _currentTargetPath);
        Become(Ready);
    }

    ...

    #endregion
}

Running in Parallell

You might be thinking that this is very much not parallell. After all, the Download Actor only downloads one URI at a time.

That is true, but we could create a "pool" of such download actors and use them simultaniously. In fact, Akka.NET makes that a one-liner, as shown here:

In our test application we have the following code to create our Download Actor:

    var props = Props.Create(() => new FileDownloadActor());
    var actor = system.ActorOf(props);

The actor is then passed to the TestActor who sends a number of download requests like so:

    private void HandleStartTest(StartTest message)
    {
        Console.WriteLine($"Starting test for Uri '{_testUri}'.");
        _downloadActor.Tell(new FileDownloadActor.RequestDownload(_testUri));
        _downloadActor.Tell(new FileDownloadActor.RequestDownload(_testUri));
        _downloadActor.Tell(new FileDownloadActor.RequestDownload(_testUri));
    }

Running this will show the three downloads happen in sequence.

To allow multiple download requests to be handled simutaniously, we can simply add a WithRouter statement to the Actor creation:

        var props = Props.Create(() => new FileDownloadActor()).WithRouter(new RoundRobinPool(10));
        var actor = system.ActorOf(props);

This is also why IActorRef is passed around and not just and instance of actor: IActorRef can refer to one instance or even a pool of instances of an Actor, optionally spread over multiple machines when that makes sense. This is where using Actors starts to pay off. It handles some of the complexity of running things in parallell.

Another approach could be to create a dispatching Actor which will create one Download Actor per hostname, for example.

Open Issues

There are some things that can improve this Actor:

• If, for some reason, the same Uri is requested twice or more by the same Actor, it will be unclear to them which of the requests was completed. This is because they only get the Uri. Better would be if the requested would provide a unique ID to each request. For example a Guid.
• When a download goes wrong, the Actor should inform the requester. Either by a message or by simply crashing and using Akka.NET's supervision capabilities.
• I'm not a 100% sure of keeping a reference to Self. Since Self is an IActorRef, and IActorRef doesn't guarantee to point that instance of the Actor class I'm not sure it's fool proof.

Conclusion

With this post, I wanted to explain to you, and to myself, how to create a Download Actor that handles asynchronous calls inside itself. Also, I hope it introduces Akka.NET in a practical way.

I find it an enjoyable and different way to think about applications. And it takes away a lot of pain from parallell programming in .NET ("threading").

Even if you don't create massively distributed applications, Akka.NET could be used in small scale applications, just to abstract away the technical details of running things in parallell. Which is exactly what I'll be trying with our download actor.

I'm wondering how to combine an Akka.NET actor system with a WPF front-end.

The Actor Model provides a nice way to build concurrent systems. There are several implementations for .NET. Orleans and Akka.Net are the best known.

As an excercise, let's build a "thermostat" system.

I'll leave out some implementation details so we can focus on the integration.

The Thermostat System

The system will display both the current and the desired temperature to the user.

It'll have two buttons to increase and decrease the desired temperature.

In the background, the system should control the heating but we'll ignore that bit here.

It'll look like this:

UI

We'll create a basic WPF application with MVVM. The ViewModel will have a property for the current temperature and the target temperature.

It also accepts two commands: IncreaseTargetTemperature and DecreaseTargetTemperature.

Back-end

The back-end should have a thermometer that takes measurements on a regular basis.

It should also keep track of the target temperature, set by the user.

The target temperature can change by two messages sent to the system:

• Increase the target temperature
• Reduce the target temperature

As output, the system should emit:

• The current temperature, when a measurement takes place
• The target temperature, when it's changed

Thermostat Actor System

Actors

This simple actor system can consist of two Actors:

• TemperatureSensorActor
• ThermostatActor

TemperatureSensor

The TemperatureSensor sends a message TemperatureMeasured to its Parent actor. This happens as a response to a TakeMeasurement message.

The TemperatureSensor sends the TakeMeasurement to itself on a fixed interval.

Thermostat

The Thermostat is in charge of creating a TemperatureSensor.

It's a "pub-sub" actor that should send to its subscribers the following messages:

• TargetTemperatureSet
• TemperatureMeasured

Bridge

With the UI and the actor system ready, we can start to make them talk with each other.

The code we start with is on github. If you want to try this yourself first, you could fork it. If not, just read on to see how I do it.

To support this, WPF should react to messages sent by the Actors. At the same time, WPF should be able to send messages to the Actor System, in some way.

To support this communication we could create a "Bridge". The Bridge will contain:

• A Bridge Actor
• One or more public functions

The Bridge Actor will have a reference to the ViewModel. The ViewModel will have a reference to the Bridge (not the Bridge Actor!).

The Bridge can expose methods the ViewModel can call which pass messages to the Actor System. For example, a method IncreaseTargetTemperature.

The Bridge Actor calls functions exposed by the ViewModel. These will update the ViewModel's properties. The actor does this only while handling messages.

Bridge Interfaces

To avoid exposing the entire ViewModel to the Bridge Actor, an interface seems like a good idea. And an interface to abstract away the Bridge passed to the ViewModel would also help.

In short, we'll have two Interfaces. One implemented by the ViewModel, the other by the Bridge itself.

We can define these two interfaces:

• IThermostatView
  • UpdateCurrentTemperature
  • UpdateTargetTemperature
• IThermostatBridge
• IncreaseTargetTemperature
• DecreaseTargetTemperature

The Bridge Actor references IThermostatView while the ViewModel implements it.

The ViewModel references IThermostatBridge while the Bridge implements it.

Implementation

Views

The views, as discussed above, are simple:

// Implemented by the Bridge
public interface IThermostatBridge
{
    void IncreaseTargetTemperature();
    void DecreaseTargetTemperature();
}

// Implemented by the ViewModel (in the WPF project)
public interface IThermostatView
{
    void UpdateCurrentTemperature(double currentTemperature);
    void UpdateTargetTemperature(double targetTemperature);
}

Bridge Actor

The Bridge Actor is small, just passing messages around. It does have two dependencies.

public class BridgeActor : ReceiveActor
{
    private IThermostatView _thermostatView;
    private IActorRef _thermostatActor;

    public BridgeActor(IThermostatView thermostatView, IActorRef thermostatActor)
    {
        _thermostatView = thermostatView;
        _thermostatActor = thermostatActor;
        Become(Active);
    }

    public void Active()
    {
        Receive<TemperatureMeasured>(message => _thermostatView.UpdateCurrentTemperature(message.Temperature));
        Receive<TargetTemperatureSet>(message => _thermostatView.UpdateTargetTemperature(message.TargetTemperature));
        Receive<IncreaseTargetTemperature>(message => _thermostatActor.Tell(message));
        Receive<DecreaseTargetTemperature>(message => _thermostatActor.Tell(message));
    }
}

Bridge

The bridge will implement the IThermostatBridge interface:

public class ThermostatBridge : IThermostatBridge
{
    private IActorRef _bridgeActor;

    private readonly IncreaseTargetTemperature increaseMessage = new IncreaseTargetTemperature(1);
    private readonly DecreaseTargetTemperature decreaseMessage = new DecreaseTargetTemperature(1);

    public ThermostatBridge(IActorRef bridgeActor)
    {
        _bridgeActor = bridgeActor;
    }

    public void IncreaseTargetTemperature()
    {
        _bridgeActor.Tell(increaseMessage);
    }

    public void DecreaseTargetTemperature()
    {
        _bridgeActor.Tell(decreaseMessage);
    }
}

System Creation

I already had a seperate class to create the Actor System. For the excercise, I'll create the system when the WPF app starts. But this might not be the ideal place.

public partial class App : Application
{
    private static WpfAkkaIntegration.ThermostatSystem.ThermostatSystem _system = new WpfAkkaIntegration.ThermostatSystem.ThermostatSystem();

    public static WpfAkkaIntegration.ThermostatSystem.ThermostatSystem ThermostatSystem => _system;
}

In that class, there's a function CreateThermostatBridge which can create our Bridge. Here's the full ThermostatSystem class:

public class ThermostatSystem
{
    private ActorSystem _system;
    private IActorRef _thermostatActor;

    public ThermostatSystem()
    {
        _system = ActorSystem.Create(nameof(ThermostatSystem));
        _thermostatActor = CreateThermostatActor();
    }

    private IActorRef CreateThermostatActor()
    {
        var props = Props.Create<Actors.ThermostatActor>();
        return _system.ActorOf(props, "thermostat");
    }

    public IThermostatBridge CreateThermostatBridge(IThermostatView thermostatView)
    {
        var bridgeActor = CreateBridgeActor(thermostatView);
        _thermostatActor.Tell(new Subscribe(bridgeActor));

        return new ThermostatBridge(bridgeActor);
    }

    private IActorRef CreateBridgeActor(IThermostatView thermostatView)
    {
        var props = Props.Create(() => new BridgeActor(thermostatView, _thermostatActor))
            .WithDispatcher("akka.actor.synchronized-dispatcher");
        return _system.ActorOf(props, "bridge");
    }
}

Synchronized Dispatcher

The code above creates the BridgeActor using the synchronized dispatcher.

This makes that actor run on the UI-thread, so calling methods on the ViewModel should be safe.

From the akka.net docs:

You may use this dispatcher to create actors that update UIs in a reactive manner. An application that displays real-time updates of stock prices may have a dedicated actor to update the UI controls directly for example.

Note: As a general rule, actors running in this dispatcher shouldn't do much work. Avoid doing any extra work that may be done by actors running in other pools.

Update April 27, 2016: The synchronized dispatcher is not required. All updates of the UI go through property changes on the ViewModel. The data-binding mechanism will take care of executing this on the UI thread. Read more in this blog post "Akka.NET Dispatchers and User Interface Thread Access" by Jason Roberts.

Bridge Creation

Finally we have everything to hook it all up in the ViewModel.

public MainViewModel()
{
    _bridge = App.ThermostatSystem.CreateThermostatBridge(this);

    IncreaseTargetTemperature = new RelayCommand(() => _bridge.IncreaseTargetTemperature());
    DecreaseTargetTemperature = new RelayCommand(() => _bridge.DecreaseTargetTemperature());
}

You can also see the implementation of the two commands the view can receive.

I do this in the constructor of the ViewModel for this excercise. In real life, I suppose dependency injection would be adviceable. But then the ViewModel would still need to assign itself to the Bridge afterwards.

Result

Running the app gives me this WPF app that updates its view based on messages that occur in the Actor System.

Additionally, the view triggers changes to the system by sending it messages. It does this when it receives a command.

When you click a button in WPF, all it does is ask the bridge to increase the target temperature. The Actor System will then process that message. When it's done, the Actor System forwards a new message back to WPF so it can update the view.

This is all asynchronous, because of the actor system.

In the example, the current temperature only goes up. This is because I did not simulate an actual room that heats. I decided to just let the temperature increase at every measurement taken. Just to show the temperature is changing.

Code

The example project is on github:

• Finished
• Before creating the bridge

Conclusion

It's possible to loosely couple a WPF app with an Akka.NET Actor System. I used something I called a Bridge, but there must be a better name.

Something I don't like here is that the Bridge and Bridge Actor should be more generic. It mentions "Thermostat" everywhere.

Do you think this is a reasonable way to put a WPF front-end on an Akka.NET Actor System? Let me know in the comments, or on twitter or something.

I maintain a small tool to track my own activities through the day. For this, I wanted a feature to automatically log when I was "idle". So I'd know how long "that meeting" actually took.

Here's how I did that.

The Code

[DllImport("user32.dll")]
static extern bool GetLastInputInfo(ref LASTINPUTINFO plii);

[StructLayout(LayoutKind.Sequential)]
struct LASTINPUTINFO
{
   public static readonly int SizeOf = Marshal.SizeOf(typeof(LASTINPUTINFO));

   [MarshalAs(UnmanagedType.U4)]
   public UInt32 cbSize;
   [MarshalAs(UnmanagedType.U4)]
   public UInt32 dwTime;
}

private TimeSpan RetrieveIdleTime()
{
    LASTINPUTINFO lastInputInfo = new LASTINPUTINFO();
    lastInputInfo.cbSize = (uint)LASTINPUTINFO.SizeOf;
    GetLastInputInfo(ref lastInputInfo);
        
    int elapsedTicks = Environment.TickCount - (int)lastInputInfo.dwTime;

    if (elapsedTicks > 0) { return new TimeSpan(0, 0, 0, 0, elapsedTicks); }
    else { return new TimeSpan(0); }
}

Be careful with "long" uptimes larger than 24 days (see further).

Details

To detect if a user has been idle, you'll have to make a call to the native win 32 function called GetLastInputInfo.

It will give you the tick count at the last user input. Careful, that's not "the amount of ticks the user has been idle". You'll have to compare against the current tickcount to get to that value.

First thing to do is import the function:

[DllImport("user32.dll")]
static extern bool GetLastInputInfo(ref LASTINPUTINFO plii);

The LASTINPUTINFO struct will have to be defined as well:

[StructLayout(LayoutKind.Sequential)]
struct LASTINPUTINFO
{
    public static readonly int SizeOf = Marshal.SizeOf(typeof(LASTINPUTINFO));

    [MarshalAs(UnmanagedType.U4)]
    public UInt32 cbSize;
    [MarshalAs(UnmanagedType.U4)]
    public UInt32 dwTime;
}

And then you'll need some boilerplate code to actually fill up that structure:

LASTINPUTINFO lastInputInfo = new LASTINPUTINFO();
lastInputInfo.cbSize = (uint)LASTINPUTINFO.SizeOf;
GetLastInputInfo(ref lastInputInfo);

int elapsedTicks = Environment.TickCount - (int)lastInputInfo.dwTime;
TimeSpan idleTime = new TimeSpan(0, 0, 0, 0, elapsedTicks);

Don't forget to manually set the cbSize value.

The definition of LASTINPUTINFO comes straight from "pinvoke.net", a great website. However, it's always good to know what you're doing.

The [StructLayout(LayoutKind.Sequential)] attribute says that the information must be laid out in memory just as it appears. This goes for the object both in managed and unmanaged memory. This is because the win32 native functions depend on the layout of the struct in memory, being native functions (please correct me if I'm wrong here).

Marshal.SizeOf returns the size of the unmanaged type in memory. We can then use that value to set the cbSize property.

Also, both cbSize and dwTime are strictly defined as unsigned, 32-bit integers. That's also what the MarshalAs(UnmanagedType.U4) attribute indicates.

Uptime Issue

One thing to think about: the "tickcount" in windows is the amount of milliseconds since the system started. We're dealing with an unsigned 32-bit integer, which means it can represent up to 50 days before it wraps around.

Normally you work around this by calling GetTickCount64, but there doesn't seem to be anything for GetLastInputInfo. In any case, if you're converting to a TimeSpan (which takes Ticks as a signed 32 bit integer), you'll have trouble after about 24 days.

That is to say, the above will fail if the system is up for a long time.

Does anyone know a way around this?

When using Windows, I always end up with a messy desktop. Temporary files, useless shortcuts, small notes, an "OldIcons" folder ...

What I would like is a way for my desktop to reset itself every day. The icons that really have to be there should be there, but all the other junk can be removed.

It's relatively easy to do in Windows, without third party software.

Solution

The goal is to use the "Public Desktop" as the template of what the desktop should look like. Everything on the personal desktop is removed at log on.

In Windows, your desktop shows the icons from the Public Desktop C:\Users\Public\Desktop, and from your user C:\Users\user\Desktop.

So all you have to do is clear C:\Users\user\Desktop at startup.

Step 1: The script

Make sure the directory C:\Users\Default\Desktop exists and is empty.

Create a file clearDesktop.cmd:

@echo off
robocopy "C:\Users\Default\Desktop" "C:\Users\user\Desktop" /Z /E /MIR

For some reason, clearing all files and folders from a folder is complicated. As seen in this stackoverflow question.

It's easy using robocopy, but you need an empty folder first. You then mirror that empty folder to the folder for which you want to remove the content.

Additionally, you could add some files to the "Default" desktop folder. That way they are also added to the user desktop.

Step 2: Create The Scheduled Task

You can make the script run at system startup.

In windows, go to "Start" and type "Task Scheduler". Start the Task Scheduler.

Add a new task, with:

• Trigger: At system startup
• Action: "Start a program"
• And pass the full path of the clearDesktop.cmd script.

Test

First run the script, and make sure you're pointing to the correct folders! See your files dissapear from the desktop.

Now put a new file on your desktop and reboot. The file should be gone now.

Conclusion

I always make weird scripts to do the thing I want. I always forget how I did it, so I wrote it down here.

That's my conclusion.

Another conclusion could be: you don't need to download third party tools for everything.

^{Latest update: 28 september 2019: I listen to a lot less podcasts and a lot more music now-a-days.}

I have a long commute and very bad radio shows. So in the car, I like to listen to podcasts.

Here are some of my favorites.

TL;DR

• Code Newbie RSS Feed
• Base.cs RSS Feed
• Risky Business RSS Feed
• .NET Rocks RSS Feed
• Crypto Gram Newsletter RSS Feed
• Hanselminutes RSS Feed
• Security Weekly RSS Feed
• Troy Hunt's Weekly Update RSS Feed
• Herding Code RSS Feed
• Software Engineering Radio RSS Feed
• Software Engineering Daily RSS Feed
• Unsupervised Learning RSS Feed
• The Changelog RSS Feed
• Web Dev Radio RSS Feed
• Starktalk Radio RSS Feed
• Nature RSS Feed
• Physics Central Podcast RSS Feed
• Scientific American Science Talk RSS Feed.
• 60 Seconds Science RSS Feed
• 60 Seconds Space RSS Feed
• 60 Seconds Tech RSS Feed
• 60 Seconds Earth RSS Feed
• 60 Seconds Health RSS Feed
• 60 Seconds Mind RSS Feed
• This Developer's Life RSS Feed
• Radiolab RSS Feed
• Intelligence Matters RSS Feed
• Pilot to Pilot RSS Feed

Update: here's someone else's list too. Via Daniel Kaplan.

Programming and IT

Code Newbie

RSS Feed

Target people new to the world of software development, in an approachable, inclusive and entertaining manner. But it's useful for everybody, because in this field you're always a beginner at something. And if there's a topic you already know very well, it helps if you hear people like (Saron)[https://twitter.com/saronyitbarek] explain it in such clear terms.

Base.cs

RSS Feed

Base.CS is a podcast mash-up of Vaidehi Joshi's BaseCS blog series and the Code Newbie podcast, with Saron Yitbarek.

.NET Rocks

RSS Feed

Carl Frankling and Richard Campbell interview a diverse set of guests about software development. Don't let the name of the podcast fool you. The topics go way beyond .NET.

Every month or so they have a Geek Out, where they go deep into a topic that's not directly related to software.

Crypto Gram Newsletter

RSS Feed

This is a spoken version of the Crypto Gram Newsletter by Bruce Schneier.

It handles topics like information security, privacy online, Cryptography, etc ... It's mostly Schneier's blog.

Hanselminutes

RSS Feed

Scott Hanselman interviews various people about a wide range of "geeky" topics.

Security Weekly

RSS Feed

Weekly interviews and discussions about Information Security. It's made by Information Security professionals, and it also targets that same audience. But there are interesting things talked about for Software Developers too.

Update: Security Weekly now has several spin-offs each with a different focus (start-up security weekly, enterprise security weekly, ...). I personally usually listen to the "Hack Naked News" (what's in a name?) because the others are a bit too long, even for my commute! The news sticks usually to about half an hour.

Risky Business

RSS Feed

"News and current afairs" in information security. Entertaining and knowledgable hosts, covering the events in "infosec". May do with a little bit less cynisism and snark, but still very informative and not too long.

Troy Hunt's Weekly Update

RSS Feed

Troy Hunt talks about his week. Which is somehow always a very interesting week. Troy Hunt specializes in teaching the world, companies and software developers in particular about information security. But sometimes he focusses on other interesting topics. Depending on what he's been up to.

Also, check out his blog for more of this stuff.

Herding Code

RSS Feed

K. Scott Allen, Kevin Dente, Scott Koon and John Galloway talk to guests about all the hip stuff in Software Development. Everything quite informal, entertaining and interesting.

Unsupervised Learning

RSS Feed

Daniel Miesslier curates about 3 to 5 hours of reading into a podcast of about. Keeps you informed and up to date.

Software Engineering Radio

RSS Feed

This podcast from IEEE magazine is pretty hardcore. It is coming from a computer science perspective. It's interesting if you're up for it. Maybe not for a Monday morning.

Software Engineering Daily

RSS Feed

Amazingly, this is a daily podcast. Covering a wide range of topics. It's almost impossible to listen to all of them. I have no idea how they're able to pull this off daily. Nonetheless, if you pick the topics you like you'll get some good info.

As a side note: What I really like is that a summary of the topic is the first thing you hear. So if you want to pass on a topic, you can do so right away. Many podcasts first have a few minutes of intro before you actually know the subject. This is sometimes not a problem, but when driving not all cars (I've seen none) display episode titles clearly.

The Changelog

RSS Feed

Focusses on giving airtime to the many smaller open source projects that are happening.

Update June 18, 2016: I feel I should say a few more things about The Changelog. They seem to have changed their angle a bit in the last few months. Focussing less on the latest hot thing and even more on the people behind open source project. Check out the shows that feature Richard Hipp (creator of SQLite), Matz (creator of Ruby) and especially the interview with Pieter Hintjens.

Easily one of the favourites.

Web Dev Radio

RSS Feed

A web developer talks about developments in the web development world. Occasionally there is an interview as well.

Science

Starktalk Radio

RSS Feed

Neil deGrasse Tyson, interviews people about science. They also answer listener's questions. He is always joined by comedians, for comic relieve.

I believe it's now also a TV show on Discovery Channel.

Nature

RSS Feed

Science news and interviews from nature.com.

Physics Central Podcast

RSS Feed

Physics news for the general public with an interest in that stuff.

Scientific American Science Talk

RSS Feed.

Interviews and news about science, from Scientific American.

Scientific American 60 seconds

60 seconds (actually, usualy 2 minutes) of rapid fire science news. Easy to digest.

They have several 60 seconds series each focussing on another field. I'm listening to Science, Space and Tech. Here's the RSS Feed for all of them:

• 60 Seconds Science
• 60 Seconds Space
• 60 Seconds Tech
• 60 Seconds Earth
• 60 Seconds Health
• 60 Seconds Mind

Aviation

Pilot to Pilot

RSS Feed

I've been training to fly lightweight aircraft, and to encourage some thinking about aviation I looked up some aviation podcasts.

I'm currently only on this one. But maybe I can find some more in the future.

Other

This Developer's Life

RSS Feed

Scott Hansleman (from the Hanselminutes podcast) explores some topics of life in general more in depth.

Noteable episode: 2.0.2 Pressure. Scott talks about pressure with the people that keep Stack Overflow running.

Radiolab

RSS Feed

Well produced documentaries about science and life in general. This is easily my favorite podcast of them all.

Notable Episode: The Rhino Hunter. This episode shows the other, less popular, side of the Cecil the Lion story. To me, it shows nicely how there is always more than one truth.

Intelligence Matters

RSS Feed

Former CIA Director, Michael Morell, interviews people from the U.S. intelligence community. Some interesting stories and knowledgeable views on the things the U.S. is up to in terms of intelligence and millitary.

I'm no american citizen, but it's interesting nonetheless. I'd like to have something like this closer to home. But this is very interesting and on topics that - let's be honest - concern people all over the world.

As you can see, the blog changed looks, because I moved from Octopress to Ghost.

Recently, I got in some serious trouble with my Octopress blog, to the point where I couldn't publish any posts and I didn't see an easy fix.

I had switched from Ubuntu to Manjaro. Since then, I tried to preview and publish my octopress blog using Docker, so that I did not have to install specific stuff on my actual machine to generate my blog.

This turned out to give me a terrible time with Octopress:

• My version of Octopress was old and not compatible with newer Ruby and Rubygem stuff
• Dependencies broke
• I had to update Octopress, which seemed hard
• I was scared to break everything (again) if I pushed to OpenShift

This was so terrible that I needed a big change, if I still wanted to have a blog.

Less posts about the blog

I have the feeling that the quality of your blogging platform is inversely proportional to how much you blog about keeping your blog up and running.

So my goal was to not worry as much about the technical underpinnings of my blog and spend that time on trying out other things.

Writing a draft blog post should be quick and painless.

Of course, when I really want to, I still want some control to fiddle around with the technical underpinnings. Preferably by my own choice and not because the blog is broken.

Options

So I started building a list for myself of what I thought was important for the blogging platform I'd use:

• Is it easy to update the software?
• Is it easy to write posts?
• Is it SaaS or do I host it myself?
• Does it cost anything?
• What dependencies does it have?
• Does it have a nice templating option?
• How much control do I have?

And so on.

After some googling, I realised there were MANY platforms. Way too many for me to evaluate them all.

I settled on another approach: I'd check some of the more famous blogs I follow and see what they are using.

I ended up with four options:

• Jekyll
• Octopress (based on Jekyll)
• Wordpress
• Ghost

Octopress and Jekyll seemed popular with some types of people. But given my recent experiences, I wanted to move away from that. Note that they are working hard on Octopress 3, and I'm sure it'll improve on version 2 a lot.

Wordpress is very popular, and an obvious choice. However, Wordpress and especially the plugins have quite a bad reputation when it comes to security. Also, taking the most obvious choice is not that exciting. I might regret not just going for Wordpress later though, when I break the blog again.

And so there was Ghost. Used by some serious bloggers. And easy to spin up on OpenShift, which I was already using (for free) to run my Octopress blog.

Getting it running with a Postgresql back-end took a few minutes and almost literally "one click".

Moving from Octopress

I had a shortlist of must-haves before I would seriously consider switching to Ghost.

It had to be possible to:

• update the Ghost software
• import my existing blog-posts
• force https

All of these turned out to be relatively easy (except the update process, but possible nonetheless).

Ghost positives

Ghost has some properties I really like:

• You can create and publish your posts in the web application itself.
• In Octopress, this required a git push to OpenShift and a prayer it wouldn't break.
• You get to see your Markdown in preview
• The default theme is nice and there are many themes available
• Easy to export the posts and configuration
• In general, less technical fiddling, more writing.

Ghost negatives

There are some downsides too:

I now have a dynamic site instead of the static pages of Octopress. You can log in and it has a database back-end. So I must watch out for security issues and update frequently.

I'm still self-hosting, so it's still a bit fiddly, but that is my own choice. I could've gone for the hosted solution too, which would eliminate the need for the tricky update process.

Conclusion

Octopress which, in all fairness, did have the slogan "A blogging framework for hackers" brought me to a tough spot. Hacking is much fun, but focussing on something else than keeping the blog alive also seemed quite promising. I felt the barrier for writing a new post was just higher than it should be, and it was too scary.

I'm looking forward to working with Ghost and hope to have a less bumpy ride than I had with Octopress.

But, lets face it, at some point I'll probably switch again just to try something new.

After a month or two of abscence from the blog I wrote up something small this week.

Quite unexpected, when I pushed my post to openshift for publishing, my blog broke.

The styles were stripped, but the blog was still usable, which is nice actually. It only looked like a website from early 1995.

Anyway, the error I got from "rake generate" (which "compiles" the blog to static HTML using Jekyll) was:

Error = jekyll 2.2.0 | Error: "\xC3" on US-ASCII

The funny thing was, it worked fine on my local machine, but not on the openshift gear. It did work fine the previous blog posts two months earlier.

First, cut the crap: I fixed it by setting the LC_ALL environment variable to en_US.UTF-8.

export LC_ALL=en_US.UTF-8

The reason for this is unknown ...

What was the problem?

A simple google on the error always ended me on some chef issues. At first I thought it was unrelated but since both were using Ruby it was plausible.

The solution they came up with was to set a value for LC_ALL explicetly.

This is an environment variable on linux, part of the "locale" settings. It seems to be empty.

This was strange, because the variable was also empty on my local machine, but I tried it anyway:

export LC_ALL=en_US.UTF-8

Just before I run rake generate. And sure enough, that solved the problem.

So I "integrated" that in my openshift action hook, where I also run rake generate (is for another day).

What I first thought was the problem

I made a new blog post, but typed it first on windows in notepad. I thought, when copying it into my blog it took some "illegal characters" (whatever that means) with it.

But removing the new blog post solved nothing, so I got stranded on that one.

What is the root cause?

I have no idea.

When I look up the error in twitter, I get a small amount of tweets, spanning several years of people not understanding what the error is about.

Perhaps related, when I was apt-get upgrade'ing my other linux machine, it complained a lot about its inability to update LC_ALL.

So I don't know what the hell was going on, but I worked around the problem for now ...

Many questions remain:

• Why was the problem not present on my local machine?
• Where was the \xC3 character?
• Why is it solved by just changing LC_ALL?
• Why did the same stuff work two months ago?
• Is it Ruby?
• And why did I bring helium instead of air?

Conclusion

I hate this error because I don't know what or how or why it happened, but since I worked around it I have no time or energy to investigate it deeper. Not even Google and Twitter are helping me out on this one.

I'm writing it here because it's one of those stupid things you know you'll run into again in a year and you'll tell yourself: "I've seen this before, but I don't remember how I solved it. If only I'd written it down somewhere".

If anyone knows what's going on, please drop me a line.

... Now let's see what happens when I push this post out.

As a reminder to myself mostly, here's a run-down of a backup "script" I made to backup a small amount of data (< 1GB) to "the cloud".

I have a local folder where I store private documents and some other things. I already back this up locally, but for these things I wanted to put an additional copy "in the cloud", with something like dropbox.

Not that these files are super secret or that I'm an important target, but still this isn't stuff I just want out in the open online, so before I upload them to whatever cloud storage I want, the script encrypts them locally.

Update december 2015: SpiderOak's client is now called SpiderOakONE. Also the executable and location was renamed. But it seems to still work the same. More importantly spideroak no longer offers 2 GB for free. They now only offer a 60 day trial or 30 GB for $7 / month.

General Idea

To create a batch script that will:

• Mount a truecrypt volume
• Copy files to the mounted truecrypt volume
• Unmount the volume
• Trigger cloud storage sync of the truecrypt container file

Completed script

"C:\Program Files\TrueCrypt\TrueCrypt.exe" /volume "C:\Users\user\Documents\SpiderOak Hive\docs.tc" /lz /quit /m ts /p "THE_KEY_FOR_THE_CONTAINER" /b
echo "Wait for beep ..."
pause
robocopy "C:\Users\user\Documents" "Z:" /E /Z
"C:\Program Files\TrueCrypt\TrueCrypt.exe" /q /dz
"C:\Program Files\SpiderOakONE\SpiderOakONE.exe" --batchmode -v

Prerequisites

• Truecrypt
• Spideroak
• Windows 7 (robocopy)

Setup TrueCrypt

Technically, TrueCrypt stopped existing somehow when the developers decided to stop everything. But you can still use it for general encryption purposes.

You currently can get an installer here: TrueCrypt from GRC.COM (yes, grc.com, let's face it, it's probably clean).

Setup SpiderOak

I'm using SpiderOak as the no-cost cloud storage for my backup. Not because of the security features it offers but because you can trigger the sync from the command line (I could not find a way to do this with dropbox).

Just go to spideroak.com and setup Spideroak. Turn off "start on windows logon", unless you want SpiderOak running all the time. I don't because I don't really use it for that purpose.

Create TrueCrypt Container

Create a new empty TrueCrypt container in your SpiderOak "hive" (the folder from which files are synced to SpiderOak).

I created a 1 GB container, which is enough to store the things I want to backup. SpiderOak gives you 2 GB at no-cost.

Part 1: Mount the container from command line

"C:\Program Files\TrueCrypt\TrueCrypt.exe" /volume "C:\Users\user\Documents\SpiderOak Hive\docs.tc" /lz /quit /m ts /p "THE_KEY_FOR_THE_CONTAINER" /b
echo "Wait for beep ..."
pause

Break-down of the truecrypt command:

• /volume: just points to the truecrypt volume we'll be mounting
• /lz: mount the volume on the "Z:" drive letter
• /quit: perform requested actions and exit
• /m ts: mount option to indicate we should NOT preserve timestamp. If we preserve the timestamp SpiderOak will not see the file as changed and it won't sync correctly
• /p: the password for the truecrypt volume. I choose to put it in the script. This makes sense: if you can read this script, you likely can also read the files I'm backing up, since they're stored unencrypted on the same system.
  • If you don't want this, just leave the /p parameter out and TrueCrypt will ask you for a password or keyfile.
• /b: Beeps when the volume has been succesfully mounted

The last option is also why I ask to "wait for the beep" after the truecrypt command. If we continue too fast we might start doing things before the volume is mounted.

Part 2: Copy stuff

This is easy with robocopy:

robocopy "C:\Users\user\Documents" "Z:" /E /Z

This will just copy everything to the Z: drive from the given folder. Adding /MIR would create an exact copy. In the above case, files deleted in the source will remain on the destination.

Part 3: Unmount TrueCrypt volume

"C:\Program Files\TrueCrypt\TrueCrypt.exe" /q /dz

This tells Truecrypt to unmount the Z drive.

Part 4: Trigger SpiderOak sync

"C:\Program Files\SpiderOakONE\SpiderOakONE.exe" --batchmode -v

Use --batchmode to ensure SpiderOak quits after the sync is complete, -v is "verbose", which makes you get some feedback during the process.

Note that for a 1 GB file this will take a while the first time, but subsequent times will be faster because SpiderOak does a differential upload.

In my previous post I talked about how I changed some of the default behaviour of Go's http.FileServer to fit my limited needs.

Basically, I had built a number of http.Handler implementations that would only hand over some of the incoming requests to http.FileServer, and ignore others.

Obviously, this approach has some issues. Since I'm just blacklisting some requests there's a chance certain requests can be accepted that I actually did not want to accept. Also, in the whole thing I completely ignore any caching support.

But that is not the point here.

The point is that at first I had created all these handlers in one big Go file. And I was wondering how I could split this functionality up into a "re-usable" Go package. Here's what I tried ...

Go's Workspace

On the golang website, there's a section called How to Write Go Code. The first thing that's introduced is the "Workspace". Where you get dictated a very specific setup of your working directories.

Three folders are suggested:

• src
• pkg
• bin

All source code in src, package objects in pkg and executables come in bin.

The name of the folder where you application or package's source code is under dictates how you can include it in other projects.

One interesting thing in the guide is this:

"If you keep your code in a source repository somewhere, then you should use the root of that source repository as your base path. For instance, if you have a GitHub account at github.com/user, that should be your base path."

So, since I have a github.com account, I created a folder in my workspace:

• github.com/AaronLenoir

I then created a new empty repository "fileserver" on github.com. And cloned it into

• src/github.com/AaronLenoir/fileserver

I now had a go package with no source code.

Without github

You could do this without github or any online source code repository and just create a local folder and reference that. But using github.com does have some advantages.

Importing the Package

To import that newly created but empty package into another project, I could use the following import statement:

import (
    "github.com/AaronLenoir/fileserver"
)

Of course, this will give an error:

No buildable Go source files in ~/go/src/github.com/AaronLenoir/fileserver

This is because there is no code, yet.

Breaking Up The Code

There are many ways to go about this. But I always like to start writing down how I want to use my package and work from there.

So my goal was something like this:

s := fileserver.Server("folder", {forceHttps: true});

The second argument would then have some flags to indicate what functionality I wanted to enable.

I'm not sure if it's "idiomatic Go" to pass a list of configuration settings to another function. Perhaps a better option is simply to pass actual arguments for each setting. But it worked for me at the time.

Doing it this way does mean that my single fileserver has quite some mixed tasks:

• Block requests for "dot-file"
• Force https
• Disallow directory listings

Creating the Server

The hook into my package is the "fileserver.Server()" function, which loads up a "server".

To create this function, I simply had to make a file in my package: fileserver.go.

In there, I simply add one function:

func Server(root string, config Config, filters []Filter) http.Handler {
    // ...
}

It looks a bit different from my original plan, but the purpose is the same: to spin up a server with the desired characteristics.

The three arguments are:

• root: simply the path of the directory from which to start serving files
• config: a simple struct with several bool fields representing configuration options
• filters: an (optional) array of "Filter" implementations, can be nil
  • I use this to allow users to add their own functionality to filter requests, next to the "built in" ones

For the config, the struct looks like this:

type Config struct {
    ForceHTTPS     bool
    AllowDirList   bool
    AllowDotPrefix bool
}

Organizing Code Inside the Package

For the Filter feature, I had to create a "Filter" interface:

type Filter interface {
    Validate(path string) bool
}

Additionally, I had some implementations of filters, like:

// Used to prevent serving of files that are private
// Currently only basic check: does the name start with a .?
type privateFileFilter struct {
}

// Check if the path refers to a file or directory whose
// names starts with a .
func (f *privateFileFilter) Validate(fullPath string) bool {
    basePath := path.Base(fullPath)
    isPrivate := (strings.HasPrefix(basePath, "."))
    if isPrivate {
        return false
    } else {
        return true
    }
}

I could just add all that code to the same fileserver.go, but that's not very convenient. So I created a second file, "Filter.go", and added it there.

I do not have to import that file anywhere in order for it to be included in the build of the package.

In the same way, I implemented the "dot-file" filter and https forcing in a seperate file called "middleware.go". I don't know if that makes much sense but it did at the time, so that's good enough for now.

Tests in Go

Go's tooling foresees some test infrastructure. If you run the command "go test", go will try to find tests in your package and run them:

~/go/src/github.com/AaronLenoir/fileserver$ go test
PASS
ok      github.com/AaronLenoir/fileserver   0.059s

I created some tests in a file "fileserver_test.go". The "go test" command will only look at "_test" files. So the filename is important.

Here's one of my tests:

// Tests the fact that a private file will yield a 404 error.
func TestGetPrivateFile(t *testing.T) {
    config := Config{ForceHTTPS: false, AllowDirList: false, AllowDotPrefix: false}
    recorder := httptest.NewRecorder()

    RunGetRequest(config, "http://localhost:4003/test/.text.txt", recorder)

    if recorder.Code != 404 {
        t.Error("Expected 404, received ", recorder.Code)
    }
}

Important to note here is that this function name must start with "Test", and that it must have one argument "t *testing.T").

Using the Package

In the end, importing the package can be done by "import" of the following package name:

• "github.com/AaronLenoir/fileserver"

If you use this package in a project, before you build you can run the "go get" command.

This will automatically fetch the git repo from github and put it in the expected location (src/github.com/AaronLenoir/fileserver).

This is the advantage of using something like github for your package.

:~/go/src/openshift/files$ go build
main.go:24:2: cannot find package "github.com/AaronLenoir/fileserver" in any of:
    /usr/lib/go/src/pkg/github.com/AaronLenoir/fileserver (from $GOROOT)
    /home/aaron/go/src/github.com/AaronLenoir/fileserver (from $GOPATH)
:~/go/src/openshift/files$ go get
:~/go/src/openshift/files$ go build

Comments and Documentation

Go has a built-in system for generating reference documentation for packages based on the comments in the source files. It's accesible through the "godoc" command.

This command can either print documentation on the comman line or spin up a local webserver so that you can browse through the docs (like you would on golang.org for the Packages).

For this to work, comments above functions, Interfaces, ... have to be a little bit structured.

For example, my main function to create a server has the following comments:

// Creates an http file server that will serve the files present in the root
// folder and its subfolders.
//
// root: directory from which to serve static files
//
// config: some configuration settings to define the behaviour of the static file
// server
//
// filters: Optionally some fileserver.Filter implementations to add custom
// content filtering (based on the full path of the file)
func Server(root string, config Config, filters []Filter) http.Handler {
    handler := http.FileServer(http.Dir(root))

    if config.ForceHTTPS {
        handler = forceHttps(handler)
    }

    internalFilters := getFilters(config)
    copy(internalFilters, filters)

    handler = protectFiles(handler, internalFilters, root)

    return handler
}

Running the following command will let me view the docs of my package on: http://localhost:6060/pkg/github.com/AaronLenoir/fileserver/

godoc -http:6060

Note how http://localhost:6060/ is now basically running the golang website. But if you browse to "Packages" you will also see your local packages documented.

Godoc.org Package Search

One other advantage of putting your packages online on github is that you will automatically get your code documentation available on godoc.org.

As an example, visit https://godoc.org/github.com/AaronLenoir/fileserver to view the latest docs from my little experiment.

This can help you browse through the mass of go projects on github.

Conclusion

To really do idiomatic Go all the way, I probably know too little about it. But at least I was able to:

• Get the functionality that I wanted
• Package and documented it correctly
• Get it online and available for re-use

And this was all relatively easy, which means it is indeed possible to quickly become productive with Go. But as always, it takes a lot longer to become an expert.

Now all I need is something to actually server through my fileserver ...