Ready to Golang

This is, hopefully, the first of a series of posts related to the Go programming language. This doesn’t mean that I change the topic to Go, but I want to have my findings on Go a little bit documented.

I won’t detail things too much as I did with Erlang and Elixir. I’ll just document the facts I find interesting.

This is a learning exercise, so please contact and correct me if you find any mistake.

Let’s Go!!!!

The first toy program

We’ll create a program that multiply some integers hold in an array by 2 and then by 4 and that will print the result on the terminal. In order to use the Go concurrency features, each step is going to be represented as a goroutine which will pass the current step’s result to the next step, as we’d do in a regular Elixir transformation, but this time each step is going to be running in a different process.

[Goroutine main: array a[1,2,3,4,5,6,7,8] for each element in a]
        element
            |> [Goroutine: x * 2]
            |> [Goroutine: x * 4]
            |> [Goroutine: print res]

Speaking in Erlang terms, our program will be composed by four processes:

  1. The main process
  2. The x2 process
  3. The x4 process
  4. The logger process

The key points here are:

  • Processes communicate through channels
  • Processes listen to channels and the number of listeners per channel is unlimited
  • Processes react to the receviced work and can place new work into other channels
  • Processes terminate only when channels are closed and emptied
  • Channels are typed
  • Channels can be marked as: read, write or read/write

Let’s review the code.

package main

import (
	"fmt"
	"sync"
)

var wg sync.WaitGroup

type pair struct {
	value  int
	result int
}

func multiplyByTwo(input <-chan int, output chan<- pair) {
	defer wg.Done()
	defer close(output)

	for value := range input {
		output <- pair{value: value, result: value * 2}
	}
}

func multiplyByFour(input <-chan pair, output chan<- pair) {
	defer wg.Done()
	defer close(output)

	for value := range input {
		output <- pair{value: value.value, result: value.result * 4}
	}
}

func logResult(input <-chan pair) {
	defer wg.Done()

	for value := range input {
		fmt.Printf("Result: %d => %d\n", value.value, value.result)
	}
}

func main() {
	// Create all three channels
	chanByTwo := make(chan int)
	chanByFour := make(chan pair)
	chanLog := make(chan pair)

	// We'll note that we'll wait for three goroutines
	wg.Add(3)

	// Create the goroutines
	go multiplyByTwo(chanByTwo, chanByFour)
	go multiplyByFour(chanByFour, chanLog)
	go logResult(chanLog)

	// Values to multiply
	values := []int{1, 2, 3, 4, 5, 6, 7, 8}

	// Insert values into the calculation pipeline
	for v := range values {
		chanByTwo <- v
	}

	// Close the production channel
	close(chanByTwo)

	// Wait for all three goroutines
	wg.Wait()
}

Goroutines

Goroutines are just regular functions. Functions, when invoked with the go keyword, run in its own execution unit. In Erlangs terms we’d say that they run in its own process.

The Wait Group

The wait group is a counter of the number of running goroutines. It’s not automatic, instead it has to be manually mantained.

Its methods are:

  • Add => Increase the counter
  • Done => Decrease the counter
  • Wait => Wait until the counter is zero

The defer keyword

The defer keyword allows to defer a cleaning operation until the end of the current function indepently of the exit path. In this case is used to mark the goroutine termination and to close channels.

Channels

Channels are quite similiar to Erlang messages, but they are pure queues. They can be buffered or unbuffered and can be marked as read only, write only or read and write. They are typed, a channel of ints is not the same as a channel of strings.

Channels are created with the make builtin function. Objects created with make are reference types, which means that they are actual values that hold a reference to the actual object. Passing a reference type means passing the object by reference, even through a channel. I think this is the trickiest part of the language, but it’s still easy enough if you are familiar with the C language.

Channels are closed with the close function. A closed channel is kept consumed until all its content is retrieved. Once a channel is empty all its listener receive a termination signal.

for range a channel

A typical pattern in Go is to loop over a range of a channel. This way the loop is alive until the close of the channel arrives.

The execution

Note the correct order of the loop. We are not throwing work to a pool, we’re making an ordered execution, just spread across several processes.

juanmi@jmimac2 ~/go/demo1/src (master) $ go run demo1.go
Result: 0 => 0
Result: 1 => 8
Result: 2 => 16
Result: 3 => 24
Result: 4 => 32
Result: 5 => 40
Result: 6 => 48
Result: 7 => 56
juanmi@jmimac2 ~/go/demo1/src (master) $

Conclusion

I like Go. It’s minimalist and quite convinient to work with. In my experience it’s quite easy to have a pretty complex working program in just a few minutes. The queues abstraction based on channels allows to develop complex systems quite easily.

Eventhough Go is not a functional language, quite a lot of the functional patterns still apply. Goroutines can be closures, solutions can be modeled as data transformation chains and to create inmutable objects is quite easy. Try it.

That’s it.

Have fun.

Permalink

On designing with sum and product types

I understand product types in statically-typed FP languages; they're records:

I understand sum (or union) types; they're cool:

As I learn Elm, I'm been having considerable trouble combining the two: should I model the problem as a sum type inside a record? as records in a sum type? For a good while in my Eecrit project, I bounced around alternatives like these:

The worst part was that I was developing iteratively, and new features kept breaking my model: I didn't find my design converging in the way that I'm used to from TDD and dynamic languages (be they OO like Ruby or FP like Clojure).

At one point I'd settled on a record something like this for a Form:

That worked well for editing existing animals (which was the first type of editing I implemented). But then I implemented the "you can create one new animal" story. The same record almost worked, except there might not be an originalAnimal.

(Editorial comment: the "this is like that except for..." problem is the core software design problem in messy domains, which are ones where all the "except for"s keep coming and can't be handled by insisting the business be "rational" - i.e., think the way we do.)

I wanted to make the distinction between the two kinds of form concrete - directly represented in the types - but I didn't have the energy. (I was already well behind my self-imposed schedule.)

So I just added a Maybe:

... and I smeared case form.originalAnimal of expressions throughout the code. That was particularly grating for places where I knew one of the cases was impossible. But I got the app working.

At some later point, I was whining on Twitter that the FP literature is really weak on dealing with messy systems. Lance Walton asked for an example. I provided this one. Mark Seemann suggested that I push all the special-case code into one place. It still might not be the Right Model, but at least that "isolates the place where you have to implement changes".

This is absolutely standard OO design practice, right? Put the functions next to the data; hide the data behind the functions. It's also what I'd naturally do in Clojure or Elixir.

However, my first pass at trying it made me think it wouldn't work: Types.Form would end up knowing too much about the rest of the program. Also, I was hung up on the idea that if I changed my app, I should do so in a way that made illegal combinations of values impossible.

Nevertheless, after one restart, it worked out pretty nicely. That is, the functions make sense: with one exception, they're "about" the Form, not about how other code uses it. Even that one function is not horrible. It's used like this:

(Note: line 5 is idiosyncratic. I use pipelining heavily so as not to have to keep messing with (model, cmd) tuples.)

The name givenOriginalAnimal isn't great, but I don't have a better one. The expression is meant to convey that if there's an animal attached to the form, that animal replaces the form in the display. Left only implied is that there will always be an associated animal. (But if the impossible happens, the whole thing is a no-op.)

Here's the code for givenOriginalAnimal:

You can find the whole file here.

To me, the only interesting thing about this episode (other than why I didn't do this in the first place) is how like this solution is to OO practice. This seems exactly what I'd do in Ruby. Perhaps after a time I might decide that distinguishing two subclasses might make sense. That'd be a pretty straightforward refactoring.

What I want to discover in Elm (and, someday soon, Purescript) is how to do that gradual, undisruptive improvement. I think that will require surfacing things that are underdiscussed and perhaps unpopular in the statically-typed FP community:

  1. Patterns as "attractors" in design space.
  2. Satisficing as a design goal, or: valuing types that are not "lawlike" in the way that monads are but are nevertheless useful.
  3. Relatedly, acceptance that designs get (haltingly!) better over time, and that such a process of design is worthy of deep study.

I very much want to see those things happen, and am tempted to charge forward on them myself. But the social/community issues are crucial, and I doubt I'm deft enough to handle them.

P.S. I've also tried using structural typing toward this same end:

It didn't work well.

Permalink

MongooseIM 2.0.1 more stability for app developers

## SSE for server to client real-time push SSE stands for [Server-Sent Events](https://www.w3.org/TR/eventsource/), it is a W3C recommendation where a browser receives automatic updates from a server via HTTP connection. It is for mobile and web developers using the MongooseIM REST API. It is not intended to be used with XMPP connections (TCP, websockets, BOSH). The problem with a simple REST API is that it is polling only: the client has to continuously poll the server at regular intervals, even when there is nothing to expect, “just in case”. This regular polling consumes a lot of battery on the client and bandwidth both on the client and server, all for nothing. Also, this introduces some delay, as data can be made available on the server between the polling interval. SSE allows a very simple and basic push from server to client, with just a tiny library to use in your app. SSE is very easy to integrate on the client, saves battery and bandwidth, and is fully real-time (no delay). ## Improved Cassandra for MAM Sysadmins and devops will love the return of support for the Cassandra database. Our support for Cassandra in versions below 2.0.0 was average. Sometimes it was slow, and it had limited support of MAM (Message Archive Management) specifications, so we decided to remove it and bring it back once it was fully production ready. Now MongooseIM supports Cassandra again, with optimised code and improved support for MAM, and it comes bundled with a migration tool. Cassandra for MAM in MongooseIM is now faster and easier to upgrade. ## HTTP file upload This will help all app makers. The file exchange in XMPP is usually and historically off chats: it is made with a synchronous user experience, with an offer and an acceptation or refusal whereby you have to wait for the other party to accept or refuse. In many cases there are failures due to timeouts. The solution is an asynchronous user experience: you send files in chats, whether your contacts are online or not, available or not. You now have a single timeline with text messages and (links to) media files. It is also compatible with archiving. ## Further improvements ### MUC hibernation All the old and inactive MUC rooms are now hibernated and eventually flushed out of memory, which saves a significant amount of memory on the cluster. ### ODBC/RDBMS backend for MUC light MUC light was only available on Mnesia, now it can also be stored in MySQL or PostgreSQL. ### Simplified MAM configuration Before, MAM had to be configured in many ways in multiple places. Now, it is done in one central place, and there is fewer things to configure. ### Changelog Please feel free to read the raw [changelog](https://github.com/esl/MongooseIM/releases/tag/2.0.1), you can even follow links and find the code changes. ## Special thanks to our contributors! Special thanks to our contributors: [@kenstir](https://github.com/kenstir), [@sstrigler](https://github.com/sstrigler), [@igors](https://github.com/igors), [@bernardd](https://github.com/bernardd), [@msantos](https://github.com/msantos)! ## Next? ### From 2.0.0 to 2.0.1: please upgrade! Version 2.0.1 will probably be the last of the 2.0.x series, unless major issues are found. As this is a massive improvement, we advise everyone to upgrade. We are now focusing on the development of the 2.1.x series. This will deliver even more value to mobile developers and sysadmins. ### Mobile clients: social and groups We are planning to release open source iOS and Android clients. Please do not expect market killer apps. The aim is very limited: we want to provide a technology and use case demonstration. Mangosta iOS and Mangosta Android will deliver very basic group chat and social network. For example, these will not be distributed on mobile app stores, they will only be available as source code on our repositories. ### Flexible push notifications You can expect more improvements on the push notifications front. We have now an existing generic mod _ http _ notification, we will add push to Amazon SNS (Simple Notification Service) and [XEP-0357: Push Notifications](http://xmpp.org/extensions/xep-0357.html). ### Peer to peer or mediated binary streaming We will deliver an ICE/STUN/TURN server, coded in Elixir, to allow peer to peer and one to one media streaming, like voice, video, screen sharing, and whiteboarding. The signalling part, named Jingle, is already available. ## Call for action Star our repo: [github.com/esl/MongooseIM](https://github.com/esl/MongooseIM) Follow us on Twitter: [twitter.com/MongooseIM](https://twitter.com/MongooseIM/) & [twitter.com/ErlangSolutions](https://twitter.com/ErlangSolutions/)

Permalink

SpellingCI: No more spelling mistakes in your markdown flies!

Spelling CI / Sheldon

You probably know the feeling...

...when you fix a very important issue in that "super cool" open-source project. You keep all your attention to those parts of the code where your changes could affect something, you run the tests until you can confirm: "It is fixed now, I am sure"; but then again, you run all the tests again up until, as the brave developer you are, you click the "open pull request" button. You are happy because you know the problem is fixed, everyone is going to love you... but a reviewer appears and rejects your changes because, on the documentation, you wrote "wihle" instead of "while". Damn!

You have to pay as much attention to your documentation as to your code, but when English is not your mother tongue, you will definitely have to spend time improving your grammar and spelling. If your problems have to do with grammar, sorry man, but you'll have to continue studying... But if they are related to spelling, here at Inaka we developed a tool to help you out with that...

Sheldon

Sheldon is a simple Erlang Application for spell checking. You can find it here. Its API accepts a text (iodata() for Erlangers) and checks all the words. If sheldon doesn't understand a word it will try to help you with some suggestions. For suggestions we were inspired highly by the Elixir project spell_check.

Let's try:

1> sheldon:start().
ok
2> sheldon:check("My name is Felipe and I am aware wrote wihle instead of while once").
#{bazinga => <<"Too bad, I'll have a virgin Cuba Libre.">>,
  misspelled_words => [#{candidates => ["while","wile"],line_number => 1,word => "wihle"},
   #{candidates => ["flipe","feline"],line_number => 1,word => "Felipe"}]}

Sheldon has detected our misspelled word ("wihle") and the line where it is; it also gave us some suggestions ("while" and "wile") and a bazinga message.

But wait! sheldon doesn't recognize my royal name ("Felipe"). I confess that the "feline" suggestion is cool too, but my parents wouldn't be happy if I switch it now. This is because sheldon works with a dictionary and, as expected, it is impossible to have all the words there. We have 2 choices here: update the dictionary or escape the word. Let's escape!

sheldon gives you the possibility to escape some words like "Felipe" in this example. We only have to create a configuration map like this.

3> Config = #{ignore_words => ["Felipe"]}.
#{ignore_words => ["Felipe"]}
4> sheldon:check("My name is Felipe and I am aware wrote wihle instead of while once", Config).
#{bazinga => <<"Does it affect me? Then suffer in silence.">>,
  misspelled_words => [#{candidates => ["while","wile"],line_number => 1,word => "wihle"}]}

Great, now "Felipe" is ignored.

That was a very simple example, but sheldon also provides the way to ignore patterns or blocks of code. For that you can check this example.

But... what if one day we are the Pull Request Reviewers? How can we prevent someone from writing "wihle", even if we think "wihle" is correctly spelled?

Spelling CI

At Inaka we also wrote Spelling CI. It is a continuous integration tool built on top of sheldon. Basically you must log in using your Github account and select the Github projects you want to check.

spellingci is only integrated with Github platform. It works for both public and private repositories.

By default, it will check the spelling for all the .md and .markdown files in your branch, but you can configure spellingci in order to check different ones or ignore some words, patterns, blocks of code... all through a configuration json file.

Every time sheldon finds a misspelled word, it will write a comment there with the suggested words.

Lovely, isn't it?

Where to find them

sheldon and spellingci are open-source. You can find the source code here:

sheldon is also available at hex

Here is spellingci deployed, feel free to use it and check your PRs!

Conclusion

As software developers, we tend to focus entirely on the code and sometimes we forget about the spelling. With sheldon and spellingci, at least we have help. For example, this blog is a Github repository, so now we are going to check our post with sheldon/spellingci

And remember: for grammar... keep studying!

Permalink

Copyright © 2016, Planet Erlang. No rights reserved.
Planet Erlang is maintained by Proctor.