Elixir Module Attributes - Alchemy 101: Part 1

Elixir is a joy to work with, an easy installation that comes packaged with a build tool, mix. In minutes you will be creating your own applications and performing releases. Over time you will question how and why things work and behave the way they do. You may even wonder if there are better ways to do things. This is where Alchemy 101 comes in. Each edition of Alchemy 101 will address common questions that are being asked throughout the community.

Today we will be looking at when module attributes are evaluated and what this means. Take a look at the Set up section and then proceed to Module Attributes.

Set up

You can follow along with the examples. You will require elixir and to perform the following steps. First create a new mix project.

mix new my_app --module Twitter
cd my_app

 

Next add distillery (for creating releases) to mix.exs as a dependency.

defp deps do
  [{:distillery, "~> 0.10.1"}]
end

 

Then download distillery and create the release configuration.

mix deps.get
mix release.init

 

The rest of the blog assumes that you are in the my_app directory.

Module Attributes

Module attributes can be used to annotate your module (e.g. with documentation) and add constants. Module attributes are evaluated at compile time, not runtime. Not knowing this can lead to confusion. All occurrences of the module attribute are replaced with whatever it evaluates to at compile time. I will demonstrate this below using a pattern that I've seen a few times.

Open lib/twitter_client.ex and add the following to it.

defmodule Twitter do
  require Logger

  @twitter_client Application.get_env(:my_app, :twitter_client)

  def log_twitter_client do
    Logger.info("Using #{@twitter_client}")
  end
end

 

There will be 2 possible values for the application's environment configuration: twitter_client, TwitterClient and MockTwitterClient. A small side note, an application's environment configuration shouldn't be used for global variables. The first value is for production and the second is for test and dev. The value used will be determined by the mix config file imported by config.exs. Go ahead and add the following files.

# config/config.exs
use Mix.Config
import_config "#{Mix.env}.exs"

# config/prod.exs
use Mix.Config
config :my_app, twitter_client: TwitterClient

# config/dev.exs
use Mix.Config
config :my_app, twitter_client: MockTwitterClient

 

So when MIX_ENV (environmental variable) is dev then config.exs will import dev.exs, when prod it will import prod.exs.

Go ahead and create the release and start it in the console.

mix release
rel/my_app/bin/my_app console

Now check to see if the configuration has been applied.

iex(my_app@127.0.0.1)1> Twitter.log_twitter_client()
09:32:16.220 [info]  Using Elixir.MockTwitterClient

 

What? Why is it using the MockTwitterClient? That's strange. Lets check sys.config (generated from the release).

cat rel/my_app/releases/0.1.0/sys.config

[{sasl,[{errlog_type,error}]},
 {my_app,[{twitter_client,'Elixir.MockTwitterClient'}]}].

 

So the value of twitter_client is MockTwitterClient. This is because we ran 'mix release' without specifying prod as MIX_ENV (defaults to dev). No big deal, lets just change sys.config to the following, restart the application and check again.

[{sasl,[{errlog_type,error}]},
 {my_app,[{twitter_client,'Elixir.TwitterClient'}]}].

rel/my_app/bin/my_app console
iex(my_app@127.0.0.1)1> Twitter.log_twitter_client()
09:32:16.220 [info]  Using Elixir.MockTwitterClient

 

The mock is still showing, but why? As mentioned before module attributes are evaluated at compile time, not runtime. All occurrences of @twitter_client at compile time were replaced with MockTwitterClient, it just so happens that this value came from a configuration file thus giving us the illusion that it can be changed.

To have TwitterClient we must create another release and set MIX_ENV to prod.

MIX_ENV=prod mix release

 

For good measure place this into a script/Makefile/something that prevents this from happening again. An alternative is to use a function instead giving you the luxury of changing the value in sys.config post release.

def twitter_client do
  Application.get_env(:my_app, :twitter_client)
end

 

The takeaway from this is: when using module attributes, always be aware of what they evaluate to, making a wrong assumption can result in having to recompile your project.

Stay tuned, in the next edition we will be looking at what really happens to your mix configuration when you create a release.

Permalink

Erlang.org website source code now on Github

img src=http://www.erlang.org/upload/news/

We have just released the source code for erlang.org website on Github. 

https://github.com/erlang/erlang-org

The website is written in Cowboy, ErlyDTL and sumo_db. It is licensed under Apache License 2.0. You can follow the instructions to set up. 

The erlang.org website is developed and maintained by Industrial Erlang User Group in collaboration with Erlang/OTP team at Ericsson.

Happy hacking!

Permalink

Build a complete iOS messaging app using XMPPFramework - Tutorial Part 1

YAXT??! Yet another XMPP tutorial?

 

Well, this is going to be another tutorial, but I'm going to try to make it a little bit different. This is an XMPP tutorial from an iOS developer's perspective. I'll try to answer all the questions I had when I started working in this area. This journey is going to go from no XMPP knowldege at all to having a fully functional instant messaging iOS app using this cool protocol. We are going to be using the super awesome (yet overwhelming at the beginning...) XMPPFramework library, and the idea is also to also mix in some iOS concepts that you are going to need for your app.

What's XMPP?

 

From Wikipedia: Extensible Messaging and Presence Protocol (XMPP) is a communications protocol for message-oriented middleware based on XML.

This basically means XMPP is a protocol for exchanging stuff. What kind of stuff? Messages and presences. We all know what messages are, but what about presences? A presence is just a way of sharing a "status", that's it. You can be 'online', 'offline', 'having lunch', or whatever you want. Also there's another important word: Extensible meaning it can grow. It started as an instant messaging protocol and it has grown into multiple fields for example IoT (Internet of Things). And last, but not least: every piece of information we are going to exchange under this protocol is going to be XML. I can heard you complaining but... Come on, it's not that bad!

Why do we need XMPP? Why not just REST?

 

Well what other options do we have? On the one hand, a custom solution means building everything from scratch, that takes time. On the other hand, we have XMPP, a super tested technology broadly used by millions of people every day, so we can say that's an advantage over a custom approach.

Everytime I talk about XMPP, someone asks me 'Why not just REST?'. Well, there is a misconception here. REST is not a protocol, it's just a way of architecting a networked application; it's just a standarized way of doing something (that I love btw). So let's change the question to something that makes more sense: "Why not just build a custom REST chat application?". The first thing that comes to my mind is what I already explained in the previous paragraph, but there is something else. How do I know when someone has sent me a message? For XMPP this is trivial: we have an open connection all the time so, as soon as a message arrives to the server, it will send us the message. We have a full-duplex. On the other hand, the only solution with REST is polling. We will need to ask the server for new messages from time to time to see if there is something new for us. That sucks. So, we will have to add a mechanism that allows us to receive the messages as soon as they are created, like SSE or WebSockets.

There is one more XMPP advantage over a custom REST chat application. REST uses HTTP, an application level protocol that is built on top of a transport level protocol: TCP. So everytime you want to use your REST solution, you will need HTTP, a protocol that is not always available everywhere (maybe you need to embed this in a cheap piece of hardware?). Besides, we have XMPP built on top of TCP that's going to be always available.

What's the basic stuff I need to know to get started?

 

Well, you know a lot already but let's make a list. Lists are always good:

  • XMPP is built on top of TCP. It keeps an open connection all the time.
  • Client/Server architecture. Messages always go through a server.
  • Everything we send and receive is going to be XML and it's called Stanza.
  • We have three different types of stanzas: iq, message and presence.
  • Every individual on the XMPP network is univocally identified by a JID (Jabber ID).
  • All the stanzas are cointained in a Stream. Let's imagine the Stream as a white canvas where you and the server write the stanzas.
  • Stream, iq, message and presence are the core of XMPP. You can find everything perfectly detailed in RFC6120
  • XMPP can be extended to accomplish different stuff. Each extension is called XEP (XMPP Extension Protocol).

 

What's a JID?

Jabber ID (JID) is how we univocally identify each individual in XMPP. It is the address to where we are going to send our stanzas.

This is how a JID looks like:

  • localpart: This is your username.
  • domainpart: Server name where the localpart resides.
  • resourcepart: This is optional, and it identifies a particular client for the user. For example: I can be logged in with andres@erlang-solutions.com on my iPhone, on my Android and on my mac at the same time... So all these will be the same localpart + domainpart but different resourcepart

I'm sure you have already noticed how similar the JID looks to a standard email address. This is because you can connect multiple servers together and the messages are rooted to the right user in the right server, just as email works. Pretty cool, right?

Sometimes you will see we have a JID with just the domain part. Why?! Because it's also possible to send stanzas to a service instead of a user. A service? What's a service?! Services are different pieces of an XMPP server that offer you some special functionality, but don't worry about this right now, just remember: you can have JIDs without a localpart.

What's a Stanza?

Stanza is the name of the XML pieces that we are going to be sending and receiving. The defined stanzas are: <message/><presence/> and <iq/>.

 

<message/>

This is a basic <message/> stanza. Everytime you want to send a message to someone (a JID), you will have to send this stanza:

<message from='andres@erlang-solutions.com/iphone' to='juana@erlang-solutions.com' type='chat'>
    <body>Hey there!</body>
</message>

 

<iq/>

It stands for Info/Query. It's a query-action mechanism, you send an iq and you will get a response to that query. You can pair the iq-query with the iq-response using the stanza id.

For example, we send an iq to the server to do something (don't pay attention to what we want to do... you just need to know there is an iq stanza and how the mechanism works):

<iq to='erlang-solutions.com' type='get' id='1'>
  <query xmlns='http://jabber.org/protocol/disco#items'/>
</iq>

And we get back another iq with the same id with the result of the previous query:

<iq from='erlang-solutions.com' to='ramabit@erlang-solutions.com/Andress-MacBook-Air' id='1' type='result'>
    <query xmlns='http://jabber.org/protocol/disco#items'>
        <item jid='muc.erlang-solutions.com'/>
        <item jid='muclight.erlang-solutions.com'/>
        <item jid='pubsub.erlang-solutions.com'/>
    </query>
</iq>

 

<presence/>

Used to exchange presence information, as you could have imagined. Usually presences are sent from the client to the server and broadcasted by it. The most basic, yet valid presence, to indicate to the server that a user is avaiable is:

<presence/>

After a sucessfull connection, you are not going to receive any <message/> until you make yourself available sending the previous presence.

If you want to make yourself unavailable, you just have to send:

<presence type="unavailable"></presence>

If we want to make the presences more useful, we can send something like this:

<presence>
      <status>On vacation</status>
</presence>

 

What's a Stream?

Before answering this, let's refresh our mind. What's a Unix socket? From Wikipedia: A socket is a special file used for inter-process communication. These allows communication between two processes. So a socket is a file that can be written by two processes (in the same computer or in different computers in the same network). So the client is going to write to this file and server too.

Ok, but how is a socket related to a Stream? Well, we are going to be connected to a server using a socket, therefore we are going to have a 'shared file' between the client and the server. This shared file is a white canvas where we are going to start writting our XML stanzas. The first thing we are going to write to this file is an opening <stream> tag! And there you go... that's our stream.

Perfect, I understand what a stream is, but I still don't understand how to send a message to the server. Well, the only thing we need to do to send a message is writting a <message/> stanza in our shared file. But what happens when the server wants to send me a message? Simple: it will write the message in the 'shared file'.

Are we ok so far?

 

I'm sure at this point you have questions like:

  • "What?! An active TCP connection open all the time? I'm used to REST! How am I going to do that?!" 

​           Easy, you don't have to care about that any more! That's why we are going to use the library, and it will take care of that.

  • "You said nothing about how to connect to the server!"

           Believe me, you don't have to care about this either. If we start adding all this info, we are going to get crazy. Trust me, I've been there.

  • "What about encrypted messages? We need security! How are we going to handle this?"

          Again, you don't have to care about this at this point. Baby steps!

 

You just need to be able to answer: "What's XMPP?", "How do you send a message?", "How do you change your status in XMPP?", "How do you ask something to the server?", "What's a Stream?". If you can answer all that, you are WAY better than me when I started.

All the concepts we described so far are the core of XMPP. In part 2 you will find out how to get started with XMPPFramework, connecting to the server and authenticating a user!

Permalink

GenStage and Bioinformatics

In this post, I will describe my first attempt at using GenStage - in partiuclar, its Flow module - to implement a bioinformatics algorithm. My day job for the last couple of years has been the building of a full-stack system for the storage and searching of the metagenomic analyses (metagenomics) of hydrocarbon resources. That's a mouthful. Basically it means that there is a need to know what

Permalink

Phoenix, Elm, and multiple single-page apps

In the running example used in Programming Phoenix, client-side apps are launched from the master app.js file. The Javascript that's loaded for every page checks to see whether this page has the id it's looking for. If so, the code to launch the app is run.

That's fine for the book, as it avoids irrelevant complexity, but I found it awkward as my overall app grew. Multiple clauses of this form:

const ivDiv = document.querySelector('#iv-target');  
if (ivDiv) {  
    Elm.IV.embed(ivDiv);
}

... are untidy. I'd rather have the "start the client-side" code exist only on the single page for the app. For this Javascript newbie, that turned out to be harder than I'd expected. So I've frozen a branch that shows how, and I explain it below.


Disclaimers:

  • This is for Phoenix 1.2.1 and Elm 0.17.
  • The code is a snapshot of a fair amount of learning-through-flailing about Elm, Phoenix, and the Bulma CSS library. The master branch may be a better guide going forward. This version might be easier to understand, though - less generalization.
  • Probably I'm doing things more awkwardly than need be. I may update this post if people tell me how.

About the Elm code

  • The Elm code won't take over the whole screen. Instead, it will be embedded inside a standard header and footer. (Some pages of the overall app are just boring old HTML.)

  • Perhaps mistakenly, I'm dividing the overall app into multiple single-page apps. The overall app will manage teaching animals for schools of Veterinary medicine. So there will be a single-page app for managing animals, one for managing medical procedures and the business rules around them, one for making or modifying reservations, and so on. Here, we'll be dealing with the very beginnings of the animal-management part. (Elm Source. Right now, it just puts a little information in a <p> tag:

        view : Model -> Html Msg
        view model =
          p []
            [text <| "Animal has been started with argument \""
               ++ model.aStringFlag ++ "\" and "
               ++ toString(model.aNumberFlag)]
  • The app uses Html.App.programWithFlags. Since I found passing arguments to Elm puzzling at first, I figure the example might be useful. However, I won't be explaining it here.

About Brunch and the layout file

My brunch-config.js file is modified from one by OvermindDL1. The key clause is this:

    elmBrunch: {
      elmFolder: "web/elm",
      mainModules: ["IV.elm", "Animals.elm"],
      outputFolder: "../static/js",
      outputFile: "critter4us.js"
    },

The important bit here is that all the various Elm apps (plus the Elm runtime) are bundled together into a single critter4us.js file.

After that's done, there's a further step that bundles critter4us.js with other Javascript files into a single app.js file that gets dumped in priv/static/js/app.js. Phoenix's standard app.html.eex layout loads that on every page:

<!DOCTYPE html>  
<html lang="en">  
  <head>...</head>

  <body>
    ...
    <script src="<%= static_path(@conn, "/js/app.js") %>"></script>
  </body>
</html>

Because of historical accident, I don't use app.html.eex, but my layout file follows the same pattern.

The question at hand is: what page-specific code to put after that <script src= line in order to launch the single-page app appropriately?

The Javascript to start the Elm app

The answer is this:

<script type="text/javascript">  
  const elmDiv = document.querySelector('#embed-elm-here');
  require("web/static/js/critter4us.js").Animals.embed(elmDiv, {
     aNumberFlag: "5845",  aStringFlag: "string", 
  })
</script>  

Notice that:

  1. The require refers to the name of an intermediate file. That is, we have to refer to the result of the Brunch step that produces web/static/js/critter4us.js. I was surprised I had to refer to it, given that it's neither a source file nor the final result (priv/static/js/app.js). But that's the way it is.

  2. Note the dereference after require(...) is Animals. That matches the definition of my single-page app: module Animals exposing (main). I'd like to think it would all work if I were trying to launch MyBigApp.Animals, but I haven't tried it.

  3. (An aside) As far as I can tell, you can only pass strings to an Elm programWithFlags. So the Elixir side has to convert the integer 5845 into a string, which the Elm side must convert back into an integer - even though Javascript would be fine handling both strings and integers.

Various files

  • You wouldn't want a reference to a specific Elm app to be in this file - the layout used for all pages. At the moment, I use this rather crunky code:
  <%= if Map.has_key?(assigns, :elm_launcher), do: @elm_launcher, else: "" %>
  • For the moment, the Javascript shown above is generated in the controller.

  • I wanted each controller action to use a <div id="embed-elm-here"> while also wrapping that div in a layout. I couldn't quickly find a way to do that, so I made a dumb template that contained nothing but that div. In the controller, it's rendered with render(Eecrit.ElmView, "elm_hook.html"). That means controllers look like this:

  def index(conn, _params) do
    conn
    |> launch_page_app_with_flags(aStringFlag: "string", aNumberFlag: 5845)
    |> render(Eecrit.ElmView, "elm_hook.html")
  end

Permalink

Copyright © 2016, Planet Erlang. No rights reserved.
Planet Erlang is maintained by Proctor.