Q&A with Programmer and #OpenErlang Illustrator Daniel Stori

We were delighted to have Daniel Stori onboard with our #OpenErlang celebrations. As an open source advocate, Daniel put his two fine talents - open source programming and illustrating - to good use when creating the #OpenErlang comics.

About Daniel

Software specialist, Daniel Stori has been in the industry for the past 20 years. He has been working in C, C++, Java, JavaScript, Golang, Python, Ruby, etc. Nowadays, he is working with cloud computing related technologies, specially with Kubernetes.

Intrigued about where his inspiration has come from, we decided to grill Daniel on getting young people into coding, the appeal of programmer humour, and why programming and creativity aren’t so far apart after all…

Q: How did you get involved with the #OpenErlang campaign?

A: I received a message from Erlang Solutions Team to be part of #OpenErlang party in the last year. Initially, the idea was to do some live drawing at the party, but I had some health issues at that time, so we decided to do a kind of remote live drawing instead. From there, we continued collaborating.

Q: Why did you agree to get onboard with #OpenErlang?

A: Because of the 3G: Great people, Great community and Great technology.

Q: Why do you like Erlang?

A: I never had a chance to work in an Erlang project but I have studied it a lot in the last years. I think that Erlang is wonderful because you need to think in distributed systems from the very beginning.

Q: What is your programming language of choice and why?

A: I’ve worked as a software developer for 20 years, my primary language in the last few years was Java. Today I am more involved with devops tools, primary with Kubernetes. So golang and Javascript (nodejs) are my favourite languages nowadays.

Q: Many would think the programming/developer industry is very mathematical, and it is, but how is it also creative?

A: I think the industry has always demanded creativity over time, but with different approaches. In the past, the problem was less complex but the resources were very limited. Today we face more complex problems, but we have tons of ways to solve them.

Q: How would you describe the creativity in programming? It’s like a blank canvas, right?

A: I think it’s hard to describe creativity. Maybe this subject deserves a comic ;) I think there is not a rational path to creativity, it’s more about inspiration.

Q: What came first for you, the illustrations or the programming?

A: When I was a little kid, I used to do cartoons with my family, so I think drawing comes first. I started to play with code when I was 12, 13 yo.

Q: Where do you get your ideas from?

A: I’ve worked in the industry in the last 23 years. The day-to-day running of the enterprise world is an endless source of ideas. Also the community itself, the online forums. We have a very controversial community.

Q: In what way controversial?

You know, we have a very passionate community, I think. People are used to defending their os, language, framework or technology like their family. In this scenario, it’s kind of easy to create jokes, because jokes always target a group.

Q: Has any idea gone awry?

A: Not a specific idea. Sometimes people think I am a kind of misogynistic person because I don’t put girls on my comics. The reality is that I really find it difficult to draw women! I also have difficulties to draw people looking to the left, so I use to draw them looking to the right and I swap it using the software.

Q: How do you deal with accommodating all programming languages? What sort of issues have you come across in the past?

A: I know that is cliché, but I really believe that different programming languages solve different problems. Erlang/Elixir is the first thing that came in mind for high scalable/resilient software. For classic enterprise software, maybe Java / c#. If you need to build easy to deploy tools for DevOps world, maybe Golang or Python. For web apps, Ruby, Node, etc. I know that these languages can solve other problems, but they work better in their domain.

Q: What’s your favourite selection of comics and why?

A: When I was a kid, I was Disney addicted. I also used to dig Monica’s Gang (Brazilian). Today I like alternative authors, like Noah Van Sciver.

Q: Programmer humour is so so popular - Reddit has 695K subscribers alone - what is it about programmer humour?!

A: My guess is because programmers are almost always online. In addition, we are more accurate when we do so directed humour.

Q: Do you have any tips when it comes to inserting more creativity in day-to-day programming?

A: My gold tip is to pass some time offline, not programming. Doing another kind of activities always help me to have good ideas for my programming issues.

Q: How about getting young people and other groups into coding? Wouldn’t highlighting the creative aspect of programming help to encourage growth in the industry?

A: In my days, programming was limited to nerds (like I was). We were weird and we liked to be weird people. I think today the community is very open to other groups because programming is hype. The hype is good to attract different people, with new mindsets, and this is very positive.

Q: Do you have any resources to help creativity flourish within programming?

A: Doing different activities. I like to do Yoga, meditation, cycling, watch movies, etc. Also, social interactions always help me to have new ideas. Sometimes we have a very well-defined approach to face a problem, so it’s important to share with other folks with different mindsets.

To discover more Daniel Stori comics, you can follow him on Twitter and Github. And also visit his website, of course!

Permalink

Twenty Years of Open Source Erlang

Erlang was released as Open Source Tuesday December 8th, 1998. Do you remember where you were that week? I was in Dallas (Texas); one of my many visits helping the Ericsson US branch set up an Erlang team working with the AXD301 switch. Waking up on Tuesday morning, I got the news.

The release was uneventful. There was no PR around the release, no build up or media coverage. Just a sparce erlang.org website (handcrafted using vi). An email was sent to the Erlang mailing list, a post made the front page on slasdot, alongside a mention on comp.lang.functional (which Joe dutifully followed up on). There were no other marketing activities promoting the fact that Ericsson had released a huge open source project. My highlight that week was not the release of Erlang, but going to a Marky Ramone and the Intruders gig in a dive in downtown Dallas. Little did I know how Open Source Erlang would affect the tech industry, my career, and that of many others around me.

Getting it out of Ericsson

What made it all happen? Many of us wanted Erlang to be released as open source, for a variety of reasons. Some of my Ericsson colleagues wanted to leave their current positions, but continue building products with what they believed was a silver bullet. Others wanted to make the world a better place by making superior tools for fault tolerant and scalable systems available to the masses. For Ericsson management, a wider adoption of Erlang would mean a larger pool of talent to recruit from.

Jane Walerud was amongst us trying to sell Erlang outside of Ericsson and one of the few who at the time knew how to speak to management; she understood that the time of selling programming languages was over. Håkan Millroth, head of the Ericsson Software Architecture Lab suggested trying this new thing called “Open Source”. Jane, armed with an early version of The Cathedral and the Bazaar paper, convinced Ericsson management to release the source code for the Erlang VM, the standard libraries and parts of OTP.

Until Erlang was out, many did not believe it would happen. There was a fear that, at the last minute, Ericsson was going to pull the plug on the whole idea. Open Source, a term which had been coined a few months earlier, was a strange, scary new beast large corporations did not know how to handle. The concerns Ericsson had of sailing in uncharted territory, rightfully so, were many. To mitigate the risk of Erlang not being released, urban legend has it that our friend Richard O’Keefe, at the time working for the University of Otago in New Zealand, came to the rescue. Midnight comes earlier in the East, so as soon as the clocks struck midnight in New Zealand, the erlang.org website went online for a few minutes. Just long enough for an anonymous user to download the very first Erlang release, ensuring its escape. When the download was confirmed, the website went offline again, only to reappear twelve hours later, at midnight Swedish time. I was in Dallas, fast asleep, so I can neither confirm nor deny if any of this actually happened. But as with every legend, I am sure there is a little bit of truth behind it.

The Dot Com Bubble Era

Adoption in first few years was sluggish. Despite that, the OTP team, lead by Kenneth Lundin was hard at work. In May 1999, Björn Gustavsson’s refactoring of the BEAM VM (Bogdan’s Erlang Abstract Machine) becomes the official replacement of the JAM (Joe’s Abstract Machine). Joe had left Ericsson a year earlier and the BEAM, whilst faster, needed that time to make it production ready.

I recall the excitement every time we found a new company using Erlang/OTP. Telia, the Swedish phone company, was working on a call center solution. And One2One - the UK mobile operator - had been initially using it for value added services, expanding its use to the core network. IdealX in Paris, did the first foray into messaging and XMPP. Vail System in Chicago and Motivity in Toronto were using it for auto dialler software. And Bluetail, of course, had many products helping internet service providers with scalability and resilience.

The use of Erlang within Ericsson’s core products continued to expand. This coincided with my move to London in 1999, where I increasingly came across the need for Erlang expertise within Ericsson. Erlang Solutions was born. Within a year of founding the company, I had customers in Sweden, Norway, Australia, Ireland, France, the US, and of course, the UK. In 2000, we got our first non Ericsson customer; training, mentorship and a code review for IdealX in Paris.

It was the Bluetail acquisition by Alteon Web Systems for $152 million (a few days later Alteon were acquired by Nortel), which sent the first ripples through the Erlang community. An Ericsson competitor developing Erlang products! And a generation of successful entrepreneurs who had the funds to get involved in many other startups; Synapse, Klarna and Tail-f being some of them.

Soon after the Bluetail success comes the dot com crash, the industry went into survival mode, and then later recovery, mode. The crash, however, did not affect academics who were moving full steam ahead. In 2002, Prof. John Hughes of Chalmers University managed to get the Erlang Workshop accredited by SIGPLAN and the ACM. We didn’t really know what this all meant, but were nonetheless, very proud of it. The ACM SIGPLAN Erlang Workshop in Pittsburgh (Pennsylvania) was the first accredited workshop. Here, a PhD student from Uppsala University named Richard Carlsson presents the Erlang version of try-catch to the world.

In September 2004, Kostis Sagonas, from Uppsala University hijacks the lightning talks at the ACM SIGPLAN Erlang workshop in Snowbird (Utah) and gives the first public demo of Dialyzer. He runs it on a code base from South African Teba Bank. It was the first of many amazing tools he and his students contributed to the ecosystem.

Erlang had for a long time been used to teach aspects of computer science in many Universities all over the world. This in turn lead to research, Master Thesis and PhD projects. The workshop provided a forum for academics to publish their results and validate them with industrial partners. Downloads from the erlang.org site kept on increasing, as did adoption.

In 2003, Thomas Arts, program manager at the IT University of Gothenburg, invites me to teach an Erlang course to his undergraduate class. Prof. John Hughes, despite already knowing Erlang, wanted to learn about it from someone who had used it in production, so he joins the class. One morning, he shows up to class tired, having pulled an all nighter. He had developed the first version of Erlang QuickCheck, which he was dutifully using to test the course exercises. That was the start of Quviq and a commercial version of QuickCheck, a property based testing tool second to none. I ended up teaching at the IT University for a decade, with over 700 students attending the course.

Getting Into Messaging

During the dot com crash, Alexey Shchepin starts work on an XMPP based instant messaging server called ejabberd. After working on it for three years, he releases version 1.0 December 1st, 2005. Facebook Chat forks it, rolling out a chat service to 70M users. At around the same time, Brian Acton and Jan Koum founded WhatsApp, also based on a fork of Ejabberd. As forking Ejabberd was all the hype, MongooseIM did the same, becoming a generic platform for large scale messaging solutions.

In May 2006, RabbitMQ is born, as we find out that work was underway to define and implement a new pub/sub messaging standard called AMQP. RabbitMQ is today the backbone of tens of thousands of systems. By the end of the decade, Erlang had become the language of choice for many messaging solutions.

The Multi-Core Years

It was not only Universities innovating during the dot com recovery. In May of 2005, a multi-core version of the BEAM VM is released by the OTP team, proving that the Erlang concurrency and programming models are ideal for future multi-core architectures. Most of the excitement was on the Erlang mailing list, as not many had realised that the free lunch was over. We took Ejabberd, and just by compiling it to the latest version of Erlang, got a 280% increase in throughput when running it on a quad-core machine.

In May 2007, the original reel of the 1991 Erlang the Movie was anonymously leaked from a VHS cassette in an Ericsson safe and put on the erlang.org site, eventually making its way to YouTube. Still no one has publically taken responsibility for this action. The world, however, finally understood the relief felt by those still under Ericsson NDA that none of the computer scientists featured in the film had given up their day jobs for a career in acting. The film got a sequel in 2013, when a hipster tries to give Erlang cool look. This time, the curpruit releasing it is identified as Chicago resident Garrett Smith.

In 2007, Programming Erlang by Joe Armstrong is published by the The Pragmatic Programmers. The following year, in June 2008, I held the first paper copy of Erlang Programming; a book Simon Thompson and I had spent 18 months writing. At the time, an O’Reilly book was the seal of approval that emerging programming languages needed, giving way to many other fantastic and diverse range of books in many languages.

The book launch party happened in conjunction with the first commercial Erlang conference, the Erlang eXchange in London June 2008. It was not the first event, as Bjarne Däcker, the former head of the Ericsson Computer Science Lab, had for almost a decade been running the yearly Erlang User Conference in Stockholm. But November in Sweden is cold, and the time had come to conquer the world. The Erlang eXchange gives way to the first Erlang Factory, taking place in March 2009 in Palo Alto (California). Much more exotic, though equally beautiful locations.

For the first time, the European Erlang community meet their American peers. We all got on like a house on fire, as you can imagine. At the conference, Tony Arcieri presents Reia, a Ruby flavoured version of Erlang running on the BEAM. Who said that a Ruby like syntax is a bad idea? Other presenters and attendees that year had stellar careers as entrepreneurs and leaders in the tech space.

An Erlang user in the US at the time was Tom Preston Werner. He was using it to scale the Ruby front-end of a social coding company called Github. In November of 2009, when in Stockholm for the Erlang User Conference, I introduced him and Scott Chacon to the OTP team. They spent an afternoon together, prompting the OTP team to move the development of Erlang to github, making it its main repository.

Conferences spread all over the world. Events have been held in Amsterdam, Bangalore, Berlin, Buenos Aires, Brussels, Chicago, (many places I can not spell in) China, Krakow, Los Angeles, Paris, Moscow, Mexico City, Milan, Munich, New York, Rome, San Francisco, St Andrews, Tel Aviv, Vancouver, Washington DC and many many other places.

The Cappuccino Years

In 2010, I teach my first graduate class at Oxford University. Erlang was picked for the Concurrency Oriented Programming course. It was also the year Bruce Tate’s Seven Languages in Seven Weeks was released. It was through this book where one of Rails’ core committers, José Valim, realized that Erlang was ahead of everyone in the concurrency race because it also tacked distribution.

In January 2011, the first commit in the Elixir repo happens. The results are presented the following year at the Krakow Erlang Factory, reaching version 1.0 in September 2014. Like all successful languages, he was trying to solve a problem, namely bringing the power of Erlang to wider communities, starting with Web.

The time was right. In January 2012, WhatsApp announce that by modifying FreeBSD and the BEAM, they achieved 2 million TCP/IP connections in a single VM and host. Their goal was to reduce operational costs, running a scalable service on a hardware footprint that was as small as possible. These results were applicable to many verticals, the Web being one of them.

The same month as the WhatsApp announcement, a group of companies pool together knowledge, time and resources to create the Industrial Erlang User Group. They worked with Ericsson to move Erlang from a derivative of the Open Source Mozilla Public License to the Apache Licence, contribute to the dirty scheduler, get bug tracking tool launched, fund a new erlang.org site, launch Erlang Central, and worked together with an aim of setting up a foundation.

Elixir Comes of Age

In July 2014, Jim Freeze organises the first Elixir Conference in Austin (Texas). There were 106 attendees, including keynote speaker Dave Thomas’ dog. Chris Mccord presented Phoenix, rising from the ashes. Robert Virding and I are part of the lineup and I recall my message loud and clear: just because you know Ruby, don’t believe them when they tell you that learning Elixir is easy. Your challenge will be thinking concurrently.

The main idea behind Elixir is concurrency, and knowing how to deal with it is critical to the success of the project. A year later, in August 2015, Phoenix 1.0 was released. It had the same effect Rails had on Ruby, bringing people to Elixir. Now, you didn’t need to master concurrency to get it! Nerves came along soon after, moving Elixir away from being a language solely for the web.

At Elixir Conf, I spoke about the book I was co-authoring with Steve Vinoski, Designing For Scalability with Erlang/OTP. At the time, it was available as a beta release. Little did I know that I had to wait for June 2016 to hold to a paper copy. The last four chapters, which should have been a book on their own, ended up taking 1.5 years to write. The main lesson to others writing a book is that if your partner tells you “you are going to become a father”, you have 8 months to finish the book. The alternative is you ending up like me, attending the release party a few days before your second child is due. The book was dedicated to Alison, Peter and our baby bump. Baby bump was born in early July, bringing truth to the Erlang saying that “you do not truly understand concurrency until you have your second child”.

The Erlang Ecosystem

Throughout 2016, Elixir adoption kept on growing. Conference talks on Lisp Flavoured Erlang and Effene - two other languages on the BEAM - revealed they had code running in production. New experimental ports kept appearing on our radar; the era of a language was over. As with .net, containing C#, F#, Visual Basic and others or the JVM ecosystem encompassing Java, Scala, Clojure, Groovy, to mention but a few. The same happened with Erlang and the BEAM, prompting Bruce Tate to coin the term Erlang Ecosystem.

Alpaca, Clojerl, Efene, Elixir, Elm-beam,, Erlog, Erlua, Fez, Joxa, Lisp Flavoured Erlang and Reia, which alongside Erlang and Elixir, are opening an era of interaction and cooperation across languages. Together, we are stronger and can continue evolving!

In December of 2018, the paperwork for the Erlang Ecosystem Foundation was submitted, setting up a non profit whose goal is to foster the ecosystem. I am looking forward to more languages on the BEAM gaining in popularity, as we improve interoperability, common tools and libraries. And as the demand for scalable and fault tolerant systems increases, so does the influence of Erlang’s constructs and semantics in the new languages within and outside the ecosystem. I hope this will set the direction for the next 20 years as a new generation of technology leaders and entrepreneurs spreading their wings.

The Future

In 2018, at Code BEAM Stockholm conference discovering the Erlang Ecosystem (formerly known as Erlang User Conference), Johan Bevemyr from Cisco announces they ship two million devices per year with Erlang applications running on them. That blew the audience away, as it meant that 90% of all internet traffic went through routers and switches controlled by Erlang. Erlang powers Ericsson’s GPRS, 3, 4G/LTE and if recent job ads are anything to go by, their 5G networks. Or IoT infrastructure through VerneMQ and EMQ, the most popular MQTT and CoAP brokers. Erlang powers not only the internet and mobile data networks, it is the backbone of tens of thousands of distributed, fault tolerant systems. Switches billions of dollars each day through its financial switches and even more messages through its messaging solutions. You can’t make this stuff up!

These are just some of my personal highlights from the last 20 years. In it all, there is a realisation that we are far from done. Joe Armstrong, in 1995, told me Erlang will not be around forever. Some day, he said, something better will come along. Fast forward to December 2018, I am still waiting, with an open mind, for that prophecy to come true. Whatever it is, there is no doubt Erlang will be a heavy influence on it.

A big thank you to Joe, Mike and Robert for making that first phone call, and to Bjarne for enabling it. A shout out to Jane who, by getting it outside of Ericsson, ensured its survival. You all started something which has allowed me to meet, work and learn with amazing and talented people using a technology that we are all passionate about. It has given us a platform enabling many of us to drive the innovation forward for the next 20 years (at least)!

Permalink

Elixir v1.8 released

Elixir v1.8 comes with many improvements at the infrastructure level, improving compilation time, speeding up common patterns, and adding features around introspection of the system.

Custom struct inspections

Elixir now provides a derivable implementation of the Inspect protocol. In a nutshell, this means it is really easy to filter data from your data structures whenever they are inspected. For example, imagine you have a user struct with security and privacy sensitive information:

defmodule User do
  defstruct [:id, :name, :age, :email, :encrypted_password]
end

By default, if you inspect a user via inspect(user), it will include all fields. This can cause fields such as :email and :encrypted_password to appear in logs, error reports, etc. You could always define a custom implementation of the Inspect protocol for such cases but Elixir v1.8 makes it simpler by allowing you to derive the Inspect protocol:

defmodule User do
  @derive {Inspect, only: [:id, :name, :age]}
  defstruct [:id, :name, :age, :email, :encrypted_password]
end

Now all user structs will be printed with all remaining fields collapsed:

#User<id: 1, name: "Jane", age: 33, ...>

You can also pass @derive {Inspect, except: [...]} in case you want to keep all fields by default and exclude only some.

Time zone database support

In Elixir v1.3, Elixir added four types, known as Calendar types, to work with dates and times: Time, Date, NaiveDateTime (without time zone), and DateTime (with time zone). Over the last versions we have added many enhancements to the Calendar types but the DateTime module always evolved at a slower pace since Elixir did not provide an API for time zone databases.

Elixir v1.8 now defines a Calendar.TimeZoneDatabase behaviour, allowing developers to bring in their own time zone databases. By defining an explicit contract for time zone behaviours, Elixir can now extend the DateTime API, adding functions such as DateTime.shift_zone/3. By default, Elixir ships with a time zone database called Calendar.UTCOnlyTimeZoneDatabase that only handles UTC.

Other Calendar related improvements include the addition of Date.day_of_year/1, Date.quarter_of_year/1, Date.year_of_era/1, and Date.day_of_era/1.

Faster compilation and other performance improvements

Due to improvements to the compiler made over the last year, Elixir v1.8 should compile code about 5% faster on average. This is yet another release where we have been able to reduce compilation times and provide a more joyful development experience to everyone.

The compiler also emits more efficient code for range checks in guards (such as x in y..z), for charlists with interpolation (such as 'foo #{bar} baz'), and when working with records via the Record module.

Finally, EEx templates got their own share of optimizations, emitting more compact code that runs faster.

Improved instrumentation and ownership with $callers

The Task module is one of the most common ways to spawn light-weight processes to perform work concurrently. Whenever you spawn a new process, Elixir annotates the parent of that process through the $ancestors key. This information can be used by instrumentation tools to track the relationship between events occurring within multiple processes. However, many times, tracking only the $ancestors is not enough.

For example, we recommend developers to always start tasks under a supervisor. This provides more visibility and allows us to control how those tasks are terminated when a node shuts down. In your code, this can be done by invoking something like: Task.Supervisor.start_child(MySupervisor, task_specification). This means that, although your code is the one who invokes the task, the actual parent of the task would be the supervisor, as the supervisor is the one spawning it. We would list the supervisor as one of the $ancestors for the task, but the relationship between your code and the task is lost.

In Elixir v1.8, we now track the relationship between your code and the task via the $callers key in the process dictionary, which aligns well with the existing $ancestors key. Therefore, assuming the Task.Supervisor call above, we have:

[your code] -- calls --> [supervisor] ---- spawns --> [task]

which means we store the following relationships:

[your code]              [supervisor] <-- ancestor -- [task]
     ^                                                  |
     |--------------------- caller ---------------------|

When a task is spawned directly from your code, without a supervisor, then the process running your code will be listed under both $ancestors and $callers.

This small feature is very powerful. It allows instrumentation and monitoring tools to better track and relate the events happening in your system. This feature can also be used by tools like the “Ecto Sandbox”. The “Ecto Sandbox” allows developers to run tests concurrently against the database, by using transactions and an ownership mechanism where each process explicitly gets a connection assigned to it. Without $callers, every time you spawned a task that queries the database, the task would not know its caller, and therefore it would be unable to know which connection was assigned to it. This often meant features that rely on tasks could not be tested concurrently. With $callers, figuring out this relationship is trivial and you have more tests using the full power of your machine.

Summing up

We are really proud of this release (as usual!) which brings many improvements at the infrastructure level. Those improvements were designed with feedback from the community and from the many different companies using Elixir in production. The full list of changes is available in our release notes.

There is only one last major feature planned for upcoming Elixir versions, which is the addition of mix release to Elixir itself, streamlining the experience provided by packages like distillery. With mix release, a developer can bundle the VM and all compiled code in a single directory, which can then be packaged and sent to production. We are glad to say the work on this feature has already started.

During my keynote at ElixirConf 2018 US, I talked about the next five years for Elixir and much of the emphasis is put on the community. Elixir was designed to be an extensible language and therefore the work on the language itself is meant to reduce with time, which we have seen in the last two releases. We trust the community to continue building on this solid foundation, bringing new challenges to the ecosystem and taking the language to new domains.

Check the Install section to get Elixir installed and read our Getting Started guide to learn more.

Your turn. :)

Permalink

ETS oddity

When working with the implementation of the new scalable ordered_set we came across a strangeness with the guarantees when iterating over a table while inserting elements in parallel.

Scenario:

> Tab = ets:new(test_table,
                [set, public, {write_concurrency, true}]).
#Ref<0.1705802953.985792516.98626>
> P1 = spawn(fun() ->
               ets:insert(Tab, {fir, 1}),
               ets:insert(Tab, {sec, 2})
             end).
> K1 = ets:first(Tab), K2 = ets:next(Tab, K1).

What are the theoretical possible values of K1 and K2? Let us first list the obvious:

  • K1 = fir, K2 = sec - both values inserted and found in term order
  • K1 = sec, K2 = fir - since this is a set, the hash algorithm may put sec before fir
  • K1 = fir, K2 = '$end_of_table' - only fir had time to be inserted
  • K1 = '$end_of_table', K2 = badarg - no elements were inserted

However it is also possible to get:

  • K1 = sec, K2 = '$end_of_table'

This was, at first, very counter-intuitive to me. How can the ets:first/1 find the second value inserted, but then when iterating not find the value inserted before it?

The answer can be found in the way that the write_concurrency functionality is implemented. Imagine we have a hash table where each bucket is protected by a mutex. When inserting a new element the mutex for the current bucket has to be taken and when iterating over the hash table we take each mutex in turn for the buckets we iterate through.

Initial Table:

Bucket # Values
1 []
2 []
3 []
4 []

Finished Table:

Bucket # Values
1 [{fir,1}]
2 []
3 []
4 [{sec,2}]

So, in the scenario that leads to the strange behaviour the following will happen:

  • ets:first/1 is called when the table is empty and iterates to Bucket #2.
Bucket # Values
1 []
2 (first) []
3 []
4 []
  • The OS does a context switch and P1 is allowed to run.
  • P1 inserts both {fir,1} and {sec,2} and then exits.
Bucket # Values
1 [{fir,a}]
2 (first) []
3 []
4 [{sec,b}]
  • The ets:first/1 call resumes and will only see sec and then '$end_of_table'.

When spelled out like this it becomes more logical that it is possible to get only the element inserted as the second element. This is not normally a problem for tables of type set which have an arbitrary iteration order that you can’t depend on anyway.

However, for ordered_set you may very well depend on the defined iteration order and expect ets:first/1 to return a key that has at least been first in the table at some point in time. But for the same reasons as with set, that is not guaranteed if you need that guarantee you have to either not use write_concurrency, find some other way to synchronize or rely on luck… these races are very rare, but in heavily used tables they will eventually happen.

The same oddity applies to all kinds of table iterations; ets:next/1, ets:select/1-3, ets:match/1-3 and friends. They may all miss concurrently inserted keys and return a key that has never existed in the table ordered directly after the previously returned key.

Permalink

Copyright © 2016, Planet Erlang. No rights reserved.
Planet Erlang is maintained by Proctor.