DIGG IT! I decided in November to learn Erlang as a new programming language and over the recent holidays I got 3 weeks to play around with it. In my 2008 RIA trends post I listed Erlang as a language trend and thought it would be good to shed some light on the matter.
Erlang - a general-purpose concurrent programming language and runtime system.
The language is very different from any programming language I have worked with and was created by Ericsson to write highly scalable mission critical telecom server products. There are several advantages to Erlang as a language that make it optimal for writing server based applications:
1. Concurrent Processes - Erlang can spawn many processes, like 250,000 processes and these run concurrently in the VM. Each process can receive messages and send messages to other processes. The VM automatically handles executing processes across multiple processors so having more processes generally runs N times faster on N core machines. In a world where cpu cores are multiplying, Erlang has a huge advantage in leveraging additional CPUs without any programming effort.
2. Message Based - Erlang processes communicate by sending messages and the language syntax is optimized to make handling messages easy. The arrity and type of data passed to a function determine which version of the function is called so message handling is done at the interface of the process not in some if/then/else/case statement. The message either matches a function or it is ignored. The way messages and processes are handled is much like the real world, we do not have shared state, rather we send messages to one another by honking, blinking, speaking or waving.
3. No shared state and Variables that do not vary - The strangest truth in Erlang is that variables do not vary. This allows apps to avoid shared state and read/write contention and locks. The rule is that where there are locks, there are keys to loose so Erlang has neither. It seems annoying but in fact it only takes a little while to get used to. In avoiding shared state, processes are much more scalable and issues of thread contention, race conditions, and collisions essentially fall onto the floor.
4. Bit Masking and Pattern Matching - The feature I like most is pattern matching and the bit masking syntax. It is trivial to parse extremely complex binary data and set variables from input data. Here is a little snippet:
<<?Header:4, Chan:4, Note:8, _:8>> = InBytes.
Basically InBytes is an array of bytes and the variables get set to the left. In this case, 4 bytes are read into Header, Chan and 8 bytes into Note. What seems odd is that equals assigns values throught the pattern on the left to become equivalent to the pattern on the right. It looked alien to me at first but once you get it parsing binary data is never the same.
5. Hot Swapping Code - Imagine swapping out server logic within a mission critical server at runtime? Erlang provides a framework called OTP for writing server behavior and swaping the behavior live even on remote Erlang VMs. You have to see it to fully appreciate what is happening and the syntax and patterns for writing the logic are really simple compared to procedural or object oriented languages.
6. RPC and Distributed Computing - Running processes in local and remote Erlange VM's could not be easier. Once the machines are physically allocated by IP you can spawn processes on remote VMs and call remote processes as if they were local. It makes writing a cluster of servers really simple and once the different VMs are trusted on a network exchanging messages is nearly identical to local messaging between processes.
Learning Erlang has already been one of the more interesting explorations in my software career and it is similar to my first experience with Python back in 1999. If you are interested in exploring large scale distributed computing, Erlang is the language and VM in use at both Google and Amazon in a large scale. I think we will see much more from this programming language in 2008 as the services market heats up. I will also be releasing some services written in Erlang for Flex later this year.
Cheers,
Ted :)
Ted,
We are in the process of re-architecting our middle tier from Java/JMS over to Erlang and RabbitMQ. If you compare the learning curve of Erlang with the time and cost of finding and hiring talented multi-threaded Java programmers + writing and debugging multithreaded code, the reality is that the Erlang approach is cheaper! Likewise, since it is a language that has been around for decades and proven in the telco industry, it isn't so much a "gee, can this Erlang stuff work" as it is simply getting used to functional programming and pattern matching. One interesting (intentional) design of Erlang is the whole list + recursion calls. While most of us have made some recursive calls here and there, Erlang lends itself to using recursion on a daily basis with its list processing.
RabbitMQ has a Java client library, so it is very convenient to both publish and subscribe to RabbitMQ from Java (e.g. dropping a message in from a Servlet). And since RabbitMQ is written in Erlang itself, it too scales incredibly well, giving us what we were looking for-- the ability to more easily manage and monitor hundreds of what were Java threads spread across many servers.
For those seriously interested in Erlang, be sure to pickup a copy of Programming Erlang by Joe Armstrong (ISBN 1-934356-00-X).
Hello Ted,
At ProcessOne, we have been developing a lot in Flex lately (And Erlang, as obviously we are an Erlang "shop").
If you need advice or simply want to share thought, you can contact me privately.
Have fun !
Hi Ted, I've been following ErlLang for a couple years now, they have some speedy db (Mnesia) and even a web-framework (ErlyWeb). I've often wanted to start tinkering in it, but never really had the opportunity. I'll be interested to see how support for some Flex stuff pans out of this. I really enjoyed seeing your AirChat app with Python Twisted-Matrix on the backend.
Being a Python fan, though, one of the things I don't like about ErlLang is that someone new to the language can't just look at it and understand mostly what the code does. Afterall, it's hard enough to find Flex programmers, even harder to find Python programmers, and ErlLang...
I'm also kind-of interested in what Adobe's stance is with you guys playing with these other alternative technologies, because from my standpoint, I'm still trying to figure out Adobe's business model for making money on Flex/Flash RIA stuff because it interops very well with everything and I can get Flex Builder so dang cheap, but have never used any of the Adobe server products. I'm not really sure what going the route of FDS buys me over using a LAMP or J2EE (via Spring) stack buys me on the backend of things. Especially now that you guys opened up AMF.
Mickael, Thanks I need to tap into some Erlang know how. What is interesting is that is it very similar to Python Twisted in terms of events and deferred although in Erlang these are built into the core via messages/processes.
I have a very small portion of the AMF3 spec running covering String/Int/Number/Object/Array. My apps never use more than that in the protocol plus in Erlang the library is only 30 lines long which is way cool.
I will contact you and thanks for the offer.
Ted :)
Todd,
Adobe's stance is that there are many different types of developers. Flex can seamlessly integrate with just about any programming language on the server side. I also like to keep an open mind and exploring new languages is a hobby. It helps me understand another developers perspective and alternative techniques. In regards to Erlang, I think the language is going to take off for services development, it has at Amazon and Jabber. To see these two examples and what they have accomplished is really stunning to me. Then learning the language it really clicked for me in terms of a language for concurrency.
In regards to FMS, it is about deep integration with Java. RPC is free now and the higher level messaging and data management is where that product shines. There is a class of Enterprise where FDS/LCDS makes a ton of sense. Actually after you have written crud one time, it hits you how much money and time LCDS can save you and allows you to use message based programming too. Writing code to syncronize 50 clients with conflict resolution is hard and this is just a config file in LCDS. Although there are many backends to integrate with Java isn't the only one out there.
My 2 cents,
Ted :)
aie! I had just put learning erlang aside on the basis that it was pure intellectual with little practical application in my programming life (= I can't see how to make money from it).
So many interesting languages, so little time.
Now you've got me excited about erlang again and I'm looking forward to what you come up with for flex + erlang.
Ted,
I have developed a proof-of-concept AMQP client for AS3 and tested it with Erlang AMQP clients using RabbitMQ. It is a relatively simple way to get Flex and Erlang to talk together without having to do something proprietary.
Ben