A journey to find those pieces of software or technology that facilitate productive and maintainable software development

Monday, June 23, 2008

Nearly defect-free software with Erlang

The fundamental flaw of both Modular and Object-Oriented Programming is that they isolate concepts, but not execution.

For the purposes of this discussion I will define program as a set of code which maintains an internal state.

It is generally accepted that developing a program of non-trivial complexity without defects is impossible. This suggests that it is generally accepted that it is possible to write trivial programs without defects.

If one were to write a program by composing trivial defect-free programs it should in principle also be defect free.

The Concurrency-Oriented Programming model of Erlang encourages the developer to build a system from the interaction of isolated processes which communicate through message-passing. A system built in Erlang isolates both concepts and execution. Erlang's exceptional-handling differs from sequential exception-handling in a very significant way: it isolates recovery effort and failure impact to dependent processes.

I believe that Erlang makes it possible to develop nearly defect-free software on a practical timeline.

Newer sequential procedural languages promised to achieve this by supporting Modular or Object-Oriented conceptual decomposition. The overall complexity of the code with access to common internal state it results in is non-trivial. A single bug in a single module or object generally takes down the entire system.

The fundamental flaw of Modular and Object-Oriented Programming in most sequential procedural languages is that they isolate concepts, but not execution. Isolation of execution is left as an exercise for the reader. Achieving reliability in a non-trivial program in these languages requires all of your developers to be well versed in shared-memory concurrency.

Shared-memory concurrent programming is manual and difficult to do correctly. It is also nearly impossible to prove the reliability of systems built in this way. Message Passing concurrent programming is similar to Object-Oriented programming in the way that the programmer separates concepts. The resulting systems have provable semantics. Message Passing is as easy in Erlang as writing a class is in Java.

I believe that retraining a team of Java developers to program in Erlang would be more cost effective than training them in the development of reliable shared-memory systems using Java's concurrency. Shared-memory concurrency is so complex that it is unlikely that the developers would be able to effectively put training into practice given their already complex conceptual workload of actual customer problems that need to be solved.

3 comments:

Martin Devlin said...

You are right to point out that this message passing concurrency model is exciting. It has great implications for utilizing multicore CPUs better.

But check out Kilim - a similar concurrency model for java.

Comparison with Erlang:
http://stephan.reposita.org/archives/2008/06/21/interesting-picture-benchmarking-erlang-versus-java-concurrency/

Main website:
http://www.malhar.net/sriram/kilim/index.html

Good webcast:
http://www.youtube.com/watch?v=37NaHRE0Sqw

Unknown said...

Good point about Kilim, Martin. I was actually discussing Kilim with my colleagues yesterday. It looks like a very good way to support isolated execution in Java and could prove instrumental to us moving to a long-sought ideal of having each object we analyze do its own analysis as an ultra-lightweight thread.

I'm checking out the Kilim webcast now. Thanks for the link.

Unknown said...

The Kilim Tech Talk was quite good. It shows a lot of potential as a way to address concurrent scalability and transparent distributed programming in Java.

I had previously read Stephan's article about the performance comparison with Erlang. It is not terribly surprising in honesty. The JVM is a very mature runtime with significant optimization work from many groups including Apple, IBM, and (of course) Sun.

I hope that Kilim gets active development support and gets adopted as part of a future Java release. I also hope this spurs competitive optimization efforts by Ericsson and the Erlang community.

SyntaxHighlighter