HOWTO: Install Erlang/OTP R14B with Homebrew on Snow Leopard with wxErlang Support

UPDATE: Hopefully this fix won't be necessary for long. I have submitted a Pull Request for the Homebrew team's consideration.

I have installed 32-bit Erlang/OTP R14B with wxErlang support successfully with modified Homebrew Formulae. It looks like it works perfectly fine so long as I don't close the Attach Process window when the debugger isn't ready for me to do that :)

I modified /usr/local/Library/Formula/wxmac.rb to enable opengl, unicode, gnomeprint and graphic_ctx and disable shared libs and install dev headers and libs resulting in:

require 'formula'

class Wxmac <Formula
  url 'http://downloads.sourceforge.net/project/wxwindows/2.8.11/wxMac-2.8.11.tar.bz2'
  homepage 'http://www.wxwidgets.org'
  md5 '8d84bfdc43838e2d2f75031f62d1864f'

  def caveats; <<-EOS.undent
    wxWidgets 2.8.x builds 32-bit only, so you probably won't be able to use it
    for other Homebrew-installed softare on Snow Leopard (like Erlang).
    EOS
  end

  def install
    # Force i386
    %w{ CFLAGS CXXFLAGS LDFLAGS OBJCFLAGS OBJCXXFLAGS }.each do |compiler_flag|
      ENV.remove compiler_flag, "-arch x86_64"
      ENV.append compiler_flag, "-arch i386"
    end

    system "./configure", "--prefix=#{prefix}", "--disable-debug",
                "--enable-unicode", "--disable-dependency-tracking",
                "--with-opengl", "--enable-unicode", "--enable-gnomeprint",
                "--enable-graphics_ctx", "--disable-shared"
    system "make && make install"
    system "cd contrib/src/stc/ && make && make install"
  end
end

I modified /usr/local/Library/Formula/erlang.rb to depend on wxmac, forced it to build in 32-bit and took out the wx skip file resulting in:

require 'formula'

class ErlangManuals <Formula
  url 'http://erlang.org/download/otp_doc_man_R14B.tar.gz'
  md5 '011530a24fbcc194be9bd01f779325a2'
end

class ErlangHeadManuals <Formula
  url 'http://erlang.org/download/otp_doc_man_R14B.tar.gz'
  md5 '011530a24fbcc194be9bd01f779325a2'
end

class Erlang <Formula
  # Download from GitHub. Much faster than official tarball.
  url "git://github.com/erlang/otp.git", :tag => "OTP_R14B"
  version 'R14B'
  homepage 'http://www.erlang.org'

  head "git://github.com/erlang/otp.git", :branch => "dev"

  depends_on 'wxmac'

  # We can't strip the beam executables or any plugins, there isn't really
  # anything else worth stripping and it takes a really, long time to run
  # `file` over everything in lib because there is almost 4000 files (and
  # really erlang guys! what's with that?! Most of them should be in share/erlang!)
  # may as well skip bin too, everything is just shell scripts
  skip_clean ['lib', 'bin']

  def options
    [
      ['--disable-hipe', "Disable building hipe; fails on various OS X systems."],
      ['--time', '"brew test --time" to include a time-consuming test.']
    ]
  end

  def install
    ENV.deparallelize
    fails_with_llvm "See http://github.com/mxcl/homebrew/issues/issue/120", :build => 2326

    # If building from GitHub, this step is required (but not for tarball downloads.)
    system "./otp_build autoconf" if File.exist? "otp_build"

    args = ["--disable-debug",
            "--prefix=#{prefix}",
            "--enable-kernel-poll",
            "--enable-threads",
            "--enable-dynamic-ssl-lib",
            "--enable-smp-support"]

    unless ARGV.include? '--disable-hipe'
      # HIPE doesn't strike me as that reliable on OS X
      # http://syntatic.wordpress.com/2008/06/12/macports-erlang-bus-error-due-to-mac-os-x-1053-update/
      # http://www.erlang.org/pipermail/erlang-patches/2008-September/000293.html
      args << '--enable-hipe'
    end

    #args << "--enable-darwin-64bit" if snow_leopard_64?

    # Force i386
    %w{ CFLAGS CXXFLAGS LDFLAGS OBJCFLAGS OBJCXXFLAGS }.each do |compiler_flag|
      ENV.remove compiler_flag, "-arch x86_64"
      ENV.append compiler_flag, "-arch i386"
    end

    system "./configure", *args
    #system "touch lib/wx/SKIP" if MACOS_VERSION >= 10.6
    system "make"
    system "make install"

    manuals = ARGV.build_head? ? ErlangHeadManuals : ErlangManuals
    manuals.new.brew { man.install Dir['man/*'] }
  end

  def test
    `erl -noshell -eval 'crypto:start().' -s init stop`

    # This test takes some time to run, but per bug #120 should finish in
    # "less than 20 minutes". It takes a few minutes on a Mac Pro (2009).
    if ARGV.include? "--time"
      `dialyzer --build_plt -r #{lib}/erlang/lib/kernel-2.14.1/ebin/`
    end
  end
end

Concise HOWTO's: Message from Java to Erlang

First create an Erlang module called server that receives messages in the format {Sender,Data} and echoes the Data portion back to the Sender. Then create a Java class called EchoClient that prompts for a node to connect to and prompts in a loop for messages to send. You will need OtpErlang.jar in your build classpath and runtime classpath. I found it in /opt/local/lib/erlang/lib/jinterface-1.5.1/priv/ on Mac OS X. I had to install Erlang from source on Ubuntu since the APT package is missing Jinterface.

Listing of server.erl:

-module(server).
-compile(export_all).

start() -> register(server,spawn(fun loop/0)).
loop() ->
    receive
        {Sender,Data} ->
            Sender ! Data,
            loop();
        shutdown -> ok
    end.

Listing of EchoClient.java

import java.io.BufferedReader;
import java.io.InputStreamReader;

import com.ericsson.otp.erlang.*;

public class EchoClient {
    public static void main(String[] args) throws Exception {
        OtpNode node = new OtpNode("java");
        OtpMbox mbox = node.createMbox("admin_gui");
        BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
        String defaultServerNodeName = "erl_node@" + node.host();
        System.out.format("Server Node to contact [%s]> ", defaultServerNodeName);
        String serverNodeName = in.readLine();
        if (serverNodeName == null || "".equals(serverNodeName)) {
            serverNodeName = defaultServerNodeName;
        }
        OtpErlangTuple serverPidTuple = new OtpErlangTuple(new OtpErlangObject[] {
                new OtpErlangAtom("server"), new OtpErlangAtom(serverNodeName)});
        while (true) {
            if (!node.ping(serverNodeName, 1000)) {
                System.out.println("Erlang node is not available: " + serverNodeName);
                System.exit(1);
            }
            System.out.print("Message (Hit Enter to send)> ");
            String message = in.readLine();
            if (message != null) {
                mbox.send("server", serverNodeName, new OtpErlangTuple(
                        new OtpErlangObject[] {
                                mbox.self(), new OtpErlangList(message)}));
                OtpErlangObject serverReply = mbox.receive(1000);
                if (serverReply == null) {
                    System.out.println("WARN: Timeout when receiving reply");
                } else {
                    System.out.format("%s replied : %s%n", serverPidTuple, serverReply);
                }
            }
        }
    }
}

Open an Erlang shell in the directory with server.erl by running erl -sname erl_node. From that shell compile and run the Erlang server module.

Listing of the Erlang shell interaction

$> erl -sname erl_node
Erlang R13B01 (erts-5.7.2) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.2  (abort with ^G)
(erl_node@alains_desktop)1> server:start().
true
(erl_node@alains_desktop)2> server ! {self(), hello}.
{<0.84.0>,hello}
(erl_node@alains_desktop)3> f(M), receive M -> {ok, M} after 100 -> {error,timeout} end.
{ok,hello}
(erl_node@alains_desktop)4>

Run the EchoClient Java program and either accept the default node or specify the name of another Erlang node you want to work with. Type messages in and see them echoed back to the EchoClient console.

Listing of the EchoClient Java program console interaction:

Server Node to contact [erl_node@alains_desktop]> [ENTER]
Message (Hit Enter to send)> Hello, Erlang![ENTER]
{server,erl_node@alains_desktop} replied : "Hello, Erlang!"
Message (Hit Enter to send)>

Concise HOWTOS: Install Erlang/OTP R13B01 from Source on Ubuntu 9.04

Generally it is better to use packages, so why have this HOWTO? Ubuntu 9.04's Erlang package is missing Jinterface, but it works when you build Erlang from source. Debian or Ubuntu will address this omission at some point. Until then we can build and install the latest Erlang release fairly easily.

The only real frustration in building from source is making sure all of the necessary build dependencies are installed first. Well, it also takes a significant chunk of time to build...

sudo apt-get install build-essential libncurses-dev \
                     unixodbc-dev libssl-dev \
                     libwxgtk2.8-dev default-jdk
curl http://erlang.org/download/otp_src_R13B01.tar.gz
tar xzvf otp_src_R13B01.tar.gz
cd otp_src_R13B01/
./configure
make
sudo make install

The make step takes a long time. It made me really appreciate the pre-built packages I can get so easily and quickly with APT.

Concise HOWTO's: Message from Erlang to Java

First create an EchoServer in Java that receives messages in the format {Sender,Data} and echoes the Data portion back to the Sender. You will need OtpErlang.jar in your build classpath and runtime classpath. I found it in /opt/local/lib/erlang/lib/jinterface-1.5.1/priv/ on Mac OS X. I had to install Erlang from source on Ubuntu since the APT package is missing Jinterface.

Listing for EchoServer.java:

 
import com.ericsson.otp.erlang.*;

public class EchoServer {
    public static void main(String[] args) throws Exception {
        OtpNode node = new OtpNode("java");
        OtpMbox mbox = node.createMbox("echo");
        OtpErlangAtom SHUTDOWN = new OtpErlangAtom("shutdown");
        while (true) {
            OtpErlangObject message = mbox.receive();
            System.out.format("%s received: %s%n", mbox.self(), message);
            if (SHUTDOWN.equals(message)) {
                System.out.format("%s shutting down...%n", mbox.self());
                break;
            } else if (message instanceof OtpErlangTuple) {
                OtpErlangTuple messageTuple = (OtpErlangTuple) message;
                if (messageTuple.arity() == 2 && messageTuple.elementAt(0) instanceof OtpErlangPid) {
                    OtpErlangPid sender = (OtpErlangPid) messageTuple.elementAt(0);
                    OtpErlangObject sendersMessage = messageTuple.elementAt(1);
                    mbox.send(sender, sendersMessage);
                }
            }
        }
    }
}

Once you have compiled and started the EchoServer you can communicate with it from the Erlang shell. First you need to start a new named node with erl -sname erl_node.

Listing of the Erlang shell interaction:

Erlang R13B01 (erts-5.7.2) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.2  (abort with ^G)
(erl_node@alains_desktop)1> net_adm:ping(java@alains_desktop).                   
pong
(erl_node@alains_desktop)2> {echo,java@alains_desktop} ! {self(),"Hello, Java!"}.
{<0.39.0>,"Hello, Java!"}
(erl_node@alains_desktop)3> % use f(M) to "forget" M (make it unbound)
(erl_node@alains_desktop)3> f(M), receive M -> {ok,M} after 100 -> {error,timeout} end.
{ok,"Hello, Java!"}
(erl_node@alains_desktop)4> {echo,java@alains_desktop} ! {self(),{tuples,can,be,sent,too}}.
{<0.39.0>,{tuples,can,be,sent,too}}
(erl_node@alains_desktop)5> f(M), receive M -> {ok,M} after 100 -> {error,timeout} end.
{ok,{tuples,can,be,sent,too}}
(erl_node@alains_desktop)6> {echo,java@alains_desktop} ! shutdown.
(erl_node@alains_desktop)6>
(erl_node@alains_desktop)7> {echo,java@alains_desktop} ! shutdown.                         
shutdown
(erl_node@alains_desktop)8> {echo,java@alains_desktop} ! {self(),"Are you there?"}.        
{<0.39.0>,"Are you there?"}
(erl_node@alains_desktop)9> f(),receive M -> {ok,M} after 100 -> {error,timeout} end.
{error,timeout}
(erl_node@alains_desktop)10>

Listing of EchoServer Java process output:

#Pid received: {#Pid,"Hello, Java!"}
#Pid received: {#Pid,{tuples,can,be,sent,too}}
#Pid received: shutdown
#Pid shutting down...

Nearly defect-free software with Erlang

The fundamental flaw of both Modular and Object-Oriented Programming is that they isolate concepts, but not execution.

For the purposes of this discussion I will define program as a set of code which maintains an internal state.

It is generally accepted that developing a program of non-trivial complexity without defects is impossible. This suggests that it is generally accepted that it is possible to write trivial programs without defects.

If one were to write a program by composing trivial defect-free programs it should in principle also be defect free.

The Concurrency-Oriented Programming model of Erlang encourages the developer to build a system from the interaction of isolated processes which communicate through message-passing. A system built in Erlang isolates both concepts and execution. Erlang's exceptional-handling differs from sequential exception-handling in a very significant way: it isolates recovery effort and failure impact to dependent processes.

I believe that Erlang makes it possible to develop nearly defect-free software on a practical timeline.

Newer sequential procedural languages promised to achieve this by supporting Modular or Object-Oriented conceptual decomposition. The overall complexity of the code with access to common internal state it results in is non-trivial. A single bug in a single module or object generally takes down the entire system.

The fundamental flaw of Modular and Object-Oriented Programming in most sequential procedural languages is that they isolate concepts, but not execution. Isolation of execution is left as an exercise for the reader. Achieving reliability in a non-trivial program in these languages requires all of your developers to be well versed in shared-memory concurrency.

Shared-memory concurrent programming is manual and difficult to do correctly. It is also nearly impossible to prove the reliability of systems built in this way. Message Passing concurrent programming is similar to Object-Oriented programming in the way that the programmer separates concepts. The resulting systems have provable semantics. Message Passing is as easy in Erlang as writing a class is in Java.

I believe that retraining a team of Java developers to program in Erlang would be more cost effective than training them in the development of reliable shared-memory systems using Java's concurrency. Shared-memory concurrency is so complex that it is unlikely that the developers would be able to effectively put training into practice given their already complex conceptual workload of actual customer problems that need to be solved.

Mercurial SVN Integration Using Erlang

I wanted to make it possible to keep a Mercurial (Hg) repository in sync with a series of Subversion (SVN) source repositories. Due to the way certain Subversion repositories are structured it is not always convenient to have a common Mercurial and Subversion root. I am actually using Mercurial to coalesce a collection of related modules into a single checkout for an agile development team. Consequently, the Python hgsvn scripts are unworkable for the scenario I am in. So I rolled my own hgsvn in Erlang and it was surprisingly easy to do.

The Erlang hgsvn module follows this process:

1. Get the current revision from svn info
2. Get all future revisions with changes from svn log -rBASE:HEAD
3. Parse out the Author, Date, and Comment from svn log
4. Update to the next revision using svn up -r #
5. Add any new files added and remove any file deleted by that revision to Mercurial using hg addremove
6. Commit the revision to Mercurial as that Author on that Date with that Comment
7. Repeat from (2) until no more revisions are available

Using Erlang for this made the code very straight-forward. I previously attempted this in Groovy and beyond failing to get it working I found the code incredibly difficult to understand without detailed comments.

To run this code you would invoke the following command in the root of a Subversion working copy in your Mercurial repository:

erl -pa <path to hgsvn.beam parent dir> -noshell -s hgsvn -s init stop

You can also specify a set of related repositories (ex: from a common server) and update them together in revision order:

erl -pa <path to hgsvn.beam parent dir> -noshell -s hgsvn -s init stop -repo_set path/to/repo1 path/to/repo2

You can even specify multiple sets of related repositories and update them set by set in revision order:

erl -pa <path to hgsvn.beam parent dir> -noshell -s hgsvn -s init stop -repo_set path/to/repo1 path/to/repo2 -repo_set path/to/repo3 -repo_set path/to/repo4

Further to that you can specify stop revisions for each repository in a set:

erl -pa <path to hgsvn.beam parent dir> -noshell -s hgsvn -s init stop -repo_set path/to/repo1 path/to/repo2 -stop_set 1023 1432

If you specify a stop_set for one repo_set you must provide one for every repo_set. It is perfectly workable to put nothing after the -repo_set as hgsvn will interpret that to mean HEAD for every repository in the repo_set.

You can have an unlimited number of repository sets and an unlimited number of Subversion paths per repository set.

This code was developed to work with the output format from Subversion 1.4.4 (r25188) on Mac OS X 10.5 and Subversion 1.4.6 (r28521) on Ubuntu Hardy Heron. It should work with any 1.4.x Subversion though. Make sure to post any issues here as comments (please include the output of svn info).

UPDATE (2008/07/15): now handles case where one or more SVN WCs have no new changes

UPDATE (2008/07/15): adds SVN watcher that automatically collects changes (ex: -watch 30min)

UPDATE (2008/08/01): this is now erlang_hgsvn on GitHub

Trying Out Functional Programming - Episode 5

I suppose I should probably stop referring to this series as Trying Out Functional Programming since Erlang is not a pure functional programming language, but rather a declarative concurrent programming language.

The semantics aside, Erlang lets you model a domain as a set of interacting processes. This is a fairly natural - at least for me - way to model most problems. For example, you take the popular board game Settlers of Catan. It is composed of a board with terrain hexes, intersections, and paths. I modeled each hex, intersection, and path as a separate process whose current state drives the determination of rules in a distributed rather than centralized manner.

For example, building a settlement on an intersection is allowed only if no neighbouring intersection has been built on. Instead of checking the neighbouring intersections on each request the neighbouring intersections notify it when they get built upon. As a result the intersection changes it's internal state (the function it is executing) to blocked. As a result it no longer responds to build requests, making it completely impossible to build on it.

Another nicety of the process-oriented or Actor-model approach that Erlang facilitates is the lack of interference from buggy code elsewhere in the system. The intersection in question cannot be coerced into accepting the construction of a settlement. This is due to the fact that the buggy code would have to send a message to change the intersection's state. Thankfully the intersection process is coded to disregard such requests.

This is a very early work in progress and is mostly a mental exercise at the moment, but the possibilities opened up by Erlang's approach are exciting...

Trying Out Functional Programming - Episode 4

My bad experience with Haskell drove me to conclude (prematurely) that there was little I could ever hope to achieve using the functional programming style. It turns out that I was wrong. Erlang actually makes practical programming possible in a functional style.

Erlang is backed by years of success for Ericsson and demonstrates the value of focussing on productivity for developing real systems rather than conforming to a philosophical ideal as Haskell does. This gives Haskell many merits, but they are mostly irrelevant to me. On to learning Erlang…

Erlang is in active use primarily in telecoms where I/O is a obviously a major concern. Haskell eschews side-effects, which makes I/O FUBAR. Haskell's reasons for this are perfectly sound, but they also make Haskell entirely inappropriate for my purposes.

The tutorials and video presentations about Erlang are very practically focussed, showing how it can be used to switch telephone calls and create servers that respond to inbound requests. Haskell tutorials (in my experience) focus on language semantics (like its type system) and focus on examples of modeling mathematical expressions. Its my fault for not researching the languages properly. It would have been immediately apparent that Haskell was an inappropriate language for implementing RESTful servers.

Stay tuned for Episode 5…