2008-02-02

My first look at Clojure

Preliminaries

So, Clojure is a pretty little language, a Lisp1 for the Java Virtual Machine. That, in and of itself, doesn't say much for or against it.

I've been playing with it for a few days now and really finding that I like more than I dislike and that the bits that I found to be difficult were, well, being fixed or changed.2

In the spirit of Dive Into Python, this is the place where I should offer up a snippet of finished code and then walk through it. Unfortunately, only having used Clojure a few days, I'm still in the first romance stage, rather than the stacks of working code stage.

So, rather than offering a block of code and then dissecting it, or alternately doing the thing that seems common in ruminations on functional languages3 and offering up an example like:

(defn fibs
  ([] (fibs 0 1))
  ([a] (fibs a 1))
  ([a b]
   (lazy-cons a (fibs b (+ a b)))))

(defn nthfib [n]
  (first (reverse (take n (fibs)))))

That, while pretty in demonstrating infinite sequences and a simple form of multiple dispatch, does little to offer a reason to actually use the language4. Instead, I'm going to try my hand at two things that have frustrated me in the past with Lisp. Namely sockets and threads5.

Goals

So, what I want is:

  • A client that:

    1. Connects to a server.

    2. Listens and prints out what it hears.

  • A server that:

    1. Listens on a port

    2. Creates a new thread on connection.

    3. Sends the current ctime to the client.

    4. Disconnects.

Or, essentially, the most primitive version of NTP imaginable.

As Clojure runs on the JVM, first step becomes to open up JavaDoc and check what objects I actually want. I'll start by importing ServerSocket and Socket.

user=> (import '(java.net ServerSocket Socket))
nil

The most primitive client and server possible

At this point, all I want to do is make functions to open the connections, so:

(defn listener-new [port]
  (new ServerSocket port))

(defn connection-new
  ([port]
   (connection-new "127.0.0.1" port))
  ([address port]
   (new Socket address port)))

listener-new is a function that binds on a port and returns a Java ServerSocket bound to that port. connection-new is a function that takes either a port (in which case it connects to localhost) or an address and a port and returns a Socket connected to that.

I'll try to explain new terms and forms as I go. The ones introduced here are defn, which creates a function, with the arguments in square brackets, returning whatever the last value in its body is.

So, (defn [port] (new ServerSocket port)) makes a function that takes one argument and then calls the new function, which instantiates a Java object with whatever arguments are passed. It then returns whatever new returns, which, unless there's an error, should be a ServerSocket.

defn can also accept multiple sets of arguments, in order to either do a certain amount of pattern matching on them or, in this case, provide a default value to a function.

The next step, then, is to try each of these and see if they work.

user=> (def serversocket (listener-new 56894))
#<Var: user/serversocket>
user=> serversocket
ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=56894]
user=> (. serversocket (close))
Reflection warning, line: 17 - call to close can't be resolved.
nil

So, the serversocket will open just fine, though I receive a (relatively harmless) type warning when I close it. That can be removed either by ensuring that the close method call ensures (via a type annotation) that what is passed to it is indeed a serversocket and that what is returned by listener-new is also a serversocket.

For the present, though, I don't actually care about this warning. Let's try connection-new.

user=> (def socket (connection-new 56894))
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
        at java.lang.reflect.Constructor.newInstance(Unknown Source)
        at clojure.lang.Reflector.invokeConstructor(Reflector.java:125)
        at user.connection_new.invoke(Unknown Source)
        at user.connection_new.invoke(Unknown Source)
        at clojure.lang.AFn.applyToHelper(AFn.java:173)
        at clojure.lang.AFn.applyTo(AFn.java:164)
        at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213)
        at clojure.lang.Compiler$DefExpr.eval(Compiler.java:253)
        at clojure.lang.Compiler.eval(Compiler.java:3086)
        at clojure.lang.Repl.main(Repl.java:59)
Caused by: java.net.ConnectException: Connection refused: connect
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(Unknown Source)
        at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
        at java.net.PlainSocketImpl.connect(Unknown Source)
        at java.net.SocksSocketImpl.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at java.net.Socket.<init>(Unknown Source)
        at java.net.Socket.<init>(Unknown Source)
        ... 13 more

Oh dear. Well, that didn't work. Though, I suppose having a port open to connect to might just help.

user=> (def ssocket (listener-new 56894))
#<Var: user/ssocket>
user=> (def socket (connection-new 56894))
#<Var: user/socket>
user=> (. socket close)
Reflection warning, line: 45 - reference to field close can't be resolved.
java.lang.IllegalArgumentException: No matching field found
        at clojure.lang.Reflector.getInstanceField(Reflector.java:175)
        at clojure.lang.Compiler$InstanceFieldExpr.eval(Compiler.java:744)
        at clojure.lang.Compiler.eval(Compiler.java:3086)
        at clojure.lang.Repl.main(Repl.java:59)
user=> (. socket (close))
Reflection warning, line: 53 - call to close can't be resolved.
nil
user=> (. ssocket (close))
Reflection warning, line: 54 - call to close can't be resolved.
nil

And that does it. The error in that block was because I referred to close as a field: close, rather than a method: (close). On the other hand, this doesn't do anything interesting. It just makes and closes the listener and the connection.

Now, the . operator. I've used it a few times so far, so I should clarify what it does. It looks up the second value within the first one. Thus, (. socket (close)) looks for a close method inside the instance socket, then calls it.

Next, I need to add behaviour to the server to detect connection and send the ctime, as well as making the client recognise that data is being passed to it.

Adding data

(import '(java.net ServerSocket Socket)
        '(java.util Date))

(defn current-time []
  (. (new Date) (toString)))

(defn listener-new [port]
  (new ServerSocket port))

(defn listener-wait [listener]
  (. listener (accept)))

(defn listener-send [lsocket]
  (.. lsocket (getOutputStream) (write (current-time)))
  lsocket)

(defn listener-close [listener]
  (. listener (close)))

(defn listener-run [port]
  (let [listener (listener-new port)]
    (listener-send (listener-wait listener))
    (listener-close)))

So now I've added a function that dumps the current time as a string, as well as functions to accept a connection, send that current time and disconnect.

Trying it out, I get:

user=> (current-time)
"Sat Feb 02 12:16:19 EST 2008"
user=> (listener-run 51245)

And then it hangs. Unsurprisingly, really, as it's blocking at (. listener (accept)), which waits for a connection before returning the socket referring to that connection. While this would be a good place to make the thread that I was talking about earlier, for the moment, I just want to see if this actually works. So, what I'll do is start the server in a separate REPL6.

So, in the one REPL, I create the server again:

user=> (listener-run 51345)

And in the other, I connect to it:

user=> (connection-new 51345)
Socket[addr=/127.0.0.1,port=51345,localport=1668]

Which results in this in the first REPL.

java.lang.IllegalArgumentException: No matching method found: write
        at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:59)
        at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:26)
        at user.listener_send.invoke(sockets.clj:15)
        at user.listener_run.invoke(sockets.clj:23)
        at clojure.lang.AFn.applyToHelper(AFn.java:173)
        at clojure.lang.AFn.applyTo(AFn.java:164)
        at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213)
        at clojure.lang.Compiler.eval(Compiler.java:3086)
        at clojure.lang.Repl.main(Repl.java:59)

Debugging data transmission

Now, despite this being an error, it does show that the listener made it to listener-send, as that's where write is called. So, I know that it's accepting network connections. Unfortunately, it's crashing when it receives them.

So, my next step is to look at what that command actually does:

user=> (macroexpand '(.. lsocket (getOutputStream) (write (current-time))))
(. (. lsocket (getOutputStream)) (write (current-time)))

The macroexpand function takes a macro passed to it and evaluates it (as well as macros inside that one, recursively). So I know that write is being called on the outputstream of the socket. For debugging purposes, I'll rewrite listener-send procedurally.

(defn listener-send [lsocket]
  (println lsocket)
  (let [outputstream (. lsocket (getOutputStream))]
    (println outputstream (current-time))
    (. outputstream (write (current-time)))
    lsocket))

This is the same function, only it's spitting out its state at each stage.

user=> (listener-run 51345)
java.lang.IllegalArgumentException: No matching method found: write
        at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:59)
        at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:26)
        at user.listener_send.invoke(sockets.clj:18)
        at user.listener_run.invoke(sockets.clj:26)
        at clojure.lang.AFn.applyToHelper(AFn.java:173)
        at clojure.lang.AFn.applyTo(AFn.java:164)
        at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213)
        at clojure.lang.Compiler.eval(Compiler.java:3086)
        at clojure.lang.Repl.main(Repl.java:59)
Socket[addr=/127.0.0.1,port=1789,localport=51345]
java.net.SocketOutputStream@26dbec Sat Feb 02 13:08:22 EST 2008

This time, with output, I can see that I indeed get an OutputStream. Time to play with it and see what it's doing.

user=> (import '(java.io InputStream OutputStream)
nil
user=> (def li (listener-new 51345))
#<Var: user/li>
user=> (def sock (listener-wait li))
#<Var: user/sock>
user=> (def by (. (current-time) (getBytes)))
Reflection warning, line: 46 - call to getBytes can't be resolved.
#<Var: user/by>
user=> (map (comp char (appl aget by)) (range (alength by)))
(\S \a \t \space \F \e \b \space \0 \2 \space \1 \3 \: \4 \9 \: \4 \4 \space \E \S \T \space \2 \0 \0 \8)
user=> (def ostream (. sock (getOutputStream)))
Reflection warning, line: 48 - call to getOutputStream can't be resolved.
#<Var: user/ostream>
user=> (. ostream (write by))
Reflection warning, line: 49 - call to write can't be resolved.
nil
user=> (listener-close li)
nil

So, there's the catch. When I pass current-time directly to the OutputStream's write method, what's expected is a write( String s ). What actually exists is write( Byte[] barr ).

The map done on by is worth looking at, though. I used it to find out what exactly by was set to, it being a Java array. So, first off, I get a list of all numbers from 0 to the length of by: (range (alength by)). That's the indices to the array. Then (comp char (appl aget by))7 is called on each of those.

Converting to and from byte arrays.

This actually raises a valid assertion. When creating a byte array from a string, the character values in the byte array should be the same as those in the string.

So, I'll create a function (that I'll use in a minute anyway) that makes a Java byte array out of a string. And, because I'm starting to get more cautious after my repeated bugs, I'll make a simple test to assert that its results are the same as the string from which it was made.

(defn byte-arr-from-string [str]
  (. str (getBytes)))

(defn test-byte-array-from-string
  ([]
   (test-byte-array-from-string (current-time)))
  ([str]
   (let [barr (byte-arr-from-string str)
         bseq (map (comp char (appl aget barr))
                   (range (alength barr)))
         chseq (map char str)]
     (and (== (alength barr)
              (count bseq)
              (count chseq))
          (== 0
              (count (filter false?
                             (map eql?
                                  bseq
                                  chseq))))))))

There. A String to Byte[] converter and a unit test. Now, to run the test:

user=> (test-byte-array-from-string)
true
user=> (test-byte-array-from-string "qwertyuiop")
true

And the test indicates that there is no loss per se in converting the string to a byte array. That done, I'll add the conversion to a byte array into listener-send. If that works, I'll collapse the expanded debuggy listener-send back down.

(defn listener-send [lsocket]
  (println lsocket)
  (let [outputstream (. lsocket (getOutputStream))]
    (println outputstream
             (current-time)
             (instance? outputstream java.io.OutputStream))
    (. outputstream (write (byte-arr-from-string (current-time))))
    lsocket))

And now a further round of testing:

user=> (listener-run 51345)
Socket[addr=/127.0.0.1,port=2564,localport=51345]
java.net.SocketOutputStream@88e2dd Sat Feb 02 15:23:04 EST 2008 true
nil

So, the server says (more or less) that it did its steps and nothing went wrong. Let's check this from the other side:

user=> (def conn (connection-new 51345))
#<Var: user/conn>
user=> (def ins (. conn (getInputStream)))
Reflection warning, line: 69 - call to getInputStream can't be resolved.
#<Var: user/ins>
user=> ins
java.net.SocketInputStream@453807
user=> (. ins (read))
Reflection warning, line: 71 - call to read can't be resolved.
83
user=> (. ins (read))
Reflection warning, line: 72 - call to read can't be resolved.
97

And, at the other end, I get the bytes that (I think) I sent. Time to wrap up my client connection and get it to read the entire input stream.

Reading the transmitted data

(defn string-from-byte-sequence [coll]
  (reduce strcat coll))

(defn connection-new
  ([port]
   (connection-new "127.0.0.1" port))
  ([address port]
   (new Socket address port)))

(defn connection-read [conn]
  (let [instream (. conn (getInputStream))]
    (loop [bytes nil
           current-byte (. instream (read))]
      (if (== current-byte -1)
        bytes
        (recur (conj bytes current-byte)
               (. instream (read)))))))

(defn connection-close [conn]
  (. conn (close)))

(defn connection-run [port]
  (let [conn (connection-new port)
        str (string-from-byte-sequence (connection-read conn))]
    (connection-close conn)
    str))

So, what this is trying to do is open a connection, read from it until the number -1 is found and then returning the resulting sequence of bytes.

Why -1? Because the JavaDoc for InputStream::Read) says:

The value byte is returned as an int in the range 0 to 255. If no byte is available because the end of the stream has been reached, the value -1 is returned. This method blocks until input data is available, the end of the stream is detected, or an exception is thrown.

So I'm trying -1 first, as by the time I'm reading, listener-close should have been called, closing the socket.

Bugs with reading the data

user=> (connection-run 51348)
"5648485032848369325048584949585449325048329810170321169783"

So, it reads fine, but it appears that something went wrong in translating the byte sequence into a string again. Looking at string-from-byte-sequence, it concatenates the string values of the items in the collection.

(defn string-from-byte-sequence [coll]
  (reduce strcat coll))

So, I'll try concatenating two numbers:

user=> (strcat 64 65)
"6465"

Right, not what I was intending at all. So, before I concatenate the numbers, I'll need to convert them to characters.

user=> (strcat (char 64) (char 65))
"@A"

That worked a lot better. And tells me what I need to do to fix the function:

(defn string-from-byte-sequence [coll]
  (reduce strcat
          (map char coll)))

Because map is lazy, it means that simply using (strcat (map char coll)) gets me the string value of map's returned FnSeq instead of the concatenated characters. The reduce acts to apply strcat to every element of the entire sequence.

And, connecting to a new server, I get:

user=> (connection-run 51349)
"8002 TSE 15:13:61 20 beF taS"

Whoops, almost it! Only backwards. Checking the byte sequence as it comes in, I find that it is indeed reversed when returned from connection-read. So, I test what connection-read does to build the byte sequence:

user=> (conj (conj nil 1) 2)
(2 1)

So I check the documentation for conj and indeed:

Conj[oin]. Returns a new collection with the item 'added'. (conj nil item) returns (item). The 'addition' may happen at different 'places' depending on the concrete type.

An amendment to connection-read later:

(defn connection-read [conn]
  (let [instream (. conn (getInputStream))]
    (loop [bytes nil
           current-byte (. instream (read))]
      (if (== current-byte -1)
        bytes
        (recur (concat bytes (list current-byte))
               (. instream (read)))))))

And I finally get the desired result:

user=> (connection-run 51354)
"Sat Feb 02 16:44:09 EST 2008"

Bugfixing the primitive NTP server

So, what I now have is a server that waits for a connection on a given port, then sends the time and a client that makes a connection and reads the time. What I don't have is a bug-free or elegant version of this.

Time for me to run down the list of bugs and desires that I know of at the moment:

  • The server fails to clean up its port correctly, meaning that throughout this testing, I've been incrementing ports any time I choose to keep the server's REPL open.

  • The client will hang, waiting for data in its stream, when fed a port that is open but not responding.

  • The server is single-threaded still. This is the only one of my six test-application goals left undone. But making it not block the REPL while waiting for input will make testing far easier.

  • No Java exceptions are currently handled, despite their being thrown.

  • Ten reflection warnings when I load the file into my REPL.

  • test-byte-array-from-string is the only test attached. The behaviour of all the functions should be testable.

  • It would be nice (though not useful in this case) to represent the data stream as a lazy sequence, rather than expecting that it all be here.

So, I'll address these in order, starting with the server port.

The server fails to clean up its port...

I'll start by looking at what listener-run actually does.

(defn listener-run [port]
  (let [listener (listener-new port)]
    (listener-close (listener-send (listener-wait listener)))
    listener))
  1. It binds listener to a new ServerSocket listening on port.

  2. It passes that ServerSocket to listener-wait, which calls the accept method, returning the opened socket.

  3. That socket is passed to listener-send, which gets its OutputStream and writes the current time to the stream, returning the socket.

  4. That socket is passed on to listener-close, which closes it.

  5. The listener is returned.

And thatfourth step would likely be the problem here. Rather than closing the listener, it's closing the socket acquired from accept, which means that rather than five, I only have four goals complete.

So, I'll try:

(defn listener-run [port]
  (let [listener (listener-new port)]
    (listener-send (listener-wait listener))
    (listener-close listener)
    listener))

But that has the net result that now, the server still holds onto that address, but the client hangs, waiting for data. A little bit of prodding later and I have this:

(defn listener-send [lsocket] 
  (.. lsocket
      (getOutputStream)
      (write (byte-arr-from-string (current-time))))
  (.. lsocket (getOutputStream) (close))
  lsocket)

(defn listener-run [port]
  (let [listener (listener-new port)]
    (. (listener-send (listener-wait listener)) (close))
    (listener-close listener)
    listener))

Which successfully sends the time across the (loopback) network, then releases the port.

The client will hang...

Not exactly true. I wrote that forgetting that I had random ServerSockets poking around. In fact, what happens when the client is passed a closed port is:

user=> (connection-new 1500)
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
        at java.lang.reflect.Constructor.newInstance(Unknown Source)
        at clojure.lang.Reflector.invokeConstructor(Reflector.java:125)
        at user.connection_new.invoke(Unknown Source)
        at user.connection_new.invoke(Unknown Source)
        at user.connection_run.invoke(Unknown Source)
        at clojure.lang.AFn.applyToHelper(AFn.java:173)
        at clojure.lang.AFn.applyTo(AFn.java:164)
        at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213)
        at clojure.lang.Compiler.eval(Compiler.java:3086)
        at clojure.lang.Repl.main(Repl.java:59)
Caused by: java.net.ConnectException: Connection refused: connect
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(Unknown Source)
        at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
        at java.net.PlainSocketImpl.connect(Unknown Source)
        at java.net.SocksSocketImpl.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at java.net.Socket.<init>(Unknown Source)
        at java.net.Socket.<init>(Unknown Source)
        ... 13 more

Conclusion: I need to not only handle connection timeouts but also refused connections. As this happens in connection-new, which only has one instruction, I need to wrap the new socket in a try/catch block.

(defn connection-new
  ([port]
   (connection-new "127.0.0.1" port))
  ([address port]
   (try (new Socket address port)
        (catch InvocationTargetException except
               (println (strcat "Could not connect to "
                                address
                                " on port "
                                port))))))

(defn connection-run [port]
  (let [conn (connection-new port)
        str (when conn (string-from-byte-sequence (connection-read conn)))]
    (when conn 
      (connection-close conn))
    str))

So now trying to connect to an invalid port spits an error message and returns nil. On the other hand, it still hangs when pushed to connect to a port that doesn't behave like it expects (send data, close socket).

user=> (connection-new 1280)
Could not connect to 127.0.0.1 on port 1280
nil

The answer, then, is to give the client a timeout after which it assumes that no data is coming.

(defn connection-new
  ([port]
   (connection-new "127.0.0.1" port))
  ([address port]
   (try (doto (new Socket address port)
          (setSoTimeout 5000))
        (catch InvocationTargetException except
               (println (strcat "Could not connect to "
                                address
                                " on port "
                                port))))))

(defn connection-read [conn]
  (let [instream (. conn (getInputStream))
        reader (fn [] (try (. instream (read))
                           (catch InvocationTargetException except
                                  -1)))]
    (loop [bytes nil
           current-byte (reader)]
      (if (== current-byte -1)
        bytes
        (recur (concat bytes (list current-byte))
               (reader))))))

So now, if nothing comes in in five seconds, connection-read signals an error and flags the current byte as -1, signalling an end of the transmission.

doto, which I used to set the socket timeout, takes an instance, evaluates the following body on it, then returns the changed instance. I used it in this case because setSoTimeout returns void while changing the object's state.

user=> (connection-run 80)
""

The server is single-threaded...

Oh bugger. I really should have put this lower on the list. I did not want to deal with threading today.

So, that leaves me the question: What should the server do? For starters, I'll make its instances persistant. That way, with a single server open, I can connect repeatedly without restarting the server each time.

(defn listener-run
  ([port]
   (listener-run (listener-new port)
                 port))
  ([listener port]
   (loop [socket nil]
     (when socket
       (. (listener-send socket) (close)))
     (recur (listener-wait listener)))
   listener))

However, what I want is for that same function to be pushed into the background. So, for starters, I'll give the loop an exit condition:

(defn listener-run [listener port]
  (loop [socket nil]
    (if (. listener (isClosed))
      listener
      (do (when socket
            (. (listener-send socket) (close)))
          (recur (listener-wait listener))))))

Now, if I happen to have captured the listener, I can kill the server process by closing the listener. This isn't useful yet, as the server will still loop infinitely when it's launched.

To make sure that I'm not holding on to inaccessable listeners, I also removed my earlier convenience form that created a listener for me.

So, time to push listener-run into its own execution thread.

(defn listener-run-in-background [port]
  (let [listener (listener-new port)]
    (listener-run listener port)
    listener))

There's a starting form. Though all it does at the moment is run the listener in the same way as the bits I removed above, I'll now poke at adding Java's concurrency bits into it.

(defn listener-run-in-background [port]
  (let [listener (listener-new port)
        exec (. Executors (newSingleThreadExecutor))
        run (appl listener-run listener port)]
    (. exec (submit run))
    listener))

So, that's a first attempt. It creates a pool of one thread, wraps running the server in an anonymous function and punts the function off to the thread pool.

And testing it:

user=> (def li (listener-run-in-background 51345))
#<Var: user/li>
user=> li
ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=51345]
user=> (connection-run 51345)
"Sat Feb 02 20:49:15 EST 2008"
user=> (connection-run 51345)
"Sat Feb 02 20:49:16 EST 2008"
user=> (connection-run 51345)
"Sat Feb 02 20:49:17 EST 2008"
user=> (. li (close))
Reflection warning, line: 560 - call to close can't be resolved.
nil
user=> li
ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=51345]
user=> (. li (isClosed))
Reflection warning, line: 562 - call to isClosed can't be resolved.
true
user=> (connection-run 51345)
Could not connect to 127.0.0.1 on port 51345
nil

You know, I was expecting that to not work. That really shouldn't have been that easy. To be fair, this doesn't do anything with that thread pool that I created. So I'll shut it down once I'm done running the server.

(defn listener-run-in-background [port]
  (let [listener (listener-new port)
        exec (. Executors (newSingleThreadExecutor))
        run (fn [] (do (listener-run listener port)
                       (. exec (shutdown))))]
    (. exec (submit run))
    (list exec listener)))
user=> (connection-run 51345)
""
user=> li
(java.util.concurrent.Executors$FinalizableDelegatedExecutorService@19bb25a ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=51345])
user=> (connection-run 51345)
""
user=> (. (first li) (isShutdown))
Reflection warning, line: 114 - call to isShutdown can't be resolved.
true

And that, on the other hand, fails miserably, as do my next six attempts. Then it finally occurs that, since in my original version, the executor is empty and unreferenced, all I'm really doing, as I try to bind exec inside itself with increasingly complex combinations of let, do, appl and fn, is creating circular references, and that I'd really just be a lot better off letting that ExecutorService fall on the floor and assuming that, because it can't be reached, it'll get picked up by the garbage collection8.

No Java exceptions are currently handled...

Well, I dealt with two of the three exceptions that I'm seeing routinely. Better catch the other one, the BindException when trying to launch two servers at the same port.

(defn listener-new [port]
  (try (new ServerSocket port)
       (catch BindException except
              (println "Address is already in use."))))

And checking that with an open server:

user=> (def li (listener-run-in-background 51345))
#<Var: user/li>
user=> (def li (listener-run-in-background 51345))
Address is already in use.
#<Var: user/li>
user=> li
nil

Afterthoughts

I'll count that one as done and take a look back at what needs doing. Type/reflection warnings, unit tests and cleaning my hacky "just get it done" code. Really, not bad for an afternoon's learning.

Further things to play with, from this tangent at least, seem to be: serialization: seeing just what I can push as an array of bytes, poking at java.lang.Concurrent and seeing how it reacts.

But as someone who's written more Lisp than Java in his life, I have to say, Clojure had me at the (.) function. Lisp plus library support is making this a very neat language to learn, and unlike my previous forays into Common Lisp, I haven't hit a wall where I found myself needing to reimplement something common9.

Also, despite fairly routinely using (basic) Java objects in this exercise, I never felt like I was having to coerce things to fit a foreign function interface. For that matter, this approach felt more or less like the right way to use objects: as a very rich system of types, rather than as a way to control program structure and flow.


  1. It's not Common Lisp and it's not Scheme. It's its own dialect. Back

  2. To wit:

    • Namespace examination and the capacity to work in multiple files without having to think too hard about how imports interact. (Fixed in the latest version with namespaces becoming first-class citizens.)

    • Metadata being applicable to everything, not just symbols and sequences. (On the agenda to be added.) Back

  3. Clojure counts, for all intents and purposes, as a functional language. It's certainly less strict than Haskell, but on a par with the ML family in terms of being functional. Back

  4. That naive implementation ran out of heap space at 100000 numbers. Likely binding it to a var would help.

    After testing, apparently not.Back

  5. I'm aware that there's lisps with good socket implementations and ones with good threading implementations. It's just that, for me at least, I've never found the two things in the same place at the same time when I needed them. Back

  6. REPL: Read Evaluate Print Loop. A shell in which you interact with your program and the language.

    In theory, I could have written all of this within the Clojure REPL. In practice, it's far simpler to write functions in a separate file and then send them to the REPL.

    This means that, when I do something silly like causing the REPL to hang due to waiting for a network connection, I can just reload my file of functions.

    Also, it does mean that, at the end of the day, I have my functions in useable and modifiable form, rather than as java bytecote that will be lost when I close the REPL. Back

  7. This is an admittedly Haskellish way to build a function. In this case, its results are directly equivalent to the anonymous function (fn [x] (char (aget by x))).

    To dissect this a little more, appl takes a function and at least one argument, then returns a new function that takes the rest of the original function's arguments. In other words, it applies the arguments given to it partially.

    This means that my (appl aget by) creates a new function equivalent to (fn [a] (aget by a)).

    comp, on the other hand, is function composition. For f(a) and g(b) f .comp. g is equivalent to f( g( b ) ) or (f (g a)). In other words, it takes functions and makes a new function that applies them from right to left to its arguments.

    Were I expanding out this function completely, on expanding appl, it becomes: (comp char (fn [a] (aget by a))) and then on composing char to it, (fn [b] (char ((fn [a] (aget by a)) b))).

    In this case, I used comp and appl because I found the resulting form easier to write and apply. By contrast, there are many circumstances where I'd prefer a simple anonymous function. Back

  8. In my defense here, I should point out that I was reading the C++ Frequently Questioned Answers last week, and it may have been channeling a few bad memories. Back

  9. To be fair, one of those involved the aforementioned choice between getting sockets or threads, before throwing up my hands in dismay.Back

1 comment:

  1. lazy-cons was removed, maybe six months ago? so anyone looking for examples, these are out of date.

    ReplyDelete