My second look at Clojure
Casting and complements
As demonstrated to me, (. listener (isClosed)) returns a fresh
boolean, rather than Boolean.TRUE or Boolean.FALSE. So, while
the symbol looks fine in the REPL, when passed to (if (. listener
(isClosed)) ...), it is always true.
So, I made the following pair of complementary1 utility functions:
(defn #^Boolean listener-closed? [#^ServerSocket listener]
(. listener (isClosed)))
def listener-open? (complement listener-closed?))
complement takes a function as its argument and returns a function
that does exactly the same thing as that function, only with its
return value inverted. Thus, (complement listener-closed?) returns a
function that does exactly what listener-closed? does and then
returns the opposite value.
In this case, it saves me from having to repeat listener-closed?'s
details. That way, if I ever need to change listener-closed, in
order, say, to handle more than simple Java ServerSockets, I can do so
without having to make any changes to listener-open?.
Note, though, that listener-open? is a def and not a
defn. That's because defn is a macro that wraps an fn
around its body. complement, on the other hand, returns an
fn, so I only need to bind it to a Var.
I then amended listener-run so that it uses this explicit cast
rather than just taking what ``ServerSocket gives it.
(defn listener-run [listener port]
(loop [socket nil]
(if (listener-closed? listener)
listener
(do (when socket
(. (listener-send socket) (close)))
(recur (listener-wait listener))))))
So, while I'm assigning casting, I should likely take care of the
warnings on loading this file, if for no reason other than that
they're starting to get a little excessively verbose.
user=> (load-file "/clojure/src/sockets.clj")
Reflection warning, line: 17 - call to getBytes can't be resolved.
Reflection warning, line: 53 - call to accept can't be resolved.
Reflection warning, line: 56 - call to close can't be resolved.
Reflection warning, line: 397 - call to getOutputStream can't be resolved.
Reflection warning, line: 396 - call to write can't be resolved.
Reflection warning, line: 397 - call to getOutputStream can't be resolved.
Reflection warning, line: 396 - call to close can't be resolved.
Reflection warning, line: 70 - call to close can't be resolved.
Reflection warning, line: 78 - call to submit can't be resolved.
Reflection warning, line: 88 - call to java.net.Socket ctor can't be resolved.
Reflection warning, line: 97 - call to getInputStream can't be resolved.
Reflection warning, line: 98 - call to read can't be resolved.
Reflection warning, line: 109 - call to close can't be resolved.
#<Var: user/connection-run>
Simply, I went through and placed type hints on functions that call
out to Java in order to ensure that their methods resolve on load.
Since I also noticed a few new things while reformatting the Clojure
manual into a single
texinfo document for my own use, I'll also change my use of (. var
(toString) to use Clojure's str function:
(str x)
Returns x.toString(). (str nil) returns ""
Building a testing framework
All of those casts are fairly minor refactorings that just make the
statement of what a listener and connection are to the
compiler. On the other hand, the bug that I started with this time
would have been caught by the other thing I've been putting off.
Namely, actual an testing framework, rather than just eyeballing
results in the REPL.
Now, I could go find JUnit, install that, and then see how well
Clojure links into it. But, well, that sounds like a new project1.
So, instead, what I'll look at is defining what it is that my
functions are supposed to be doing, write a few tests, then add a
framework to glue those tests together.
Ugliness ensues...
Which ended up looking like:
(def DEFAULT-PORT 51345)
(defn test-byte-array-from-string
([]
(test-byte-array-from-string (current-time)))
([str]
(let [barr (byte-arr-from-string str)
bseq (map (comp char (appl aget barr))
(range (alength barr)))
chseq (map char str)]
(and (== (alength barr)
(count bseq)
(count chseq))
(nil? (first (filter false?
(map eql?
bseq
chseq))))))))
(defn test-listener-new
([]
(test-listener-new DEFAULT-PORT))
([port]
(with-open listener (listener-new port)
(when listener
true))))
(defn test-listener-predicates [])
(defn test-all []
(loop [test-list '(("string->byte array conversion"
test-byte-array-from-string)
("creating a listener",
test-listener-new)
("listener open?/closed? predicates"
test-listener-predicates))
failure false]
(let [this-test (first test-list)
test-list (rest test-list)]
(if (not this-test)
(if failure
(println "---\nSome tests failed.")
(println "---\nAll tests succeeded."))
(let [test-result (time (eval (rest this-test)))]
(println "Testing" (first this-test) ":" test-result
(if (first test-list) "\n" ""))
(recur test-list
(when (or failure
(not test-result))
And, running it, I get:
user=> (test-all)
"Elapsed time: 8.437664 msecs"
Testing string->byte array conversion : true
"Elapsed time: 68.38411 msecs"
Testing creating a listener : true
"Elapsed time: 2.831645 msecs"
Testing listener open?/closed? predicates : nil
---
Some tests failed.
nil
So, it works. (Noting that I deliberately built one of those tests to
fail, to make sure that the testing function catches failure.) Except
that, well, how best to put it... Eww. I know that so far this is
only 150 lines, but I'm not going to remember to update the list of
tests down in the testing function every time I make a new one. So,
I'll run test-all, realise that my new test is missing, then go
back and add the new test.
Adding structure...
No, I think what I need here is for the test itself to hold its own
payload. Sounds like a reasonable time to add in a struct, really. A
struct is just a defined hashmap taking keywords as arguments. For
starters, I'll build one that just duplicates the list that I have
below:
(defstruct unit-test :string :function)
Well, that does let me wrap tests into structures, but doesn't
exactly let me manipulate them:
user=> (struct unit-test :string "creating a listener" :function test-listener-new)
{:function user.test_listener_new@d5eb7, :string "creating a listener"}
What I'm going to need is a way to build these tests and load them
when I load the file, otherwise I'm just making the process of
eyeballing my results in the REPL harder on myself.
After a bit of thought, I settle on this as my test list:
(defstruct unit-test :name :string :function)
(def ALL-TESTS {})
(defn unit-tests-add [test]
(def ALL-TESTS
(assoc ALL-TESTS (get test :name) test)))
(defn unit-test-new
([description function]
(unit-tests-add (struct unit-test
:name function
:string description
:function function))))
Nothing special at the moment. Just a simple structure defining a
unit test as a name, description and function. Now, let's use that
one one of the three rudimentary tests that I already have:
(defn test-listener-new
([]
(test-listener-new DEFAULT-PORT))
([port]
(with-open listener (listener-new port)
(when listener
true))))
(unit-test-new "creating a listener" test-listener-new)
All right. Improving a tiny bit. Except that, after evaluating the
file twice, I still have:
user=> ALL-TESTS
{user.test_listener_new@1a0d866 {:function user.test_listener_new@1a0d866, :string "creating a listener", :name user.test_listener_new@1a0d866}, user.test_listener_new@3f96ee {:function user.test_listener_new@3f96ee, :string "creating a listener", :name user.test_listener_new@3f96ee}}
Makes sense, really. My new instance of test-listener-new has a new
symbol. So, even though I know that they're the same function,
there's no good reason for the compiler to.
What I really want, though, is to make defining a new test as simple
as adding a new (albeit specialised) function, rather than playing
around with remembering to call unit-test-new after. So, rather than
trying to get unit-test-new to parse out the name of the function,
I'll rethink my approach.
What do I actually want? Well, the function, name and description
added to my set of tests seamlessiy. I think it's fair to assume that
before I'm done, I'll want a few more qualifiers on tests as well.
Since I don't know what those are, I won't add them to my structure yet.
Admitting that I need macros...
However, I'm in a conundrum here, because I don't want to add logic to
my test definition every time I think of something new. Really, this
is starting to sound like a good argument for a macro.
I'll start by just imitating what defn does. Because if I can't
imitate that in a macro, the rest of this is going to start smelling
like week-old moose.
(defmacro deftest [fname description extras & fdecl]
`(defn ~fname ~@fdecl))
So that takes a function name, a descriptive string and a bunch of
unspecified "extras" and then proceeds to ignore the second two before
shoving the function name and the declaration into a defn. The
backtick (`) marks the following list as a template, meaning that the
defn won't be resolved right away.
The ~ and ~@ macro characters inside that template mean,
respectively, to unquote fname (so that I'm not defining a
function called user/fname) and to resolve fdecl into the
sequence of values which it represents.
I have to do this because the & rest argument to an fn is
considered to be a sequence. Were I simply passing it as an unquoted
symbol, I'd end up with an extra set of parentheses around the
function body when it's evaluated, which I don't want.
So, having rationalised every jot and tittle of what I just did, I'm
going to dump test-listener-new - that is to say, a working
function - into deftest via macroexpand-1 and see if it generates a
plausible defn. (macroexpand-1 because I don't really want to see
the full expanion of defn's internals, just this first layer that
I added.
user=> (macroexpand-1 '(deftest test-listener-new "creating a listener" '()
([]
(test-listener-new DEFAULT-PORT))
([port]
(with-open listener (listener-new port)
(when listener
true)))))
(clojure/defn test-listener-new ([] (test-listener-new DEFAULT-PORT)) ([port] (with-open listener (listener-new port) (when listener true))))
And that resulting definition looks just like the definition of
test-listener-new, only on one line and with defn as a
qualified namespace/symbol pair.
Seeing as that worked, it means that all I should have to do is wrap
defining the function in a do and add, as the second half, a
function call adding the test to my list of tests.
(defmacro deftest [fname description extras & fdecl]
`(do (defn ~fname ~@fdecl)
(unit-tests-add (struct unit-test
:name (name '~fname)
:string ~description
:function ~fname
~@extras))))
Which, when expanded, gats me:
user=> (macroexpand-1 '(deftest test-listener-new "creating a listener" '()
([]
(test-listener-new DEFAULT-PORT))
([port]
(with-open listener (listener-new port)
(when listener
true)))))
(do (clojure/defn test-listener-new ([] (test-listener-new DEFAULT-PORT)) ([port] (with-open listener (listener-new port) (when listener true)))) (user/unit-tests-add (clojure/struct user/unit-test :name (clojure/name (quote test-listener-new)) :string "creating a listener" :function test-listener-new quote ())))
Which counts as an almost-but-not-quite. Because I'm quoting the
empty list in the arguments that I'm passing to deftest, it
expands to (quote ()), which is then unquoted to quote ().
Which isn't exactly what I want.
On the other hand, the working syntax:
user=> (macroexpand-1 '(deftest test-listener-new "creating a listener" ()
([]
(test-listener-new DEFAULT-PORT))
([port]
(with-open listener (listener-new port)
(when listener
true)))))
(do (clojure/defn test-listener-new ([] (test-listener-new DEFAULT-PORT)) ([port] (with-open listener (listener-new port) (when listener true)))) (user/unit-tests-add (clojure/struct user/unit-test :name (clojure/name (quote test-listener-new)) :string "creating a listener" :function test-listener-new)))
It works fine, and saves me a spurious quote. Furthermore, values in a
struct default to nil, so if I add more bits to the unit-test
structure later, I don't have to cope with accounting for them in any
way other than doing nothing for nil, and my previously-defined
tests will still work.
Applying these new tests.
So, I modify my test to use this macro and get:
(deftest test-listener-new "creating a listener" ()
([]
(test-listener-new DEFAULT-PORT))
([port]
(with-open listener (listener-new port)
(when listener
true))))
And evaluating that adds it to ALL-TESTS.
user=> ALL-TESTS
{"test-listener-new" {:function user.test_listener_new@1dcc2a3, :string "creating a listener", :returns nil, :name "test-listener-new"}}
I could have put the extras (that part that I have a suspicion
that I'll need, but don't know why yet,) at the end of the deftest
macro. However, that means that I'd likely forget to add them. Also,
it means that I have to make more changes to an existing test function
than simply adding two pieces of information.
On the other hand, because it returns the result of
unit-tests-add, deftest isn't quite a drop-in replacement to
defn yet. Time to get it to return the function instead.
So, knowing the function name, and establishing the reasonable
qualification that the function has been defined (which it should have
been, as defining it is a part of the macro, what I want to return is
the Var named by fname. Which leads me to:
(defmacro deftest [fname description extras & fdecl]
`(do (defn ~fname ~@fdecl)
(unit-tests-add (struct unit-test
:name (name '~fname)
:string ~description
:function ~fname
~@extras))
#'~fname))
Which, when I evaluate my test, returns the test function, ensuring
that this is now a wrapper around defn that actually returns the right
values.
That done, I can finally get to converting my remaining two tests and
the test-all function. The tests are easy:
(deftest test-byte-array-from-string "string->byte array conversion" ()
([]
(test-byte-array-from-string (current-time)))
([str]
(let [barr (byte-arr-from-string str)
bseq (map (comp char (appl aget barr))
(range (alength barr)))
chseq (map char str)]
(and (== (alength barr)
(count bseq)
(count chseq))
(nil? (first (filter false?
(map eql?
bseq
chseq))))))))
(deftest test-listener-predicates "listener open?/closed? predicates" () [])
To make the testing function work, it needs to be able to get at my
vector of tests and to examine the tests.
(def unit-test-name (accessor unit-test :name))
(def unit-test-string (accessor unit-test :string))
(def unit-test-function (accessor unit-test :function))
A side-trip into accessors...
Here, however, I'm noticing that I'm duplicating what's already
written once in my struct, just to have accessors. Also, :string
is a daft name.
So, I'll make :string into :description and generate my
accessors based directly off of the structure.
(map (fn [key] (def (sym (name *current-namespace*)
(strcat "unit-test-" (name key)))
key))
(keys unit-test))
And testing that:
user=> clojure.lang.Compiler$CompilerException: REPL:1026: Second argument to def must be a Symbol
at clojure.lang.Compiler.analyzeSeq(Compiler.java:3065)
at clojure.lang.Compiler.analyze(Compiler.java:3011)
at clojure.lang.Compiler.analyze(Compiler.java:2986)
at clojure.lang.Compiler.access$400(Compiler.java:38)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:2740)
at clojure.lang.Compiler$FnMethod.parse(Compiler.java:2626)
at clojure.lang.Compiler$FnMethod.access$1300(Compiler.java:2540)
at clojure.lang.Compiler$FnExpr.parse(Compiler.java:2327)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:3056)
at clojure.lang.Compiler.analyze(Compiler.java:3011)
at clojure.lang.Compiler.analyze(Compiler.java:2986)
at clojure.lang.Compiler.access$400(Compiler.java:38)
at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:2255)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:3060)
at clojure.lang.Compiler.analyze(Compiler.java:3011)
at clojure.lang.Compiler.analyze(Compiler.java:2986)
at clojure.lang.Compiler.eval(Compiler.java:3085)
at clojure.lang.Repl.main(Repl.java:59)
Caused by: java.lang.Exception: Second argument to def must be a Symbol
at clojure.lang.Compiler$DefExpr$Parser.parse(Compiler.java:296)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:3058)
... 17 more
Hmm, not so good. Ok, time to poke at what exactly it is that I'm
doing wrong here.
user=> (sym "user" "a")
user/a
user=> (def (sym "a") 1)
clojure.lang.Compiler$CompilerException: REPL:1030: Second argument to def must be a Symbol
at clojure.lang.Compiler.analyzeSeq(Compiler.java:3065)
at clojure.lang.Compiler.analyze(Compiler.java:3011)
at clojure.lang.Compiler.analyze(Compiler.java:2986)
at clojure.lang.Compiler.eval(Compiler.java:3085)
at clojure.lang.Repl.main(Repl.java:59)
Caused by: java.lang.Exception: Second argument to def must be a Symbol
at clojure.lang.Compiler$DefExpr$Parser.parse(Compiler.java:296)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:3058)
... 4 more
user=> (instance? clojure.lang.Symbol (sym "user" "a"))
clojure.lang.Compiler$CompilerException: REPL:3: Unable to resolve classname: clojure.lang.PersistentList@d165fe67
at clojure.lang.Compiler.analyzeSeq(Compiler.java:3065)
at clojure.lang.Compiler.analyze(Compiler.java:3011)
at clojure.lang.Compiler.analyze(Compiler.java:2986)
at clojure.lang.Compiler.eval(Compiler.java:3085)
at clojure.lang.Repl.main(Repl.java:59)
Caused by: java.lang.IllegalArgumentException: Unable to resolve classname: clojure.lang.PersistentList@d165fe67
at clojure.lang.Compiler$InstanceExpr$Parser.parse(Compiler.java:1868)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:3058)
... 4 more
So, it appears that both def and instance? are seeing the
unexpanded list before it gets turned into a symbol. Curiouser and
curiouser.
user=> (map (fn [key] (let [my-sym (sym (name *current-namespace*)
(strcat "unit-test-" (name key)))]
my-sym))
(keys unit-test))
(user/unit-test-name user/unit-test-function user/unit-test-description)
user=> (map (fn [key] (let [my-sym (sym (name *current-namespace*)
(strcat "unit-test-" (name key)))]
(def my-sym 1)))
(keys unit-test))
(#<Var: user/my-sym> #<Var: user/my-sym> #<Var: user/my-sym>)
Ok, so I redefined my-sym three times. After I got as far as:
(map (fn [li] (def ~@li))
(map (fn [key] (list (sym (name *current-namespace*)
(strcat "unit-test-" (name key)))
(accessor unit-test key)))
(keys unit-test)))
I've decided that I'll no longer make a big deal out of building my
accessors from my structure, as I seem to have a merry clash between
what def wants and what I know how to offer to it.
I'll admit that this failure irks me, but I'm going to leave it as
something to poke at later, as I'm getting confused as to operator and
macro precedence.
Sobeit:
(def unit-test-name (accessor unit-test :name))
(def unit-test-description (accessor unit-test :description))
(def unit-test-function (accessor unit-test :function))
Giving up and going back to the test function.
Now, on to making the test function actually work with this structure,
rather than playing with trying to be clever with generating
accessors.
(defn test-all []
(loop [test-list (vals ALL-TESTS)
failure false]
(let [this-test (first test-list)
test-list (rest test-list)]
(if (not this-test)
(if failure
(println "---\nSome tests failed.")
(println "---\nAll tests succeeded."))
(let [test-result (time (unit-test-function this-test))]
(println "Testing" (unit-test-description this-test) ":" test-result
(if (first test-list) "\n" ""))
(recur test-list
(when (or failure
(not test-result))
However, testing that, it shows only that the unit test functions
exist. Nothing more.
user=> (test-all)
"Elapsed time: 0.181588 msecs"
Testing creating a listener : user.test_listener_new@1878144
"Elapsed time: 0.028216 msecs"
Testing string->byte array conversion : user.test_byte_array_from_string@137d090
"Elapsed time: 0.019277 msecs"
Testing listener open?/closed? predicates : user.test_listener_predicates@15db314
---
All tests succeeded.
nil
I need it to evaluate the returned function, preferably with the
capacity to insert arguments as needed. Hmm.
(defn test-all []
(loop [test-list (map (fn [x] (list (unit-test-description x)
(unit-test-function x)))
(vals ALL-TESTS))
failure false]
(let [test (first test-list)
test-list (rest test-list)]
(if (not test)
(if failure
(println "---\nSome tests failed.")
(println "---\nAll tests passed."))
(let [test-result (time (eval (list (second test))))]
(println "Testing" (first test) ":" test-result)
(recur test-list
(when (or (not test-result)
failure)
true)))))))
Will leave that as a working test framework for the moment and
actually move back to writing tests.
(deftest test-listener-predicates "listener open?/closed? predicates" ()
([]
(test-listener-predicates DEFAULT-PORT))
([port]
(let [listener (listener-new port)
tests (and (not (listener-closed? listener))
(listener-open? listener))]
(. listener (close))
(and tests
(not (listener-open? listener))
(listener-closed? listener)))))
(deftest test-listener-close "closing a listener" ()
([]
(let [listener (listener-new DEFAULT-PORT)]
(and (listener-open? listener)
(not (listener-close listener))
(listener-closed? listener)))))
(deftest test-connection-new "creating a connection" ()
([]
(let [listener (listener-run-in-background)
connection (connection-new)]
(and (connection-new)
(not (listener-close listener))
(not (connection-new))))))
(deftest test-connection-run "running a complete connection" ()
([]
(let [listener (listener-run-in-background)
result (connection-run)]
(and (listener-open? listener)
(not (listener-close listener))
(listener-closed? listener)
(> 0 (. result (length)))
(== 0 (. (connection-run) (length)))))))
So there's my complete set of tests to date. Which I can then without
changing the test-all that I had already defined.
user=> (test-all)
"Elapsed time: 11.890338 msecs"
Testing creating a listener : true
Could not connect to 127.0.0.1 on port 51345
"Elapsed time: 1194.393981 msecs"
Testing creating a connection : true
"Elapsed time: 4.478223 msecs"
Testing string->byte array conversion : true
"Elapsed time: 35.4397 msecs"
Testing running a complete connection : false
"Elapsed time: 11.453132 msecs"
Testing closing a listener : true
"Elapsed time: 13.804548 msecs"
Testing listener open?/closed? predicates : true
---
Some tests failed.
nil
Ok, so I notice two things here. One is that every test works except
what should be the last one. The other is that, well, it isn't the
last one. In fact, the tests are in no particular order when, in
fact, they should have a certain level of ordering in them.
Setting up after statements...
Sounds like I finally have a use for that extras argument that,
until now, I haven't been using.
(defstruct unit-test :name :description :function :after)
(def unit-test-after (accessor unit-test :after))
(deftest test-listener-predicates "listener open?/closed? predicates"
(:after "test-listener-new")
([]
(test-listener-predicates DEFAULT-PORT))
([port]
(let [listener (listener-new port)
tests (and (not (listener-closed? listener))
(listener-open? listener))]
(. listener (close))
(and tests
(not (listener-open? listener))
(listener-closed? listener)))))
(deftest test-listener-close "closing a listener"
(:after "test-listener-predicates")
([]
(let [listener (listener-new DEFAULT-PORT)]
(and (listener-open? listener)
(not (listener-close listener))
(listener-closed? listener)))))
(deftest test-connection-new "creating a connection"
(:after "test-listener-close")
([]
(let [listener (listener-run-in-background)
connection (connection-new)]
(and (connection-new)
(not (listener-close listener))
(not (connection-new))))))
(deftest test-connection-run "running a complete connection"
(:after :all)
([]
(let [listener (listener-run-in-background)
result (connection-run)]
(and (listener-open? listener)
(not (listener-close listener))
(listener-closed? listener)
(> 0 (. result (length)))
(== 0 (. (connection-run) (length)))))))
There. Now tests that should have sequence can have them. But this
means that I need to actually extract my function for generating the
list of unit tests and make it respect the rules that I just added.
(defn unit-test-list
([]
(map (fn [x] (list (unit-test-description x)
(unit-test-function x)))
(vals ALL-TESTS))))
Which gives me the same result for test-all. Now, to sort this
data rather than just dropping it in a list.
(defn unit-test-list-helper [unsorted sorted]
(let [test (first unsorted)
unsorted (rest unsorted)]
(cond (or (not (unit-test-after test))
(and (eql? :all (unit-test-after test))
(not unsorted))
(first (filter (fn [x] (eql? (unit-test-name x)
(unit-test-after test)))
unsorted)))
(list unsorted
(concat sorted
(list test)))
:t
(list (concat unsorted
(list test))
sorted))))
So what that's supposed to do is run through the unsorted list, adding
things to the sorted list when they either have no after statement
or their after statement is met.
Now, I'll bind that helper into a loop:
(defn unit-test-list
([]
(cond (rest (filter (fn [test] (eql? :all
(unit-test-after test)))))
(do (println "Error: Only one test may be marked as"
"coming after all others.")
[])
:t
(unit-test-list (vals ALL-TESTS) (list))))
([unsorted sorted]
(if (not (first unsorted))
(map (fn [test] (list (unit-test-description test)
(unit-test-function test)))
sorted)
(let [helped-list (unit-test-list-helper unsorted sorted)
unsorted (first helped-list)
sorted (second helped-list)]
(recur unsorted sorted)))))
And running it:
user=> (unit-test-list (vals ALL-TESTS) nil)
It hangs.
Stepping through the helper function in a long series of examinations
tells me that it's hanging at the last step. Then, after changing it
to reflect my actual intent (I typoed sorted as unsorted. I
start to think about whether this is actually a good way to build a
list. After all, it will take up to the factorial of the length of
the list of tests to actually build it.
I poke with comparators for a bit and find that my somewhat fuzzy
requirements don't exactly meet comparator's exacting demands2. Then I
decide to revisit my iffy list function, but move blocks of tests at a
time.
(defn unit-test-list-helper [unsorted sorted]
(let [nils (filter (comp not unit-test-after)
unsorted)
alls (filter (comp (appl eql? :all) unit-test-after)
unsorted)
others (filter (fn [t] (not-any? (fn [x] (eql? (unit-test-after t)
(unit-test-after x)))
(concat nils alls)))
unsorted)
in-sorted (fn [t] (some (fn [x] (eql? (unit-test-name x)
(unit-test-after t)))
(concat nils sorted)))]
(cond (first nils)
(list (concat alls
(filter (complement in-sorted)
others))
(concat nils
(filter in-sorted others)))
(== (count alls)
(count unsorted))
(list nil
(concat sorted alls))
:t
(list (concat alls
(filter (complement in-sorted)
others))
(concat sorted
(filter in-sorted others))))))
This one takes the unsorted list, breaks it into three parts, and
moves what it can over to sorted at each step. It sorts my list
of functions in three steps, rather than twenty. However, those steps
are relatively expensive. I might as well compare the two functions'
performances while I've them both on my screen.
user=> (do (dotimes x 10 (time (unit-test-list-old))) (println "----------") (dotimes x 10 (time (unit-test-list))))
"Elapsed time: 3.240635 msecs"
"Elapsed time: 0.478832 msecs"
"Elapsed time: 0.459835 msecs"
"Elapsed time: 0.713778 msecs"
"Elapsed time: 0.462908 msecs"
"Elapsed time: 0.450616 msecs"
"Elapsed time: 0.433575 msecs"
"Elapsed time: 0.506489 msecs"
"Elapsed time: 0.453689 msecs"
"Elapsed time: 5.946846 msecs"
----------
"Elapsed time: 8.428725 msecs"
"Elapsed time: 1.182832 msecs"
"Elapsed time: 1.732902 msecs"
"Elapsed time: 1.130031 msecs"
"Elapsed time: 1.121931 msecs"
"Elapsed time: 1.130869 msecs"
"Elapsed time: 1.137016 msecs"
"Elapsed time: 1.390959 msecs"
"Elapsed time: 1.163555 msecs"
"Elapsed time: 1.132546 msecs"
Interesting. The old one actually did better than the new. Now, I
could count their operations, but it would be more fun to get the REPL
testing this for me.
user=> (dotimes x 100 (eval `(deftest ~(gensym) "generated fake test" (:after nil) [] true)))
There. Made a hundred test macros with no structure to them.
gensym just makes a symbol with a guaranteed-unique name. That
ensures that, in generating these tests, I don't have to first come up
with my own means to make unique nmes.
Now to try again:
user=> (do (dotimes x 10 (time (unit-test-list-old))) (println "----------") (dotimes x 10 (time (unit-test-list))))
"Elapsed time: 108.745944 msecs"
"Elapsed time: 44.778876 msecs"
"Elapsed time: 38.915002 msecs"
"Elapsed time: 47.836247 msecs"
"Elapsed time: 41.699434 msecs"
"Elapsed time: 40.098393 msecs"
"Elapsed time: 41.917059 msecs"
"Elapsed time: 40.536157 msecs"
"Elapsed time: 39.040995 msecs"
"Elapsed time: 39.437135 msecs"
----------
"Elapsed time: 21.603863 msecs"
"Elapsed time: 17.80338 msecs"
"Elapsed time: 17.744992 msecs"
"Elapsed time: 19.674568 msecs"
"Elapsed time: 13.584687 msecs"
"Elapsed time: 19.662834 msecs"
"Elapsed time: 17.736891 msecs"
"Elapsed time: 18.160967 msecs"
"Elapsed time: 13.494174 msecs"
"Elapsed time: 16.875608 msecs"
A gap is starting to form, I think. What about adding some test macros that do rely on other tests?
user=> (import '(java.util Random))
nil
user=> (def RNG (new Random))
#<Var: user/RNG>
user=> (. RNG (nextInt))
Reflection warning, line: 948 - call to nextInt can't be resolved.
1410779829
user=> `(deftest ~(gensym) "z" (:after nil) [] true))
user=> `(deftest ~(gensym) "z" (:after nil) [] true)
(user/deftest G__2659 "z" (:after nil) [] true)
user=> `(deftest ~(gensym) "z" (:after ~(let [keys (keys ALL-TESTS)] (first (drop (rem (. RNG (nextInt)) (count keys)) keys)))) [] true)
Reflection warning, line: 951 - call to nextInt can't be resolved.
(user/deftest G__2665 "z" (:after "G__2491") [] true)
user=> (dotimes x 1000 (eval `(deftest ~(gensym) "z" (:after ~(let [keys (keys ALL-TESTS)] (first (drop (rem (. RNG (nextInt)) (count keys)) keys)))) [] true)))
So, that adds a thousand tests in that do follow a structure, as
they all have an after statement this time.
And I run the test again ...
And I go get coffee ...
5 minutes...
It seems to be still running...
Maybe a thousand was a bad seed...
Oh, there it is:
user=> (do (dotimes x 10 (time (unit-test-list-old))) (println "----------") (dotimes x 10 (time (unit-test-list))))
"Elapsed time: 21438.660779 msecs"
"Elapsed time: 24259.176236 msecs"
"Elapsed time: 20528.833693 msecs"
"Elapsed time: 20170.246777 msecs"
"Elapsed time: 23642.198659 msecs"
"Elapsed time: 20390.313015 msecs"
"Elapsed time: 19931.102975 msecs"
"Elapsed time: 22232.556093 msecs"
"Elapsed time: 21692.653954 msecs"
"Elapsed time: 20286.386119 msecs"
----------
"Elapsed time: 5885.515896 msecs"
"Elapsed time: 7089.577027 msecs"
"Elapsed time: 6119.980053 msecs"
"Elapsed time: 6135.949681 msecs"
"Elapsed time: 6105.716229 msecs"
"Elapsed time: 6035.137985 msecs"
"Elapsed time: 6120.137894 msecs"
"Elapsed time: 5858.970903 msecs"
"Elapsed time: 5882.34203 msecs"
"Elapsed time: 6120.462796 msecs"
So, definitely, as the structure gets more complicated, the slightly
more sensible approach is pulling ahead.
But just for fun, as I'm going to bed:
user=> #<Var: user/ALL-TESTS>
user=> (dotimes x 100 (eval `(deftest ~(gensym) "generated fake test" (:after nil) [] true)))
nil
user=> (dotimes x 100000 (eval `(deftest ~(gensym) "generated fake test" (:after ~(let [keys (keys ALL-TESTS)] (first (drop (rem (. RNG (nextInt)) (count keys)) keys)))) [] true)))
Reflection warning, line: 957 - call to nextInt can't be resolved.
(do (dotimes x 10 (time (unit-test-list-old))) (println "----------") (dotimes x 10 (time (unit-test-list))))
Ok, somewhere before generating a 100000-length tree of tests, it
crashed, due to a lack of symbols. Fair enough. That test was just
waving things around to see if I can.
clojure.lang.Compiler$CompilerException: REPL:749: PermGen space
user=> (count (keys ALL-TESTS))
42577
Revisiting generating my accessors.
However, going back one step, all that silly mucking around with
gensyms and defines gives me an idea:
(map (fn [key] (eval `(def ~(sym (name *current-namespace*)
(strcat "unit-test-"
(name key)))
(accessor unit-test ~key))))
(keys unit-test))
There. Now I know that, should I later decide to add more keys to
that structure, the accessors will be there and waiting for me.
Except that, in a freshly-loaded REPL (with this file loaded via
load-file rather than dumping it in, I find that my accessors are
no longer loaded.
All right, I'll take the hint, stop picking at this, and leave them as
normal functions rather than generated ones.
Revisiting actually running the tests
So, I try running all my tests again. And the test function (which I
haven't changed since I started that little digression into sorting my
tests) still does what it's told, albeit with a tiny wart and a failed
test.
user=> (test-all)
"Elapsed time: 306.707138 msecs"
Testing creating a listener : true
"Elapsed time: 216.1839 msecs"
Testing string->byte array conversion : true
"Elapsed time: 11.579405 msecs"
Testing listener open?/closed? predicates : true
"Elapsed time: 10.170846 msecs"
Testing closing a listener : true
Could not connect to 127.0.0.1 on port 51345
"Elapsed time: 1064.018472 msecs"
Testing creating a connection : true
"Elapsed time: 46.276552 msecs"
Testing running a complete connection : false
---
Some tests failed.
And here's the offending test. I'll step through it one bit at a time
and see what's going wrong in actuality.
(deftest test-connection-run "running a complete connection"
(:after :all)
([]
(let [listener (listener-run-in-background)
result (connection-run)]
(and (listener-open? listener)
(not (listener-close listener))
(listener-closed? listener)
(> 0 (. result (length)))
(== 0 (. (connection-run) (length)))))))
user=> (def listener (listener-run-in-background))
#<Var: user/listener>
user=> (def result (connection-run))
#<Var: user/result>
user=> (listener-open? listener)
true
user=> (listener-close listener)
nil
user=> (listener-closed? listener)
true
user=> (> 0 (. result (length)))
Reflection warning, line: 306 - call to length can't be resolved.
false
user=> (. result (length))
Reflection warning, line: 307 - call to length can't be resolved.
28
user=> (> (. result (length)) 0)
Reflection warning, line: 308 - call to length can't be resolved.
true
Ok, so I messed up the ordering of greater-than and less-than. I find
that I do this a lot in Lisp, because of how I read (> 0 ...
mentally. Namely, as a single function, testing if what follows is
greater than the integer that I've put there. Or, in other words,
exactly backwards.
So, change that to (> (. result (length)) 0) and re-run the test
function:
user=> (test-all)
java.lang.NullPointerException
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:24)
at user.test_connection_run.invoke(Unknown Source)
at clojure.lang.AFn.applyToHelper(AFn.java:171)
at clojure.lang.AFn.applyTo(AFn.java:164)
at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213)
at clojure.lang.Compiler.eval(Compiler.java:3086)
at clojure.eval.invoke(boot.clj:635)
at user.test_all.invoke(Unknown Source)
at clojure.lang.AFn.applyToHelper(AFn.java:171)
at clojure.lang.AFn.applyTo(AFn.java:164)
at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213)
at clojure.lang.Compiler.eval(Compiler.java:3086)
at clojure.lang.Repl.main(Repl.java:59)
"Elapsed time: 9.756547 msecs"
Testing creating a listener : true
"Elapsed time: 4.988344 msecs"
Testing string->byte array conversion : true
"Elapsed time: 26.513984 msecs"
Testing listener open?/closed? predicates : true
"Elapsed time: 10.239011 msecs"
Testing closing a listener : true
Could not connect to 127.0.0.1 on port 51345
"Elapsed time: 997.681828 msecs"
Testing creating a connection : true
Could not connect to 127.0.0.1 on port 51345
Erk. All right, that's getting a bit odd. Changing it back gets me
the previous, broken test, but that's not exactly helpful. Looking at
that dump, I can see that, first off, the error happened in
test-connection-run, but, secondly, it happened when trying to
invoke an instance method.
Further, on that line, the only one I've been changing, there is
indeed an instance method. However, on a successful connection, even
one with no data, it should return an empty string.
The answer here lies with the next line.
user=> (== 0 (. (connection-run) (length)))
Reflection warning, line: 894 - call to length can't be resolved.
java.lang.NullPointerException
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:24)
at clojure.lang.Compiler$InstanceMethodExpr.eval(Compiler.java:950)
at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2212)
at clojure.lang.Compiler.eval(Compiler.java:3086)
at clojure.lang.Repl.main(Repl.java:59)
Could not connect to 127.0.0.1 on port 51345
Before, the failure at the successful length was causing and to stop
and return false, in a "short-circuiting" behaviour that's fairly
normal3.
Now, what that reminds me is that (connection-run) is going to
return nil when it can't open a connection at all. So thus, I'm
calling (length) on nil: (. nil (length)) which,
deservedly, throws an error. So, in this case, the reflection
warnings about length not being resolved were helping me rather
than nagging.
(defn #^Int length-of-string [#^String string]
(if string
(. string (length))
0))
user=> (length-of-string "asdf")
4
user=> (length-of-string nil)
0
(deftest test-connection-run "running a complete connection"
(:after :all)
([]
(let [listener (listener-run-in-background)
result (connection-run)]
(and (listener-open? listener)
(not (listener-close listener))
(listener-closed? listener)
(<= 0 (length-of-string result))
(== 0 (length-of-string (connection-run)))))))
user=> (test-all)
"Elapsed time: 75.76773 msecs"
Testing creating a listener : true
"Elapsed time: 3.349029 msecs"
Testing string->byte array conversion : true
"Elapsed time: 9.635583 msecs"
Testing listener open?/closed? predicates : true
"Elapsed time: 9.517131 msecs"
Testing closing a listener : true
Could not connect to 127.0.0.1 on port 51345
"Elapsed time: 944.630393 msecs"
Testing creating a connection : true
Could not connect to 127.0.0.1 on port 51345
"Elapsed time: 1001.659708 msecs"
Testing running a complete connection : true
---
All tests passed.
And, testing all of that, it does indeed work out the way it ought to,
except for the aforementioned wart. To point it out, "Could not
connect to 127.0.0.1 on port 51345" would appear to indicate an error,
whereas I know that it's actually a valid part of both the tests for
which it appears.
So, here's the problem: those errors are useful rather than throwing
an exception when connecting manually, but they give the wrong
indication (to me at least) when connecting automatically.
So, what I'd like is to shove any messages generated into something
other than standard output, where I can inspect them if I so choose.
(defn test-all []
(let [test-out (new StringWriter)]
(loop [test-list (unit-test-list)
failure false]
(let [test (first test-list)
test-list (rest test-list)]
(cond (not test)
(do (if failure
(println "---\nSome tests failed.")
(println "---\nAll tests passed."))
(list failure test-out))
:t
(let [test-result (time (binding [*out* test-out]
(eval (list (second test)))))]
(println "Testing" (first test) ":" test-result)
(recur test-list
(when (or (not test-result)
failure)
true))))))))
So, what that does is, before doing anything, creates a StringWriter
outside the loop. Then, when it comes time to perform the test,
temporarily re-binds out (the variable telling Clojure where to
print to) to that StringWriter.
This means, now, that I can return the success/failure of the set of
tests, as well as any error messages, as a list.
user=> (def test-results (test-all))
"Elapsed time: 8.808941 msecs"
Testing creating a listener : true
"Elapsed time: 3.364394 msecs"
Testing string->byte array conversion : true
"Elapsed time: 8.342401 msecs"
Testing listener open?/closed? predicates : true
"Elapsed time: 63.791348 msecs"
Testing closing a listener : true
"Elapsed time: 982.201928 msecs"
Testing creating a connection : true
"Elapsed time: 1002.468191 msecs"
Testing running a complete connection : true
---
All tests passed.
#<Var: user/test-results>
user=> test-results
(nil Could not connect to 127.0.0.1 on port 51345
Could not connect to 127.0.0.1 on port 51345
)
Afterthoughts
So, now I have a very basic (albeit ostensibly extensible) testing
framework, as well as a setup that generates that framework
automatically.
So, now a few (code-dump-free) thoughts as I actually make sure that
my in-file commentary is up to date.
Mistakes
What are the current downsides to this system and this approach?
Well, for starters, the test framework only recognises nil and
true. This means that, when an early test manages to hang and
throw an exception, it brings down the test framework as well.
In laziness, I chose to use return values in order to not have to play
too much with caught/thrown errors, and instead passed truth/falsity
around. Not necessarily the best approach, but I haven't decided
whether throwing/catching is better in this case5.
Also, I'll be the first to admit that I'm handling errors in a bit of
a cheap way, and one that, were I actually turning this into a real
component, rather than something that I'm building as the "build one
to throw away" of my learning process, I'd have started to question at
about the point that I was wrapping a binding around println
in order to isolate my error handling away from my testing function.
Further, the 100 lines defining my simple testing framework should
really be extracted from this client-server application. They're only
loosely attached to each other, so I should be able to give them
their own namespaces with minimal hassle. (I'll wait for the next
Clojure release to do that, though, because I know I'll have to change
a bunch in order to handle upcoming changes, and would prefer to
discuss namespaces with myself once the new system is in.)
Lessons
On contrast, what have I learned? Well, for one, do not try to
generate 17 000 lines of Lisp at once via an elisp command. Or, at
least, save your document first. (I lost all of my poking at
comparators via that gaffe.)
Also, that macros really aren't especially terrifying. Believe me or
don't, but deftest was the first Lisp macro that I've ever
written. I'm still tripping over order of operations and how forms
get expanded and names get resolved, especially when dealing with
things like def and load_file, but I do think that that falls
simultaneously with me trying to be too clever and not actually
understanding what it is that I'm doing.
As I write more, I keep finding new boot.clj functions that let me
write a verbose expression a lot more succinctly and clearly. My new
ones are with-open, some, not-any? and anything beginning
with sort.
I'm finding that having the manual as a single, indexed document is
making me try harder to use Clojure's functions for doing something,
rather than rolling my own off of Java's library.
As I've found before, having a single unified environment in which to
write code, then evaluate it selectively is a great asset. It
means, for me at least, that I spend less time having to mentally
page-swap as I try to remember what I was going to do next.
There's some caveats to that, though: Number one is that my current
environment state doesn't necessarily match what I have written
down. I found myself, this session, killing by REPL on occasion just
to make sure that it was matching my work as exactly as possible.
Also, it makes me lazy about making sure that, e.g., functions are
declared before another function references them. Which works just
fine, right up until I try to load the file and realise that my
selective evaluation has (yet again) created a file that needs minor
reordering in order to load.
But that's a trivial irritation. And far outweighed by being able to
write this (as HTML-heavy
Markdown), manipulate
the actual source of this program and run the REPL, all in the same
environment, and all sharing data6.
Future / Goals
Well, beyond the nebulous goals I dropped at the end of my initial
look, I now have a few more.
Cease using nil as an error value.
Move to separate test / application namespaces.
Look at existing functions that do the same thing (the closes
and length-of-string come to mind) and push them into
multimethods instead.
(Yes, the long-term geal here is to meander through all 21
chapters of the manual as I find needs.)
Explore wrapping an extant Java unit testing facility in Clojure
rather than rolling my own (a nice learning exercise but not exactly
a valid solution to anything beyond making myself try out the
language.
Ok, it sounds like a fine idea for a project, but
I'm trying not to let my focus wander all over the place, so I'll keep
this experiment/application self-contained. Back
To clarify my failure to use comparators a little
bit, a comparator is an object/function that grabs some items,
compares them, and replies which one is greater.
user=> (sort (comparator >) [5 4 8 4 3 8 4 6])
(8 8 6 5 4 4 4 3)
user=> (sort (comparator (fn [x y] (println x y (> x y)) >)) [5 4 8 4 3 8 4 6])
5 4 true
4 8 false
8 4 true
3 8 false
8 4 true
4 6 false
4 3 true
(5 4 8 4 3 8 4 6)
user=> (sort (comparator (fn [x y] (println x y (>= x y)) >)) [5 4 4 4])
5 4 true
4 4 true
4 4 true
(5 4 4 4)
Here's where that falls apart for my purposes, though: when
sorting numbers, letters, names, etc., I know the relative
ordering of any two. On the other hand, with these tests, unless
one of four conditions is met, the ordering of the two items is
unknown.
:after is nil. The test comes before all others.
:after is :all. The test comes after all others.
Both tests have the same :after. They are equivalent.
One test's :name is the other test's :after. They have
to go in that order.
My problem is that most of my comparisons do not meet these
standards. Hence my sort functions both having a means to defer
looking at an item until a condition is met. Back
This meaning, in a logical condition, to stop
evaluating the condition as soon as its truth / falsity becomes known.
Here, and in other tests, I was using it as somewhat of a shorthand to
say, "If the test fails at any point, the entire test has failed."
On the other hand, using logical predicates (and, or,
etc.) as control structure can quickly get
unreadable at a glance.
(or (or (and (test1) (test2))
(test3))
(or (and (test4) (test5) (test6))
(and (test6) (test7) (or (test8)
(test8a)
(test8b)))
(test9)))
A contrived example, sure, but I lost the flow of that logical
tree midway through writing the test. I also tried to devise it
so that the tree structure would bear at least a passing
resemblance to something that had been modified to deal with a
later condition4 Back
(For what it's worth, it says: Perform test1. On
success, perform test2. On success of test1 and test2, return
success. Otherwise, perform test3. On success of test3, return
success. On failure of test3, try test4, test5, test6, stopping if
there's failure. And so on, until, if all other tests have failed,
return the success/failute of test9.) Back
Yes, I'm aware that the consistent (and possibly
even correct) answer is throw/catch. However, for the purposes of a
simple demo application, building Throwables to toss around strikes me
as an unnecessarily obfuscating the mechanics of what I'm
doing. Back
Emacs, 164 characters wide, divided into two
columns, with a full column devoted to the REPL, 20 lines to the
source and 40 to this text. Which raises a question: With the
admonition to not write any function longer than a single screen,
with whose screen length?
For the record, the longest function in this example is 28 lines
and the shortest is 1 line long. That's likely a direct
consequence to my having kept my source scrunched up with only a
little window into it7.
Back
Done via the script:
(save-excursion
(goto-char (point-min))
(re-search-forward "^ *$" (point-max) nil)
(message (mapconcat (lambda (lc) (format "%d" lc))
(let ((lcount nil))
(while (not (eobp))
(forward-sexp)
(setq lcount
(append lcount
(list (count-lines
(save-excursion
(backward-sexp) (point))
(point))))))
lcount)
" ")))
And that's why Emacs one-liners are scary. Because before you
know it, they're actually 15 lines long, all scrunched into an
Eval: prompt. Back
0 comments:
Post a Comment