wet pixels

Spiraling Twist Hat

2010-03-05T20:47:00.004-05:00

This hat is based around a simple pattern of twists, creating a design that spirals around the crown. The goal was to make a hat simple enough to knit without a pattern (on a bus ride) while still being more interesting than simply knitting stockingette forever and ever.

Casting on

Cast on a multiple of 6 stitches. I used 120 stitches on 5.5mm needles to give a fairly loose fit. Join without twisting and mark the beginning of the round.

Brim

Knit 10 rounds of k1, p1 rib. (k1, p1, repeat to the end of the round.)
Knit 2 rounds of stockingette (knit stitch).
Knit 9 rounds of k1, p1 rib.

Crown

The crown is made up of twists, followed by ribbing. On each row of twists, move any markers one stitch to the right. This means that, rather than purling the last stitch of the round before a round of twists, you instead twist it with the first statch of the next round.

Move marker one stitch right. RT (knit into the second stitch on the left needle and, without removing the stitch from the needle, knit into the first stitch, removing both stitches from the left needle.) Repeat, twisting every two stitches together until the end of the round.
k1, p1, repeat to the end of the round.
k1, p1, repeat to the end of the round.

Repeat these three rows four times.

Move marker one stitch right. RT until the end of the round.
k1, p1 to the end of the round.

Repeat these two rows four times.

Move marker one stitch right. RT until the end of the round.
Remove the marker, slip the last stitch on the right needle onto the left needle and place the marker on the right needle. RT until the end of the round.

Decreasing

If you are using circular needles, when it becomes uncomfortable to knit the smaller rounds here, switch to double-pointed needles.

k1, p1, k1, p1, k2tog, repeat to the end of the round.
k1, p1, k1, p1, k1, repeat to the end of the round.
k1, p1, k1, p1, k1, repeat to the end of the round.
k1, p1, p2tog, k1, repeat to the end of the round.
k1, p2, k1, repeat to the end of the round.
k1, p2, k1, repeat to the end of the round.
Move marker one stitch right. k2tog, p2, repeat to the end of the round.
k1, p2, repeat to the end of the round.
k1, p2, repeat to the end of the round.
k1, p2tog, repeat to the end of the round.
k1, p1, repeat to the end of the round.
k1, p1, repeat to the end of the round.
k2tog, repeat to the end of the round.
k1 to the end of the round.
k1 to the end of the round.

At this point, the hat has 1/6 of the original number of stitches. On a 120-stitch hat, this left me with 20 stitches. Repeat the last three rounds until you have ten stitches or fewer, then cut your yarn, draw the tail through the remaining loops and pull tight.

Golden Spiral Blanket - One square metre

2009-10-31T21:45:00.003-04:00

This is the blanket I'm working on, with the eventual goal of having it be a bed-sized spiral.

Like so:

Twirling twist socks - One foot

2009-10-31T12:54:00.002-04:00

This is how far I've got in knitting the socks that woke me up. I'm just about ready to start knitting the heel flap. After that, the cable that features prominently on the foot here will split into two branches, each of which will twirl in opposite directions.

The cables will continue twisting in the same way as the cable on the foot, but will gradually make their way to the back of the sock, cross each other, then come back to the front of the sock again.

And again, the nice thing about this design is that, when it comes time to write down the pattern (which I seem to do after one sock, then follow it for the second) it's actually relatively easy to explain. Or, rather, it'll be easy to write down the exact stitches to make to end up with the same result.

Woken by socks

2009-10-29T21:10:00.002-04:00

Dear knitblog,

I know I've been bad at updating you. I didn't knit from March through October. But I've started again. Working on a blanket now. All placid and black hole. And what I would like to cite as a sign of obsessive attachment:

I was woken up tonight by a pair of socks.

Not just any socks, but a sock pattern pretty enough that I can use my Knitpicks pretty-pretty-pet-it-pretty yarn on. (An xmas gift)

Socks where cables spiral around each other in a manner that is wholly an illusion, and is easy to knit. A pattern that I think I'll be able to actually chart and put up, unlike some of the more, shall we say, esoteric¹ patterns I did in February and March.

The idea is this: A line of c2x2 twist up the foot, splitting into two cables that twist around the ankle, in opposite directions, propelled by KFBs and p2togs.

Anyway, I'm casting on a toe, and I'll see where it takes me!

By which I mean 'needlessly complicated'.

Sierpinski/Koch Scarf - Halfway there

2009-04-04T17:54:00.004-04:00

Now that I'm halfway done this project and no longer routinely swearing and ripping it out, I think it's time to show off my progress.

The project was partially conceived out of a desire to play with triangle-based fractals and partially out of a desire to make myself more comfortable with lace.

The bottom panel is a Sierpinski sieve after three iterations of the algorithm. Each panel goes back one iteration, until nothing remains but a triangle.

I'm currently knitting said triangle. After I'm done that, I'll go forward two iterations into the 2-dimensional Koch curve (or Koch snowflake). At the size I'm knitting, I only have the resolution to knit two Koch curves.

However, the Koch curve pattern (which I've sketched, but not yet plotted out with all its decreases) has the fortuitous property of being half again as large as the Sierpinski triangle. Hence, two Koch curves will be as long as three Sierpinski triangles.

I had a few false starts with the Sierpinski pattern, though. First, I misguessed how to angle the decreases, resulting in a random web of yarn. Second, I saw that once I'd knitted more than half the triangle, the corners of the fabric drooped dramatically. To fix that, I spread out and flattened the scarf, then placed a straight edge from the middle over the drooping corners and noted the row that it intersected. Having done that, I cast on with twice the number of stitches that that row had outside the pattern (twice 14 --$gt; 28). From there, I increased every row without a corresponding decrease, until the scarf was 45 stitches across, the width of the pattern.

I intend to, on the second Koch curve, decrease every pattern row for the last third of the pattern, giving it the opposite curve to the bottom of the scarf, rather than the simple curve that would result from casting off on a flat edge.

Mutations

2009-03-30T16:45:00.004-04:00

A few days ago, I was approached by colorlessblue. She had a pretty sock pattern made up of triangles, but it required her to use exactly 64 stitches around.

Her difficulty was that she couldn't find a gauge in which that worked. Me first reaction was to say, "well, that's not a big deal, the pattern is trivial to make fit on a multiple of 12 or 14 stitches." Then I tried to explain what stitches to remove, and it really quickly became messy.

So I stopped for 15 minutes or so and drew a few patterns. As I started, she clarified that she didn't want to purl.

This is what resulted:

However, the absence of purl stitches meant that the resulting pattern would be triangles of vertical knit stitches with a zigzag border of slanted stitches. So, my next thought was, "Why not make the slanted stitches into the motif. That resulted in this pattern: It's not perfect by any means. The k2tog and SSK being next to each other results in a ladder. Were I continuing that tangent, I'd turn all paired decreases into double decreases. Also, as it turns out, I didn't like how the paired yarn overs at the apex of the triangle looked. At the same time as I was thinking that, I also wondered what would happen if the triangles were offset by half a repeat, giving the whole pattern a longer period before it repeats.

This pattern (which is almost the same as a Barbara Walker one) has the property that the knit stitches actually form themselves into vein-like curves: the diamonds end up looking like oval leaves.

I tried knitting it half in knit and half in purl, to see if that would give the leaves more shape, but that ended up being a step too far: The lace lost enough of its definition that the pattern looked awkward and no longer anything like leaves.

I think I'm likely to use this one at some point, possibly as a motif one some leaf-patterned socks that I keep not getting around to. Before I do, I'm likely to put it through one more mutation, reducing it to about half the size and extending the widest part of the leaf by two rows, so that it more accurately matches the shape of a beech leaf.

Most of my charts seem to follow a similar pattern of starting point and progressive mutation, followed by a test knit, correction of discovered problems, a further test knit and possibly correcting steps that went too far.

It's like saying "just one more row", but "just one more change" instead.

Braid-pattern pullover -- Prelude

2009-03-20T10:59:00.005-04:00

I'd be the first to admit that I think of myself as a sock-and-hat knitter. I like my projects the way I... Never mind, I'm not going to finish that chestnut. Suffice it to say that I like my projects small enough that I can always keep the end in sight.

That said, I picked up two bales of wool this week; bulky natural wool in a pleasant brown. Along with that, I expanded what is apparently becoming an Addi collection. Essentially, I've committed to knitting a sweater.

As is my wont, I started by browsing the internet to determine what kind of sweater I wanted, then searching Ravelry for patterns. As usually happens, I got frustrated. Fortunately, Elizabeth Zimmerman has a pattern, described in proportions, for a warm baggy pullover. Less-fortunately, it's steeked. After more research and not a little bit of swearing, I decided to blissfully close my eyes and pay no mind to those instructions. After all, I have many many many stitches between me and slicing two holes into my knitting.

Plenty of time to chicken out and re-think the pattern as two trapezoids sewn together with the arms attached by picking up off the selvedge. Plenty of time to consider that sewing two trapezoids together will be far less solid than biting the bullet and knitting the whole thing as a tapered cylinder.

Plenty of time to knit swatches.

This is what I'm using as a swatch. This pattern, repeated over and over, is intended to make up two front panels of the sweater. The intent is to swatch it to see if it looks right and also to test texture and gauge of the wool and needles. As I'm more than mildly worried that 5mm needles (my preferred size for worsted) will simply be too tiny for bulky.

After only a little bit of discussion and plotting, I realised that the back of the sweater really deserves a different pattern from the front.

Rather than two panels of four-strand braids, the back has a single panel, centred, that makes a twelve-strand braid or knot. Fortunately, this pattern will look pleasing so long as the front panel does, so I only need to knit the one swatch as a test pattern.

In keeping with my three-project rule, I'm not to start a pair of socks or a hat or anything else until I've finished with this swatch. Further, once I've committed to the sweater, that's one of the three available slots for patterns taken up. So, likely, I'll knit the swatch, knit some socks (I have a pattern that I want to try), possibly knit some more socks (I actually have two patterns waiting for me), maybe finish a hat, and then get to the sweater, just in time for it to be too warm out for sweaters.

Ravelry Hack #1 - Counting downloads

2009-02-18T21:56:00.003-05:00

So, I like Ravelry's designer pages. They're great for indulging my vanity. I tend to check my downloads every once in a while, just out of curiosity, to see if anyone's downloading my patterns.

But one thing it's lacking, and that I'm generally curious about, is how many times they've been downloaded in total. The page doesn't indulge that curiosity. I initially considered putting in a feature request, but then got sidetracked by geekery.

See, the Ravelry downloads page is remarkably readable HTML. Which led me to a quick and easy conclusion: I can fix this in Greasemonkey.

Even having forgotten most of what I learned about the Javascript HTML DOM last time I wrote a Greasemonkey script, it took just over two hours.

Becomes:

With added daily counts on the right-hand column and a "Total Downloads" field at the bottom of the graph.

Installation

If you are using Firefox without Greasemonkey, a compiled Firefox addon is available here. Download it, unzip it and drag it into your Firefox window. It should ask to install, like a normal add-on.

If you have Greasemonkey installed or are using Opera, download the Greasemonkey script file and install the user.js file by:

Firefox: by dragging the file into your Firefox window. Greasemonkey should open, displaying the affected URL and ask you if you wish to install.

Opera: The path where User Javascript files are loaded from is indicated in Preferences/Advanced/Content/Javascript Options. If no path is indicated, choose one. Place the user.js file into that directory.

Knitting is an acceptable Lisp

2009-02-16T17:36:00.008-05:00

So, I'm a knitter. And I pretend to be a designer some days, when I'm not knitting endless swathes of k2, p2 rib. But first and foremost, I'm an unabashed geek.

As such, I took a look at KnitML. And it's neat. Well, in theory, it's neat. In practice, I haven't got the java dependencies set up on my computer yet to play with it. I do intend to do that, and am downloading them, but while I wait for them, I want to ponder through an idea that's been percolating in my brain.

Namely, I think of knitting patterns in two ways. One is the chart. This is how I write patterns, really. And given the choice between a chart and a list of stitches, I'll pick the chart every time.

It's a question of information density. The chart presents all the information that I need, in the least amount of space while still being human-readable.

So, let's look at a chart.

It's a fairly simple pattern, embedded moss stitch rib. And can be easily transformed into a written pattern:

p2, k2, p1, k1, repeat to end of round
p2, k2, p1, k1, repeat to end of round
p2, k1, p1, k2, repeat to end of round
p2, k1, p1, k2, repeat to end of round

This is where my brain starts making clicky noises. See, my immediate reaction to that is that it's a simple program. One that generates a handsome little ribbing.

However, my second line of thought runs straight for a programming principle: Don't Repeat Yourself. My process for generating a pattern is to draw charts in Inkscape, test those charts by knitting them, then translate the chart into written instructions. In future, I may also want to make my charts available in KnitML.

What this immediately says to me as a programmer is that I should have a single piece of data from which I can generate:

PNG graphics
This is the format I use to show charts. It's a useful format in that it does a good job of reducing low-colour line drawings (like knitting charts) to a fairly small file size with no loss of detail or crispness.
This is an even worse format to use as a data source, though. Rather than just interpreting the XML that SVG gives, I'd have to start into OCR territory.
SVG graphics
This is the format that I actually use to make charts. Unfortunately, it would be a considerable effort to use it as a starting point: Anything more complicated than a knit or purl stitch are relatively complex "groups", rather than simple objects.
However, they work admirably as a way to draw charts, and from my perusal of the spec, should be able to be drawn programmatically fairly easily.
HTML lists
This actually isn't a bad starting point. Not good enough that I'd want to use it as a data source, but not bad enough, either, to discard it out of hand.
Specifically, if I look at the HTML describing the pattern:
<ol>
  <li> p2, k2, p1, k1, repeat to end of round
  <li> p2, k2, p1, k1, repeat to end of round
  <li> p2, k1, p1, k2, repeat to end of round
  <li> p2, k1, p1, k2, repeat to end of round
</ol>
Other than the <ol> and <li> tags, this actually isn't a bad representation of the data at all.
Plaintext
What if I were to use the plaintext representation of the aforementioned list:
p2, k2, p1, k1, repeat to end of round
p2, k2, p1, k1, repeat to end of round
p2, k1, p1, k2, repeat to end of round
p2, k1, p1, k2, repeat to end of round

That's actually, as I alluded before, a programmatic representation of knitting. And, were it not for that pesky "repeat to end of round", would be nearly perfect.
In fact, this is relatively close to what KnitML uses as a human-readable format, according to my reading of the spec.
S-Expressions
These little gems are the core of Lisp, and are easily transformable to and from XML if the need arises.
The core elements to them are lists and atoms. A list is multiple atoms surrounded by parentheses and separated by spaces. An atom is some text representing something.
Thus, the pattern described above becomes:
(repeat 2
(repeat-to-end (p2 k2 p1 k1)))
(repeat 2
(repeat-to-end (p2 k1 p1 k2)))

And any Lisp programmer reading this just started twitching. As that's not properly-formed Lisp. On the other hand, it does represent a very interesting way to mix data and instructions.
Patterns become lists of stitches and transformative operations performed on those stitches. It's knitting as a pseudo-mathematical notation.

So, taking s-expressions as a base, what does it all mean?

Well, in that list of representations, the further along I went, the more the representation changed from being a depiction of the data to a representation. By the time we reach the plaintext version of the pattern, it is recognisable as a rudimentary program.

The s-expression version isn't even a rudimentary one; if it were fed into an interpreter that recognised the words and symbols used, it is a program to generate a knitting pattern.

Now that offers some interesting prospects, because the output of the interpreter by no means has to be fixed. For instance, I could have one interpreter that reads the pattern as a way to generate a plain text file, another interpreter that generates an SVG graphic and a third that reads the pattern as a way to generate the equivalent KnitML file.

Why does all this matter? Well, I've become interested, in the past weeks, in the idea of an editor where I type in a row of stitches and it appears on the screen as a graphical pattern. Thinking about how to represent stitches is a first step on the road to knitting pattern zen.

Swatch, knit thyself

2009-02-14T11:49:00.003-05:00

Someday, I'm going to learn to knit tiny swatches. The first one I did for this hat was enormous, the size of a tea towel, but highlighted more than a few mistakes that I was making.

Luckily, the cable pattern is composed of repeated motifs. So, seeing these motifs work once was enough to know how they work throughout.

Unfortunately, my first sketch for the lace pattern was done somewhat foolishly. I believed that I could guess, from the pattern, what it would look like, conveniently forgetting that lace isn't just a pattern of holes: it also relies on a structure of slanted and upright stitches.

What I'm trying to say is that my first sketch was uglier than ugly. I've revised it now, and on paper, it looks nice. In practice, I don't know. So I'm doing the sensible thing this time and knitting it again.

There needs to be a pattern to the decreases anyhow, and I've no idea what that'll be. Thus, he sensible thing is 30 rows of swatch, a lifeline and then some puzzling out what decreases go where.

Why I like circular knitting

2009-02-13T14:26:00.004-05:00

So, I'm swatching a project. A project that's 2/3 cabling and 1/3 lace. A project where there's at least one stitch pattern that's heavily used and may not even be possible.

And as is my wont, perhaps foolishly, I'm swatching it flat. 20 rows, 51 stitches. Which is fine for the lace pattern: It was meant to be flat and I can live with that. The cable pattern is much worse.

It actually looks reasonable enough charted in the round. I have a very rough sketch, written before I realised a few things. Flat, though, that's a different story.

I'm deliberately only knitting a fraction of the pattern as a swatch, and that still took me an hour and a half to draw. It's going to be fun when I draw the entire pattern, seeing as that chart is 1/125th of the pattern.

At least the pattern will be flat.

Well, after realising that I was going to be working the chart forever and ever, I re-charted the lace pattern to run 30 rows, the minimum possible to see the pattern working.

The cable pattern is only 20, but if I'm feeling ambitions, I'll extend it, just to see the whole sweep of the pattern. Unfortunately, this actually makes the swatch larger. Fortunately, only by 100 stitches.

Shadow Check Socks - Pattern

2009-02-12T16:01:00.007-05:00

Well, they took two weeks to get together, but I'm happy to say that these socks are finally a pair. Happily, they are indeed as thick, dense and warm as I'd hoped when I first started planning for them.

Requirements

300m sport weight yarn
3.5mm double-pointed needles
3.5mm straight needles
A crochet hook

Measurements

The circumference of the ball of the foot.
The length of the foot.
The circumference of the ankle.
The circumference of the middle calf.
Your gauge in circular heel stitch.
Your gauge in seed rib.

For the gauges, fear not. You'll be knitting a number (quite a number, in fact) of rounds of those stitches before you need to know gauge, so it's simple to measure directly on the sock itself.

Instructions

Casting On

Perform a provisional cast on of eight stitches, knitting four rows, as for the Simple Toe-Up Socks

Heel Stitch Toe

Stitches between {braces} are repeated the number of times indicated. Stitch counts are in brackets and indicate the number of stitches on both the top and the bottom.

Knit this pattern until you have enough stitches around to go snugly around the ball of the foot. End on an even-numbered round.

Knit the entire round.
Top: k1, kfb, k4, kfb, k1 (10)
Bottom: k1, kfb, k4, kfb, k1
Top: k2, {s1 wyib, k1}(3 times), k2
Bottom: k2, {k1, s1 wyib}(3 times), k2
Top: k8, kfb, k1 (11)
Bottom: k1, kfb, k8
Top: k2, {s1 wyib, k1}(4 times), k1
Bottom: k1, {k1, s1 wyib}(4 times), k2
Top: k1, kfb, k7, kfb, k1 (13)
Bottom: k1, kfb, k7, kfb, k1
Top: k3, {s1 wyib, k1}(4 times), k2
Bottom: k2, {k1, s1 wyib}(4 times), k3
Top: k11, kfb, k1 (14)
Bottom: k1, kfb, k11
Top: k3, {s1 wyib, k1}(5 times), k1
Bottom: k1, {k1, s1 wyib}(5 times), k3
Top: k1, kfb, k10, kfb, k1 (16)
Bottom: k1, kfb, k10, kfb, k1
Top: k2, {s1 wyib, k1}(6 times), k2
Bottom: k2, {k1, s1 wyib}(6 times), k2
Top: k14, kfb, k1 (17)
Bottom: k1, kfb, k14
Top: k2, {s1 wyib, k1}(7 times), k1
Bottom: k1, {k1, s1 wyib}(7 times), k2
Top: k1, kfb, k13, kfb, k1 (19)
Bottom: k1, kfb, k13, kfb, k1
Top: k3, {s1 wyib, k1}(7 times), k2
Bottom: k2, {k1, s1 wyib}(7 times), k3
Top: k17, kfb, k1 (20)
Bottom: k1, kfb, k17
Top: k3, {s1 wyib, k1}(8 times), k1
Bottom: k1, {k1, s1 wyib}(8 times), k3
Top: k1, kfb, k16, kfb, k1 (22)
Bottom: k1, kfb, k16, kfb, k1
Top: k2, {s1 wyib, k1}(9 times), k2
Bottom: k2, {k1, s1 wyib}(9 times), k2
Top: k20, kfb, k1 (23)
Bottom: k1, kfb, k20
Top: k2, {s1 wyib, k1}(10 times), k1
Bottom: k1, {k1, s1 wyib}(10 times), k2
Top: k1, kfb, k19, kfb, k1 (25)
Bottom: k1, kfb, k19, kfb, k1
Top: k2, {s1 wyib, k1}(10 times), k2
Bottom: k2, {k1, s1 wyib}(10 times), k2
Top: k23, kfb, k1 (26)
Bottom: k1, kfb, k23
Top: k11, kfb, k1 (14)
Bottom: k1, kfb, k11
Top: k3, {s1 wyib, k1}(5 times), k1
Bottom: k1, {k1, s1 wyib}(5 times), k3
Top: k1, kfb, k10, kfb, k1 (16)
Bottom: k1, kfb, k10, kfb, k1
Top: k2, {s1 wyib, k1}(6 times), k2
Bottom: k2, {k1, s1 wyib}(6 times), k2
Top: k14, kfb, k1 (17)
Bottom: k1, kfb, k14
Top: k2, {s1 wyib, k1}(7 times), k1
Bottom: k1, {k1, s1 wyib}(7 times), k2
Top: k1, kfb, k13, kfb, k1 (19)
Bottom: k1, kfb, k13, kfb, k1
Top: k3, {s1 wyib, k1}(7 times), k2
Bottom: k2, {k1, s1 wyib}(7 times), k3
Top: k17, kfb, k1 (20)
Bottom: k1, kfb, k17
Top: k3, {s1 wyib, k1}(8 times), k1
Bottom: k1, {k1, s1 wyib}(8 times), k3
Top: k1, kfb, k16, kfb, k1 (22)
Bottom: k1, kfb, k16, kfb, k1
Top: k2, {s1 wyib, k1}(9 times), k2
Bottom: k2, {k1, s1 wyib}(9 times), k2
Top: k20, kfb, k1 (23)
Bottom: k1, kfb, k20
Top: k2, {s1 wyib, k1}(10 times), k1
Bottom: k1, {k1, s1 wyib}(10 times), k2
Top: k1, kfb, k19, kfb, k1 (25)
Bottom: k1, kfb, k19, kfb, k1
Top: k2, {s1 wyib, k1}(10 times), k2
Bottom: k2, {k1, s1 wyib}(10 times), k2
Top: k23, kfb, k1 (26)
Bottom: k1, kfb, k23

Knitting the body of the sock

First, you'll need to divide the stitches again, this time onto three needles.

Count the stitches on needles 3 and 4 and divide it by three, rounding down. This is the number of stitches that should be moved from the sole to the sides and top of the sock. For the rest of the pattern, these will be referred to as “side stitches”.

For example, I have 25 stitches on needles 3 and 4 combined:

25 / 3 = 8 1/3
8 1/3 ≈ 8

So I want to move eight stitches, or four from each side. First, transfer all the stitches from needle 4 to needle 3. Needle 3 will knit the sole of the sock, needles 1 and 2 will knit the top. Next, place a marker at the end of needle 1 and transfer half of the side stitches from needle 3 to needle 1. Place a marker at the end of needle 2 and transfer the other half of the stitches from needle 3 to needle 2. If you are moving an odd number of stitches, move the extra stitch to the needle that has fewer toe increases (needle 1 for a right sock, needle 2 for a left.)

Count the stitches on needle 3 again. If there is an odd number of stitches, perfect. If, on the other hand, it's even, transfer one more stitch. If you transferred an odd number before, place this extra stitch on the opposite needle from the odd stitch. Otherwise, place it on the needle with fewer toe increases, as above.

The pattern for the body of the sock is 6 rounds long. Stitches in {braces} are repeated, either to the end of needle 3 (for bottom stitches) or until the end of needle 2 (for top). The side stitches are knit in pattern for the top.

Top: {Knit 1}
Bottom: {Purl 1}
Top: {k1, p1}
Bottom: p1, {k1, p1}
Top: {p1}
Bottom: p1, {yop, s1 wyib, purl 1} Here, the yarn should wrap over the slipped stitch, making one stitch crossed over the other.
Top: {k1}
Bottom: p1, {drop the yo from the previous row, si wyib, p1} Dropping the yarn over looks like it should leave a very loose stitch around the slipped stitch. However, purling the next stitch adjusts its tension, bringing it tighter once more.
Top: {p1, k1}
Bottom: p1, {s1 wyib, p1}
Top: {p1}
Bottom: p1, {s1 wyib, p1}

Knitting a gusseted heel

This is a slight adaptation of the heel described in the Simple Toe-Up Socks pattern.

Continue knitting the 6 rounds of pattern for the body until it measures 70% of the length of the foot. For me, each pattern repeat is less than a centimetre long, so you can expect this to be rather a lot of knitting. My foot is 22cm long, so I calculate that as follows:

22 cm ∗ 0.7 = 15.39 cm

Finish on round 6 of the body pattern, having just knit the bottom of the sock.

For the next while, needles 1 and 2 will not be used, so do what you will with the stitches on them, be it transferring them to a holder, placing them on a spare needle or scrap of yarn or simply leaving them on the two double-pointed needles.

The heel is knit flat and the yarn is currently placed so that the next row is the wrong side, so, to complete the shadow check pattern, knit the row, which will finish the sole with a row of purl stitches. Then, with straight needles of the same size as your double-pointed needles, knit the following 2-row pattern:

Heel Stitch

(right side) k1, {s1, k1}
Purl the row

This is heel stitch again. However, since there are no increases or decreases at play, it's 16 times simpler than it was at the toe.

Knit this until the heel is 95% of the length of the foot (i.e. 4-6 rows below the foot length).

Turning the heel

Next, knit row 1 of the heel stitch pattern, stopping 2 stitches before the end of the row. Turn the sock and knit row 2, stopping 2 stitches before the end of the row.

Knit row 1 again, stopping 4 stitches before the end of the row, then row 2, stopping 4 stitches before the end of the row.

Continue like this, losing two more stitches each row, until you have either 9 or 7 stitches left live and have just completed a purl row (row 2).

Next, knit row 1 again, knitting the last live stitch together with the first saved stitch and knitting the second saved stitch. Turn your work and knit row 2, purling the last live stitch together with the first saved stitch and purling the second saved stitch.

Repeat those two steps until all the stitches are live again. Congratulations. You're halfway done the heel.

Picking up from the heel flap

You should have just completed a purl row so, to get your yarn in position to knit in the round, knit across the heel one more time, this time using two double-pointed needles, one holding each half of the heel.

Begin picking up stitches along the side of the heel flap with the second double-pointed needle used to knit the heel. If you look at the selvedge, you should have a raised row of slipped stitches below a slightly-curled row of knit stitches. You'll want to pick up one stitch per row along the heel, so insert a needle in the hole beneath the two strands of the knit stitch, insert your double-pointed needle into the hole underneath it and knit one stitch.

Repeat this for every row of the heel, as well as picking up two stitches from the first stitch on the top of the sock in order to ensure that there's not a hole between the heel and the top of the sock. Note the number of stitches you picked up, place a marker at the end of the needle and knit the side stitches onto it. This keeps you from working the decreases across a gap between needles.

Knit across the top of the sock, transferring stitches onto two double-pointed needles if needs be. With the second double-pointed needle, pick up the same number of stitches along the heel flap as you did on the other side, leaving a marker before the picked-up stitches.

Finally, transfer the picked-up stitches and the side stitches to the other needle on the back of the sock. Time for the final change in needle numbering. As before, the needles are numbered in clockwise order. The two across the top of the foot are needles 2 and 3. The two making up the heel are 1 and 4.

That's it. The heel has been turned, the stitches have been picked up. All that's left is to decrease the heel gusset until the sock is the same size as it was at the ball of the foot.

Gusseting

Compared to everything you've been through so far, this bit is easy. Knit each round using the seed rib pattern. To determine where in the pattern to start count the number of stitches on needle 1. If it's even, start the pattern on round 1, if it's odd, start on round 4.

Two stitches before the marker on needle 1, stop. If you are on a knit round, K2TOG. If you are on a purl round, P2TOG. If you are on a seed round, do not decrease.

Continue knitting seed rib until you get to the marker on needle 4. If you are on a knit round, SSK. If you are on a purl round, S1, P1, PSSO. If you are on a seed round, do not decrease.

Continue knitting this pattern until you have the same number of stitches as you did for the ball of the foot.

Seed Rib

Knit the round
k1, p1, repeat
Purl the round
Knit the round
p1, k1, repeat
Purl the round

The half above the black line is the normal pattern repeat. The part below is the general pattern for the gusset decreases.

Ankle

If the ankle is to be smaller than the ball of the foot, continue knitting seed rib for three rounds. Once you have done that, work decreases as follows:

If you are on a seed round, do not decrease.

If you are on a knit round, knit the first stitch of needle 1, SSK, knit to the last three stitches on needle 4, K2TOG, K1.

If you are on a purl round, purl one stitch, S1, P1, PSSO, purl to the last three stitches on needle 4, P2TOG, P1.

Decrease in this way until you have the right number of stitches for the ankle.

Calf

Knit four repeats of the seed rib pattern. Count your stitches. If they are not divisible by 6, increase via KFBs in the knit rounds as you knit rounds 1-5 of a fifth repeat. Otherwise, knit those rounds without increase.

That done, it's time to begin ribbing. The pattern for the rib is another Barbara Walker one: embedded moss stitch ribbing, with a single amendment. It'll be worked over 6 stitches, rather than 7, to ensure that its repeat length matches that of the seed rib.

Embedded Moss Stitch Ribbing

Repeat these 6 stitches around the entire circumference of the sock.

p2, k2, p1, k1
p2, k2, p1, k1
p2, k1, p1, k2
p2, k1, p2, k2

This gives a rib that is a 2-stitch purl gutter between columns of knit stitches, with a twocolumn moss stitch pattern between them.

Knit this ribbing until the sock is as long as you want it, knit two rounds in P2, K4 ribbing, then cast off very loosely with the following pattern: P2, K4.

Shadow Check Socks - One down!

2009-02-08T16:21:00.003-05:00

Well, I got lucky. My first knit of the ankle and calf turned out to be exactly what I wanted and my lifeline ended up being wholly unnecessary. So, with only a dozen metres or so of yarn left on my first ball, I turn to the second and begin knitting the second sock.

This becomes helpful proofreading, as for the second sock, I follow the pattern that I wrote while knitting the first. If it matches, I post the pattern as-is. If not, I edit the erroneous parts of the pattern.

While I was terribly happy to open Ravelry and mark the pattern as 50% done, I recognise that the second sock is likely to take almost as long as the first, as the slowness wasn't due to revising, but rather because the sock was simply a hell of a lot of knitting.

Stitch Patterns - Charts

2009-02-07T15:36:00.008-05:00

In a little more thinking aloud, I charted the stitch patterns that I'm currently thinking about. The sensible next step would be to swatch them, but I'm going to be daft and simply lifeline my socks in another few pattern repeats and use them as the swatch.

Because I'm thinking about these patterns for socks, they're all given as circular directions, rather than flat.

Seed Rib

The half above the black line is the normal pattern repeat. The part below is the general pattern for the gusset decreases.

Stitches with a green background are repeated.

Mistake Stitch Rib

This pattern uses a multiple of 4 stitches plus 3. The stritches in green are repeated.

Embedded Moss Stitch Rib

The six stitches of this pattern are repeated throughout.

ETA (08/02/2009): Updated with better chart images.

ETA (12/02/2009): Changed the Seed Rib pattern, because the pattern of decreases in the seed rows was near-impossible to explain.

Stitch patterns

2009-02-07T13:07:00.004-05:00

So, these socks take three, possibly four stitch patterns: heel stitch, shadow check, a stitch whose name I can't find, but will call seed rib and a pattern for the ribbing along the calf.

Shadow check and heel stitch are described in my original post about the shadow check socks. The unnamed stitch, which I shall, in future, call seed rib, is two rows garter, one row seed, two rows garter and one row seed (in the opposite pattern to the previous seed row).

My first humbling lesson was that heel stitch is incredibly difficult in the round. I've played with that some more, and in my working pattern, have managed to simplify the pattern for the toe dramatically.

My second was that shadow check draws the fabric far too tight to have any stockingette in the same row with it. That realisation led directly to my changing the seed rib from a part of the top to an all-over pattern, continuing it along the heel gusset. The decision to include it over the gusset created its own problems, though.

When decreasing for the gusset, to keep the pattern shrinking at an appropriate rate, there need to be decreases every round. The problem with that is that every third row is seed, and the decreases need to happen in pattern. Approaching the decreases, there are two possible patterns: k1, p1, k1, (marker) p1, k1 or p1, k1, p1, (marker) k1, p1. As you can see, either one of those patterns means that a 2-stitch decrease will double up a stitch. Taking the first as an example, if you k2tog, you end up with k1, k2tog, (marker) p1, k1 and if you p2tog, you have the opposite problem: k1, p2tog, (marker) p1, k1.

After several rounds of deliberation, I came up with the following pattern for decreasing the gusset in seed rows:

Knit in pattern to two stitches before marker.
If the last stitch was knit, k2tog. If it was purled, p2tog.
Knit in pattern to the second marker.
If the last stitch was knit, s1, p1, psso. If it was purled, ssk.
Knit in pattern to the end of the round.

This ensures that the paired stitches match up nicely. Its biggest advantage, though, is that it stops what I was doing before, namely looking at the previous seed row and guessing based on that.

I still haven't decided what to do for the ribbing around the calf, though. Conceivably, I could knit the lot in seed rib, but I think I'd feel that I was ducking out. I expect and anticipate that I'll end up increasing to a multiple of five stitches and relying on that lovely standby, slip-stitch rib.

On the other hand Barbara Walker has two patterns that intrigue me. One is mistake-stitch ribbing, which looks interesting from the photos, but would have to be rather altered to work as a circular pattern. What it is is k2, p2 rib, worked on one stitch less than the needed multiple of 4 stitches, so that there is a column of knit surrounded by jagged knit and purl stitches.

The other is embedded moss stitch ribbing. That stitch pattern actually fits, interestingly, with seed rib. What it is is a pair of knit stitches enclosing two moss stitches. Converting that would be easy and looks like it'll fit with the overall texture.

Shadow Check Socks -- Time to start the heel flap

2009-02-06T11:35:00.009-05:00

So I spent two days playing with blogger, poking at layouts and widgets, all the while glancing over at the toe of the sock with no small amount of frustration. The sock was giving me a slight case of knitter's block, not helped by my ballpark calculation that I needed 600 stitches for each centimetre of sock.

Calling them Simple Textured Socks had passed from misnomer to outright lie. The pattern may be 5 stitches and six rows, but it requires enough attentiveness that I was unable to knit it while reading (my habit for stockingette and cables).

I finally put on a TV show, buckled down and started knitting, and was quite pleased to find that the interesting nature of the textures was carrying me along. As I knitted, I was intensely curious to see how the two patterns would grow.

I'm not sure whether to call this sock pattern beautiful or ugly. "Idiosyncratic" certainly describes it. The soles are dense but soft, about three times as thick as knit stitch. Even when flattened, the fact that the sock is so very dense and thick means that it's rounded rather than sitting flat.

The uppers are about half the thickness of the soles, but have an amazing amount of stretch to them. Horizontally, they can stretch to double their width and vertically to half again their size.

As well, and thankfully, the upper part of the sock gathers tightly in the vertical direction. Since the soles take 6 rows to get the same amount of length as two rows of knit stitch, they are incredibly tight in the vertical sense. Horizontally, they have the same density as heel stitch, something that I noted with my test swatch, and am using to my advantage.

For these socks, I've been adopting my usual modus operandi, namely knitting enough of the pattern that I have a clear picture of how it should look, then typing out an actual pattern and following it, amending mistakes and missteps as I go.

To date, there have been three changes to my pattern as written:

I initially said that some stitches along the sides of the socks should be knit in stockingette, rather than working them in pattern. I was wrong about that; it resulted in two columns of stitches that were trying to grow vertically at a rate three times that of the rest of the sock, resulting in unfortunate bulges.
Because I was following my Simple Toe-Up Socks pattern, I indicated several rounds of decreases between the ball of the foot and the instep. As it turns out, knitting a dense and stretch material neatly removes the need for that shaping, while making the socks' fit quite close.
I had planned to knit the gussets and the calf in simple knit stitch, but it's becoming evident to me that changing the stitch that dramatically would look bad.
At the moment, I'm vacillating between four choices:
1. Knit the gusset and the calf in garter stitch.
  Simple, straightforward, but I worry that it'll be too plain.
2. Knit the gusset in stockingette but knit all stitches that are not decreased as a part of the gusset in garter. This will eventually mean that the back garter stitches and the front garter stitches meet, making the calf a continuous band of garter stitch.
  This adds some decoration to the previous pattern, but I'm afraid that what happened before with combining stockingette and a tighter stitch will recur, leaving the gusset to bulge outward.
3. Knit the gusset and calf in the same pattern as the upper part of the sock, decreasing in pattern.
  The only reason to not do this is that it continues the pattern of this sock requiring careful attentiveness. In all honesty, I should bite the bullet, suck it up and knit the pattern.
4. Knit the upper and back of the sock in pattern, but the gusset in stockingette.
  I'm nearly certain that this idea will fail due to bulgy, unsightly gussets.
What I'm actually going to do is run a lifeline through the stitches once I've picked up off the edges of the heel flap, as, fortunately, all four of my possibilities start with one round of knit stitch.

Upper pattern
This is the as-yet-unnamed stitch that I adopted in order to keep the fabric throughout the sock in similar tension. It comes out as two rows of garter, making a single garter ridge, followed by one row of seed. If you look carefully, you can see that the stitch pattern in the seed rows alternates, making this a 6-row pattern. I decided on this for the upper part of the socks in order to have a stitch that was elastic in both directions but gathered vertically. And by "decided", I mean that I guessed at a texture and happened to get one that's quite functional.
Shadow Check - Right Side
This shows the outer part of the soles. The right side of the shadow check pattern is a row of purl stitch, a stretched slip stitch sitting atop a purled background, followed by another row of slip stitch. This pattern has next to no give in either direction, because the stitches that are slipped across five rows pull it very tight. Below the shadow check pattern is the toe of the sock, showing how the raised (slipped) columns of heel stitch match up exactly with the slip stitches in the shadow check pattern.
Shadow Check - Wrong Side
This is the wrong side of the shadow check pattern and why I picked it in the first place. The first picture is deliberately taken with the lighting highlighting the incredible depth of the texture. The pattern makes the sole of the sock incredibly dense and soft.

Dogwood Cabled Hat -- Pattern Amendments

2009-02-04T15:26:00.005-05:00

I've made a little change to the instructions: In decreasing the ribbing, I said to p1, s1, psso. That's not really possible. I meant to say s1, p1, psso.

Also, I noticed that the pattern was written with my old method of typing stitches into a spreadsheet. Since then, I've set up Inkscape with a template SVG document that does a far far better job at giving me good-looking knitting patterns.

So, while updating the PDF for the dogwood cabled hat, I added in a proper graphical pattern, which looks far far prettier than the previous ASCII-art one.

Simple Textured Socks -- Rip number 3

2009-02-02T21:31:00.003-05:00

Well, I've ripped these things three times. One because I realised that I wanted to do the toe in heel stitch. Two because circular heel stitch while increasing is harder than I thought.

So, in the interests of not ripping a fourth time, I'm going to write out one full repeat of the heel stitch-while-increasing pattern.

It uses the needle numbering I've used elsewhere, where the two needles on top are 1 and 4, and the ones on the bottom are 2 and 3.

Heel Stitch Toe

First, knit four rows from a provisional cast on of 8 stitches and divide the 16 stitches between four needles. You'll be on needle 4, so knit that needle to get to needle 1.

Knit the entire round.
Needle 1: k2, kfb, k1 (5)
Needle 2: k1, kfb, k2 (5)
Needle 3: k2, kfb, k1 (5)
Needle 4: k1, kfb, k2 (5)
Needle 1: k1, s1 wyib, k3
Needle 2: k3, s1 wyib,, k1
Needle 3: s1 wyib, k1, s1 wyib, k2
Needle 4: k2, s1 wyib, k1, s1 wyib
Needle 1: k3, kfb, k1 (6)
Needle 2: k1, kfb, k3 (6)
Needle 3: k5
Needle 4: k5
Needle 1: k1, s1 wyib, k1, s1 wyib, k2
Needle 2: k2, s1 wyib, k1, s1 wyib, k1
Needle 3: s1 wyib, k1, s1 wyib, k2
Needle 4: k2, s1 wyib, k1, s1 wyib
Needle 1: k4, kfb, k1 (7)
Needle 2: k1, kfb, k4 (7)
Needle 3: k3, kfb, k1 (6)
Needle 4: k1, kfb, k3 (6)
Needle 1: k1, s1 wyib, k1, s1 wyib, k3
Needle 2: k3, s1 wyib, k1, s1 wyib, k1
Needle 3: s1 wyib, k1, s1 wyib, k3
Needle 4: k3, s1 wyib, k1, s1 wyib
Needle 1: k5, kfb, k1 (8)
Needle 2: k1, kfb, k5 (8)
Needle 3: k6
Needle 4: k6
Needle 1: k1, s1 wyib, k1, s1 wyib, k1, s1 wyib, k2
Needle 2: k2, s1 wyib, k1, s1 wyib, k1, s1 wyib, k1
Needle 3: s1 wyib, k1, s1 wyib, k3
Needle 4: k3, s1 wyib, k1, s1 wyib

The rules are:

Knit on increase rows.
Needle 1 starts on a knit stitch.
The last two stitches on the needle are never slipped, otherwise you get holes.
Needle 2 is needle 1 knit backwards (ends on a knit stitch).
Needle 3 starts on a slip stitch.
Needle 4 is needle 3 knit backwards (ends on a slip stitch).

I think, when I write this pattern out in full, that I'll write it from 4 stitches on each needle through to 60 stitches around, simply because it's not a pattern that follows easily.

ETA (7/2/2009): I've considered and toyed with this pattern more since noting it down, and the correct answer is that I'm numbering the needles wrong. Realistically, 1 and 2 should be the top of the foot, 3 and 4 the bottom.

I worked that out when trying to chart the pattern and not wanting to either write a chart with directions like "start at the middle, read right to left for needle 1, read left to right for needle 2 and 3, then right http://www.blogger.com/img/blank.gifto left for needle 4, stopping at the middle" or to write a separate pattern for each needle.

By changing how I thought of the needles, I was able to write a single chart for the top and the bottom, presented as two charts, as the bottom is the enantiomer of the top.

Deliberately, this chart goes on past what I reasonably expect as sizes, just to avoid having to try to give the extrapolation of the pattern as a process, as the algorithm for generating the toe relies somewhat on anticipating the next line.

Simple Textured Socks

2009-02-02T15:32:00.008-05:00

This is meant as an alteration to my simple toe-up socks, but will en up with its own pattern when I'm done.

Since knitting them, I've been thinking about how to improve the pattern, and have realised that I'd like the soles thicker and softer. Barbara Walker has a stitch pattern called shadow check on page 103 of A Treasury of Knitting Patterns.

It's a lovely slip-stitch pattern, with a beautiful wrong side. The slipped stitches make it exceptionally dense, but the way it works makes it quite soft. Better still is that it can be worked over the same number of stitches as heel stitch.

However, there's a catch. As with all of her patterns, shadow check is given as a flat pattern with a right and a wrong side. Further, it doesn't easily translate to knitting in the round, at least, not easily enough that I can do it in my head, so I'm jotting down the stitch pattern, converted to circular knitting, so that I can refer to it when needed.

Shadow check (circular)

Odd number of stitches. Stitches in {braces} are repeated.

{Purl 1}
Purl 1, {knit 1, purl 1}
Purl 1, {yarn over, slip 1 with yarn in back, purl 1}
Purl 1, {drop the yarn over from the previous round, slip 1 with yarn in back, purl 1}
Purl 1, {slip 1 with yarn in back, purl 1}
Purl 1, {slip 1 with yarn in back, purl 1}

That pattern will make up the sole of the sock, with the right side facing out and the dense, corrugated wrong side facing in as padding. Because this will make the sole denser than simple knit stitch, I plan on having a small amount of ribbing on the top of the sock, to give the whole thing some give.

The heel of the sock will be made from standard heel stitch, which lines up exactly with shadow check and begins with a row of purl, which will serve nicely to complete the previous pattern.

Heel Stitch

Odd number of stitches. As this is worked flat, it has a right and a wrong side.

(Wrong Side) {Purl 1}
Knit 1, {slip 1 with yarn in back, knit 1}

Again, this is denser than simple knit stitch, but because the heel is knit flat and then gusseted, the only significant difference will be that there will be more decreases along the gusset.

Because slip-stitch patterns look really interesting when worked in multiple colours, rather than doing what I did before and knitting the toe, heel and cuff in self-striping yarn and the foot and ankle in plain, instead I'm going to use self-striping yarn for the whole thing.

Like everything, it's an experiment, but the only downside I see to this project is going to be that I can't work the sole while reading a book. Anyhow, time to cast this project on.

Two-Piece Knotwork Hat

2009-02-01T23:47:00.009-05:00

Well, after six attempts at the brim and two test-knits plus one failed attempt at the body, this hat came together.

It began as a simple concept: I wanted to make a hat that had a single braid pattern running around the brim. From that, form ollowed design. I knew that I'd have to graft the brim into a loop and pick up stitches.

That said, when starting, I noted grafting as something that I'd have to learn and had no clue what pattern I'd attach to the body of the hat. In the end, I failed at grafting but learned a lot and ended up knitting four patterns for the body before finding one that wasn't overwhelming.

The hat itself has the shift in density that I wanted, though. Around the ears, the fabric is thick, dense and warm. Atop the head, it flates out just slightly and sits loosely on my hair, hopefully preventing it from getting tangled and staticked by the hat.

Requirements

150-175 metres of worsted weight yarn, preferably in two colours.
4mm straight needles.
6mm circular needles and/or double-pointed needles.
Circular or double-pointed needles in a size smaller than 4mm.
A tapestry or sewing needle with an eye large enough to accommodate worsted weight yarn.

Instructions

Brim

Casting on

Cast on onto 4mm straight needles using a crocheted provisional cast-on of 22 stitches. Crochet a chain, placing the yarn behind a needle after every crochet stitch and pulling it through the loop on the hook such that it wraps around the needle. Chain a few more stitches and then pull the yarn through the loop. This gives a provisional cast-on that can be easily undone later on.

Knit the following pattern (brim chart):

(wrong side) p2, k2, p6, k2, p6, k4
(right side) p4, k4, c2l, c2r, k4, p2, k2
p2, k2, p4, k1, p4, k1, p4, k4
p4, c2x2l, p1, c2x2r, p1, c2x2r, p2, k2
p2, k2, p4, k1, p4, k1, p4, k4
p4, k4, c2r, c2l, k4, p2, k2
p2, k2, p6, k2, p6, k4
p4, k2, c2x2r, p2, c2x2l, k2, p2, k2
p2, k2, p6, k2, p6, k4
p4, k4, c2l, c2r, k4, p2, k2
p2, k2, p4, k1, p4, k1, p4, k4
p4, c2x2l, p1, c2x2l, p1, c2x2r, p2, k2
p2, k2, p4, k1, p4, k1, p4, k4
p4, k4, c2r, c2l, k4, p2, k2
p2, k2, p6, k2, p6, k4
p4, k2, c2x2r, p2, c2x2l, k2, p2, k2

Note here that the only difference between rows 1-8 and rows 9-16 is that on lines 4 and 12, the centre cable slants the opposite way.

Knit this pattern until, at the end of a set of 16 rows, the flat band that you are knitting is large enough to fit around the ears. Knit row 1 again. Cut the yarn, leaving a 1 metre long tail.

Grafting

Untie the knot at the end of your crochet chain and tug on the tail of yarn to undo it. As it releases stitches, pick them up on your free needle. The stitches may be slightly twisted here, so make sure before grafting that they are in the order specified by row 1 of the pattern.

Holding up the two needles, they should be facing the same way without twisting the brim. Turn them so that the brim forms a loop with the wrong side facing out.

You'll need to graft the stitches as they lie, using knit grafting (kg) for the knit stitches and purl grafting (pg) for the purls. Or, in other words, kg 4 times, pg 6 times, kg twice, pg 6 times, kg twice, pg twice.

Body

Picking up

Hold the looped brim so that the k2, p2 rib is at the top and the p4 stockingette is at the bottom. Pick up one stitch per row around the brim. Because the body is knit on larger needles, there is no need to adjust for gauge.

Knitting the body

Count the picked up stitches. If you didn't miss any rows, you should have a multiple of 16 stitches, plus one. Find the nearest multiple of 16. If it is greater than your number of stitches, you'll be increasing. If it is less than the current number of stitches, you'll need to decrease.

Mark the beginning of the round.

Using the 6 mm needles, knit one round in p2, k2 rib. If you need to increase, knit, evenly around the round, kfb, p2. If you need to decrease, knit k2tog, k1, p2 evenly spaced around the body of the hat.

Knit the following pattern:

p2, k2, p2, k2, p2, k2, p2, k2
p2, k2, p2, k2, p2, k2, p2, k2
p2, c2l, c2r, p2, c2l, c2r
p3, c2x2l, p4, c2x2r, p1
p3, k4, p4, k4, p1
p3, c2x2l, p4, c2x2r, p1
p3, k4, p4, k4, p1
p2, c2r, c2l, p2, c2r, c2l
p2, k2, p2, k2, p2, k2, p2, k2
Do not knit the last two stitches of this round. Instead, move your round marker back two stitches. This is the new beginning of the round.
c2l, c2r, p2, c2l, c2r, p2
p1, c2x2r, p4, c2x2l, p3
p1, k4, p4, k4, p3
p1, c2x2r, p4, c2x2l, p3
p1, k4, p4, k4, p3
These two rounds make a 4-stitch-wide twist in the columns of 4 knit stitches.
c2r, c2l, p2, c2r, c2l, p2
Knit the first two stitches of the next round and move your round marker forward two stitches. This is the new beginning of the round.
p2, k2, p2, k2, p2, k2, p2, k2
p2, c2l, c2r, p2, c2l, c2r
p3, c2x2l, p4, c2x2r, p1
p3, k4, p4, k4, p1
p3, c2x2l, p4, c2x2r, p1

Reducing

If you are knitting on circular needles, as you knit the following rounds, when the stitches become too tight, switch to double-pointed needles.

p3, k2tog, k2, p4, k2, ssk, p1 (14 stitches per repeat)
p2, s1, p1, psso, c2l, p2, c2r, p2tog (12 stitches/repeat)
p2, s1, p1, psso, k2, p2, k2, p2tog (10 stitches/repeat)
p3, c2l, c2r, p1
s1, p1, psso, p2, c2x2r, p2tog (8 stitches/repeat)
p3, k4, p1
p3, c2x2r, p1
p3, ssk, k2tog, p1 (6 stitches/repeat)
s1, p1, psso, p1, k2tog, p1 (4 stitches/repeat)
s1, p1, psso, p2tog (2 stitches/repeat)

Finishing

Cut the yarn and draw the tail through the stitches on your double-pointed needles. Pull it tight, knot it and weave it in.

The bottom of the brim is 4 rows of purl stitches and likely rolls in on itself. Taking a tapestry needle and yarn of the same colour as the brim, sew the selvedge to the uppermost row of purl stitches.

Chart

Brim

Body

Spiral-stitch hat

2009-01-31T14:45:00.005-05:00

Here's a nice simple hat pattern that popped into my head as I was re-knitting the knotwork hat.

Cast on enough stitches (a multiple of 4) on a circular needle to go loosely around the head. Join in the round and mark your starting point.

Knit one round of k2, p2 rib.
Cable two left (place two stitches on a cable needle, purl one stitch, knit two stitches off of the cable needle), purl one, repeat.
At the end of the round, move your starting point forward one stitch.

When the hat fits from ears to the top of the head, begin decreasing on the purl stitches. As before, after every cable row, move your round marker forward one stitch.

Essentially,ssk the last knit stitch of every fourth rib together with the following purl stitch. Cable the next round, then ssk on the cable before the decreased one, cable again.

The chart shows four cables; the parts in purple are repeats of the pattern. Rounds 1 and 2 should be repeated until the hat is tall enough, as described above.

These decreases should spiral around to the middle, where they meet.

After you're done decreasing, pass the yarn through the remaining stitches and pull tight.

Two-Piece Knotwork Hat -- Ripping

2009-01-31T13:24:00.002-05:00

By round 20 of the hat body, I'd made four mistakes. The biggest was that I'd cabled one of the twists left then right, but I'd also lost a stitch, realised that I wanted to alter the pattern at round 5, and didn't like how the knitted join between brim and body looked.

Solution: Rip 20, start over. I am, however, liking the feel of the second attempt more than the first, but this brings the false-start tally on this project to:

5 attempts at knitting a nice cable pattern for the brim.
2 test knits for the body pattern.
1 nearly-complete body ripped out.

Picking up a knit selvedge

2009-01-29T14:46:00.000-05:00

This is a step that I first tried two months ago. I was pleasantly surprised that it was far less difficult than I was anticipating.

Preparation

You'll need the following:

Needles smaller than the ones you intend to use, of the same type (straight, circular or double-pointed.)
Knowledge of your vertical gauge (rows/cm) in the material from which you're picking up and your horizontal gauge (stitches/cm) with the yarn and needles that you'll be using to knit new material.
It is utterly vital that your gauge be over the same measurement. Further, this will be a lot easier if the number of stitches is a whole number. 3.5 rows / cm is much harder to work with than 7 rows / 2 cm.

Picking Up

For the sake of discussion, I'll presume that your stitches on the fabric from which you're picking up run left to right. That is to say that the material has been rotated ninety degrees clockwise. If it's turned the other way, reverse all lefts and right in the directions.

Looking at the fabric, the knit stitches should make "V"s, all pointing left. Directly above those should be your selvedge stitches. Between those, above the point of the "V", there is a hole in the fabric.

Insert one needle into this hole, then the other one beneath it. There should be two strands of yarn above the needles, as you're inserting the needles between two knit stitches. Knit one stitch.

Your new stitches should pull slightly to the right, making the hole through which you knit larger and more visible. Fortunately, as you pick up the other selvedge stitches, this becomes less visible.

Repeat this process with the next stitch, inserting the needle at the hole left of the knit "V" and knitting another stitch. This will give you one picked-up stitch per row.

Equalizing Sizes

There's a problem here, as you might guess from the gauge measurements. Namely, vertical gauge is different from horizontal gauge. There's two ways to solve that problem, one easy, the other giving more control over the fabric.

To cover the easy way first, simply knit the new material on needles where your horizontal gauge is the same as the vertical gauge on the fabric from which you're picking up. That is to say, if your vertical gauge is 5 rows / 2 cm, increase the needle size to ones where you have a horizontal gauge of 5 stitches / 2 cm.

Simple enough. The drawback is that, by necessity, the new fabric will be looser than the fabric from which you were picking up. If that doesn't bother you, carry on. If it does, read on.

To figure out how the fabric has to change requires a little math.

( stitches / cm ) / ( rows / cm ) == stitches / row

So, for example, if I have a horizontal gauge of 9 stitches / 2 cm and a vertical gauge of 7 rows / 2 cm, it works out as:

( 9 stitches / 2 cm ) / ( 7 rows / 2 cm ) == 9 / 7 stitches / row

The above is why I emphasized that the measurements need to be over the same distance. So long as your measurements are the same distance, the distances cancel out. It's also why the number of stitches need to be whole numbers: to avoid having fractions like 4.5 / 3.5. This is important because later on, these numbers are used for counting stitches, and 4.5 stitches is not a valid count.

If, on the other hand, I'm going from 24 rows / 5 cm to 8 stitches / 5 cm (small needles to larger ones), I get the calculation:

( 8 stitches / 5 cm ) / ( 24 rows / 5 cm ) == 8 / 24 stitches / row

Hopefully, it's evident that this part is simply creating a fraction with horizontal gauge over vertical gauge. There's nothing more to it if your measurement is the same.

If you want, reduce the fraction, so that both halves are as small as possible. This is only necessary for mathematical correctness (or ease of counting) though, so if you don't remember how or don't want to bother, leave it as it is.

Increasing

If the numerator (top part of the fraction) is greater than the denominator (bottom part of the fraction), that indicates that you need to increase the number of stitches to meet gauge.

Subtract the denominator from the numerator. The result is how many stitches you have to gain.

If I start with 9 / 7 stitches / row:

numerator - denominator == 9 - 7 == 2

So in that example, I need to gain 2 stitches over every 7. In other words, in every 7 stitches knit, I need two increases.

Decreasing

If the numerator (top part of the fraction) is less than the denominator (bottom part of the fraction), you'll need to decrease the number of stitches to meet gauge.

Subtract the numerator from the denominator. The result is how many stitches you need to lose. The result times 2 (because a decrease takes 2 stitches and makes them into 1 is how many stitches are required to make that many simple decreases.

If I start with 8 / 24 stitches / row:

denominator - numerator == 24 - 8 == 16

16 * 2 == 32

The result tells me that I need to lose 16 stitches, meaning that I need 32 stitches to decrease, or that I need to start rethinking needle sizes.

If I start with the more reasonable 7 / 9 stitches / row:

denominator - numerator == 9 - 7 == 2

2 * 2 == 4

This time, I need to lose two stitches every 9. That is to say, I need to decrease over 4 stitches, making them into 2. Because 4 is less than 9, this change in gauge is plausible.

Actually changing the number of stitches

The easiest way is to either knit or purl a row (depending on the rest of the pattern). Work all the increases or decreases on this row, which should sit close to the original fabric.

If you're working increases, do so with an eye to the bottom of the pattern, that way a stitch knit front and back can become two knit stitches on the next row.

In either case, it looks neater if the increases and decreases are spread out across the pattern, rather than being clustered together.

Two-Piece Knotwork Hat -- Grafting and Failure

2009-01-29T13:51:00.007-05:00

So, the brim to this hat is done; 60 cm of dense cabling. That led me to a technique that I knew I was going to have to use. Namely, grafting.

Knowing that, I turned to Google, which gave me the helpful entry on nonaKnits blog However, there was a catch: that and every tutorial I could find that made sense to me wanted to talk about stockingette. My fabric is p4, k6, p2, k6, p2, k2.

So I thought a bit. Then I forged ahead, full of hubris. And, to be blunt, I fucked up. After the fact, it occured to me that, on a project that, to date, has had six test swatches knit to get the cable patterns right, I could have spared the ten minutes to knit two pieces of stockingette-plus-ribbing to screw around with. Especially since I know better than to carry out experiments in a production environment.

That said, I learned a lot from my mistakes:

Always, always, always swatch.
Kitchener stitch has four actions for each stitch.
If the yarn goes to the middle, that's a knit. If it goes to the outside, that's a purl.
It's easiest to start on a knit stitch.
Grafted stitches are nearly impossible to pull out.
Always swatch, you fool.

So, that said, I now see the geometry of the two basic stitches and I'm writing it down so that next time, I don't dig myself a hole.

To start, have all stitches live and on two needles with the tips facing left as you hold them in front of you. Have a tail of yarn at least 3 times the length of the edges that you're about to graft together.

If the right side starts with a purl stitch, turn the work inside out, so that you're starting on a wrong side knit stitch.

Knit Stitch

Pass the yarn knitwise through the first stitch on the needle nearest you. Since it is going toward the middle, it's a knit stitch.
Drop the stitch off the needle.
Pass the yarn purlwise through the next stitch on the needle nearest you.
Leave this stitch on the needle.
Pass the yarn purlwise through the first stitch on the farther needle. Since it is again going toward the middle, it's a knit stitch. Drop the stitch off the needle.
Finally, pass the yarn knitwise through the next stitch on the farther needle.
Leave this stitch on the needle.

Repeat those four steps for every knit stitch, even if the next is a purl. I count them in my head as 'in, out, in, out'

Purl Stitch

Unsurprisingly, the purl is simply the inverse of the knit.

Pass the yarn purlwise through the first stitch on the needle nearest you. Since it is going away from the middle, it's a purl stitch.
Drop the stitch off the needle.
Pass the yarn knitwise through the next stitch on the needle nearest you.
Leave this stitch on the needle.
Pass the yarn knitwise through the first stitch on the farther needle. Since it is again going toward the outside, it's a knit stitch. Drop the stitch off the needle.
Finally, pass the yarn purlwise through the next stitch on the farther needle.
Leave this stitch on the needle.

In the case of this hat, I'm going to suck it up, live with my mistakes and carry on. Fortunately, the next step in this project is far less arduous: picking up one stitch per row of ribbing from a knit selvedge.

Two-piece Knotwork Hat -- Brim 1

2009-01-27T21:56:00.005-05:00

I'm knitting, flat, the brim of a hat which will eventually be joined together and picked up. This is the chart that I'm following, a paired three-strand braid. The left side will be picked up and be knitted with a further knotwork pattern. The right side will be folded over and sewn together, to make a rolled border.

Dogwood Cabled Hat

2009-01-24T19:58:00.008-05:00

I've just finished knitting this hat, with a cable pattern of stylised trees. The trees themselves are based on a binary tree structure, but have twists and knots around themselves and around other branches. The way the branches converge into a cluster reminded me of groves of dogwood, hence the name.

The trunk expands (via a pair of decreases and increases), then splits into two branches. These branches twine once around themselves then split again, the four split branches twining before dividing yet again into a meshwork of twisting branches that stretch to the top of the hat.

Joy of Cables Hat -- Completed extension

2009-01-24T11:56:00.004-05:00

Here is the travelling cable hat after I got done adding eight rounds of k2p2 rib to it. It's warmer, especially around the ears.

Joy of Cables Hat -- extension

2009-01-22T17:32:00.002-05:00

I've worn the Joy of Cables Hat for a few days now, and am finding that my fear of running out of yarn made for a hat that was a little too shallow. It just covers my ears and does so loosely.

Fortunately, I have plenty of yarn in other colours, so I'm going to extend the hat by 5cm or so. The edge is a simple k2p2 rib, so what I'll do is pick up stitches, purl one row to give the hat a line where it'll fold easily, then knit a brim in the same rib.

Because I'm trying to tighten the hat a bit as well, I'll keep the rib the same. Were it already a snug fit, I'd increase on rows 4 and 8, in the purl part of the ribbing.

The hat was knit on 5mm needles, so I'll pick up with much smaller needles. The closest smallercircular needles at hand were 3.5mm, so I'm using those. Picking up stitches is a simple enough prospect: there's a hole just right of each stitch, so I insert a needle under both threads of the cast on, insert the point of my circular needle and knit a stitch. Then I repeat that 120 times, one per stitch.

After that, I'll knit from the 3.5mm needles onto 6mm ones, as that was the size used for the hat.

Broad Street Mittens - Sleeve part 2

2009-01-22T11:23:00.004-05:00

I have knit the right sleeve now. To fit my arm, it took a total of 68 rounds on 5mm DPNs. As well, to work the increases that were necessary to making the sleeve taper outward, I chose a style of increase that fit the pattern.

Starting at the beginning of a rib, k3, p1, purl the front of the next stitch then knit the back (pfb). (6 stitches)
k1, s1, k1, p3
k2, p1, knit the front loop, then the back, of the next stitch (kfb), p1 (7 stitches)
k1, s1, k1, p1, k2, p1
k3, pfb, k2, pfb (9 stitches)
k1, s1, k1, p2, k2, p2
k3, p2, k1, kfb, p2 (10 stitches)
k1, s1, k1, p2, k1, s1, k1, p2
k3, p2, k3, p2

At this point, another rib has been added, appearing as a line of knit stitches in the middle of a column of purl stitches.

I did this twice, once after knitting 30 rounds and again after knitting 50. This allowed me to keep the glove tight around the wrist, flaring slightly as it approached my elbow.

Because I like the zigzag that the ribbing pattern gives, I alternated green and brown yarns for the majority of it, ending the upper sleeve with a change to green and grey for ten rounds each.

Broad Street Mittens -- Sleeve

2009-01-21T11:26:00.004-05:00

I've now knitted one pair of Janis Cortese's Broad Street Mittens in worsted yarn. They fit nicely, but, unfortunately, only come to my wrist as I've not yet started knitting the cuff/sleeve.

I want the sleeve to be at least 30 cm, so left it out of knitting the mittens until I saw how they turned out. Though, I suppose that I could have started with the sleeve and converted it into arm warmers after the fact.

I know that the current cuff of the mittens is 40 stitches around. I also know that my wrist is tiny, so what I intend to do is reduce to 30 stitches around (I pulled the cuff of the glove tight and counted the number of stitches around my wrist) and then add 5 stitches halfway up my arm.

Why 5? Well, I've found myself knitting a lot of k2p2 rib recently, so want to do something different, especially seeing as it looks serviceable but ultimately uninteresting when worked in multiple colours. I have a nice olive drab yarn that I bought when I was first considering these mittens, and I'd like the sleeves to be striped.

With those two constraints in mind, I considered various ribbings and decided that I'd work the following pattern:

knit 1, slip 1 purlwise with yarn in back, knit 1, purl 2 (repeat from start)
knit 3, purl 2 (repeat from start)

That way, the slipped stitches break the monotony of simple circular stripes, forming a slight wave pattern with the purl stitches, while still providing the texture and stretchiness of ribbing.

The final consideration is what gauge to use. I'd considered knitting the sleeve on 3.75mm needles, but gradually decided that, since it's a layer that will be worn along with a shirt and coat, that I might be wise to use 5mm needles to get a looser fabric. That means that, rather than using 30 stitches as I'd intended, I'll need to use fewer.

My gauge on the gloves was 8 stitches / inch and my gauge on the 5mm needles with worsted yarn is 5 stitches/inch. From there, it's simple algebra to figure out how many I need:

desired stitches / 30 == 5 st/in / 8 st/in
desired stitches == ( 5 / 8 ) * 30
desired stitches = 18.75

Since the pattern I'm using for the ribbing is a repeat of 5 stitches, I round 18.75 to 20 and know that I need to decrease half the stitches to make it fit. In order to keep the glove from puffing out, I'm going to try picking up 40 stitches then decreasing ten of them the next row, knitting one row straight, then decreasing ten times on the next row.

If that doesn't work, I'll rip it out, pick up and knit a round with 3.75mm needles, decrease to 30 stitches and then knit onto the 5mm needles.

A note, though: to keep the fabric from stretching as I pick up, that will be done on 3.75mm needles and the second round will be knit onto the 5mm ones.

Broad Street Mittens (errata)

2009-01-20T11:37:00.003-05:00

I'm currently knitting Janis Cortese's lovely Broad Street mittens, from the fall 2002 issue of Knitty. In looking over the pattern, I decided to make a few adjustments in the interest of warmth.

This is not a pattern so much as it is a list of modifications to the Broad Street pattern.

Firstly, the pattern is meant to be knit with small needles and fingering-weight wool. I decided that I wanted worsted wool, partly for warmth and partly because, when contemplating the pattern, I bought two balls of worsted in preparation.

In what could be considered a moderately silly gauge swatch, I knit the palm, following the pattern. Then, comparing the half-glove hat I had to the size of my hand, I worked out that it was about 50% too big.

I cast on again, starting from the wrist as before. This time, knowing the difference in size, I used 40 stitches of worsted weight on 3.75 mm needles, for a tight, dense fabric. The reason for starting at the wrist rather than the cuff is that I'm planning to knit elbow-length sleeves onto the gloves, so will pick up around the wrist and knit until I'm satisfied.

I knit two rows before getting to my second change: rather than knitting the palm in stockingette, I want it to be a thicker, denser fabric. A quick browse of Barbara Walker's Treasury of Knitting Patterns gave me k1s1 rib, or heel stitch.

Thus the pattern for the palm became:

knit the whole round
knit the whole round
k2, *k1, slip one stitch purlwise with the yarn in back, repeat from * until the second-last stitch on the second needle. p1, knit the last stith front and back. Knit the rest of the round.
knit the whole round
k2, *k1, sl1 wyib, repeat from * until reaching the purled stitch. p1, kfb. Knit the rest of the round.
knit the whole round.

I repeated steps 5 and 6 until I'd added eight stitches, set those stitches aside and cast on nine stitches on the next row.

For the thumb gusset decrease, I used the SSK/k2tog decrease until I again had ten stitches on the second needle.

To finish the palm, I knit 12 rows with heel stitch on the palm, stockingette on the back. The glove is longer lengthwise because the heel stitch used makes a far denser fabric vertically. Also, my hands are very long. Following the pattern, 8 rows would be more accurate.

Pinky finger was 4 from the palm, 5 from the back, plus one cast-on stitch.

Extending the palm, I picked up 3 stitches from the pinky, rather than 2, to ensure no holes. Knit 8 rounds, heel stitch on the front, stst on the back.

The other fingers were 12 stitches, 16 stitches and 16 stitches, using 3 stitches picked up from the finger beside them each time.

For the thumb, I picked up 12 stitches from the cast-on stitches and immediately decreased from 22 to 18, at a rate of two decreases every other row. The thumb was finished in the same way as described in the pattern.

Joy of Cables Hat

2009-01-19T11:04:00.005-05:00

This is a pattern for a hat that just popped into my head. I thought it would be neat to use cables in a spiral pattern, so that half the cables turned right and the other half turned left. As I was knitting it, I added a few more things, like the cables being woven into an over/under/over basketweave.

The hat is surprisingly soft and warm (I knit it from heavy worsted weight wool) and, because it is, in essence, a k2 p2 rib all the way around, it has a good amount of give to it.

The pattern has cabling on three rounds of every eight, but it repeats often enough that it's elatively simple to keep in your head.

Requirements:

Double-pointed needles (I used 5mm DPNs)
Optionally circular needles in the same gauge.
100 to 150 metres of worsted weight yarn.

I knit the hat with a gauge of 2 stitches / cm in k2p2 rib and 5 rounds / 2 cm, but the pattern adapts easily to different sizes by adding or removing repeats both around and vertically.

Cast on enough stitches on circular or double-pointed needles to go loosely around the head, with the number of stitches being a multiple of 8. The pattern repeats are 8 stitches long, with two cables being worked every 8 stitches.

Join in the round, then knit 2, purl 2 and mark that point as the start of your round. The pattern from here on will be expressed as a set of eight stitches. Repeat that as many times as needed to finish the round.

Rows 1-12	k2, p2, k2, p2
Row 13	Cable 2 left (c2l: Place two stitches on a cable needle or double-pointed needle, hold them in front of the work. Purl the third stitch then knit the two stitches off the cable needle.), cable 2 right (c2r: Place one stitch on another needle, holding it to the back of the work. Knit the next two then purl the stitch off of the cable needle), p2
Row 14	p1, cable 2x2 right (c2x2r: Place two stitches on a cable needle, holding it to the back of the work. Knit the next two then knit the stitches off of the cable needle), p3
Row 15	p1, k4, p3
Row 16	C2r, c2l, p2

At this point, every pair of cables has moved toward each other and crossed over, with the left cable going over the right.

Rows 17-19	k2, p2, k2, p2
Row 20	k2, p2, k2, p2
	Stop this round four stitches before the end and mark this point. There should be one column of ribbing left. For the next four rounds, this is the new beginning of the round.
Row 21	c2l, c2r, p2
Row 22	p1, cable 2x2 left (c2x2l: Place two stitches on a cable needle, holding it to the front of the work. Knit the next two then knit the stitches off of the cable needle.), p3
Row 23	p1, k4, p3
Row 24	c2r, c2l, p2
Row 25	k2, p2, k2, p2
	At the end of this round, move the beginning of the round forward four stitches, so that it's back to where it was originally.
Rows 26-28	k2, p2, k2, p2
Row 29	c2l, c2r, p2
Row 30	p1, c2x2r, p3
Row 31	p1, k4, p3
Row 32	c2r, c2l, p2
Rows 33-35	k2, p2, k2, p2
Row 36	k2, p2, k2, p2, marking 4 stitches before the end as the new beginning of the round.
Row 37	c2l, c2r, p2
Row 38	p1, c2x2l, p3

This is where I chose to begin decreasing. If you want a deeper hat, continue the pattern, knitting rows 23-38 until the hat is as long as you want it and stopping on a row where you are either cabling 2x2 left or cabling 2x2 right.

For the decrease rows, the pattern repeats quicker and quicker as you go because the pattern is decreasing in size with every row. Row 1 repeats every 8 stitches, row 2 every 6, row 3 every 5, etc.

If you are using circular needles, when the hat becomes too tight on them, switch to double-pointed needles in the same size.

Decrease row 1	Slip slip knit (ssk: slip two stitches, inserting the right needle as if to knit them. Insert the left needle into the front loops of the slipped stitches, wrap the yarn around the right needle and draw it through the loops), knit two together (k2tog: insert the right needle through the front loops of the first two stitches on the left needle. Knit these stitches as if they were a single stitch), p4
Decrease row 2	SSK, p4
Decrease row 3	SSK, p3
Decrease row 4	SSK, p2
Decrease row 5	SSK, p1
Decrease row 6	SSK

Repeat row 6 until you have fewer than ten stitches on your needles. Take the yarn and pull it through all of the remaining stitches before sliding them off the needle. Pull the yarn inside the hat and tug it tight.

Simple Toe-up Socks

2009-01-18T15:30:00.003-05:00

This is a pattern for toe-up socks. I amended several sock patterns and blended them together after finding that none of the patterns that I found on the internet fit my foot exactly.

These socks are fitted, so knowing your gauge with the needles and yarn that you'll be using is a must. As well, you'll need to know the length of the foot, the circumference around the ball of the foot, the circumference around the instep and the circumference around the ankle.

Casting on

The sock starts with a provisional cast on. The way that I find easiest is to crochet a chain around a needle, that way it incorporates both the crochet chain and picking up the stitches into a single step. The toe is an adaptation of a pattern in Knitty.

Using scrap yarn, start by tying a slip knot around a crochet hook. Then, wrapping the yarn behind a knitting needle, draw the yarn through the loop on the crochet hook, making a stitch.

You should have one loop on the crochet hook and one stitch below it, with the yarn wrapped around your knitting needle.

Repeat this process until you have at least eight stitches wrapped around the needle, then continue crocheting a chain without wrapping the yarn around the needle. This tail is useful to show what side you'll pull on to unzip the chain later on. (Video of crocheting a provisional cast on)

Next, with your working yarn, knit eight of the stitches on the needle, then continue working in stockingette until you have four rows, two knit and two purl.

Separate the eight live stitches onto two double-pointed needles, then pull on the tail of the crochet chain to unfasten it, picking up the stitches of working yarn onto another two needles.

Knitting the toe

The first two needles are the top of the foot and the second two are the bottom. Knit the first four stitches, mark the second needle (it will be the start of the round, or needle 1) and continue knitting in the round until you're back to the first needle.

I like having the tip of the sock sit over my big toe, so I work the increases in a way that makes the toe lopsided. The pattern for this is:

Knit all but two stitches on needle 1, then knit the front and back (KFB) of the second-last stitch, inserting the right needle for a knit stitch, pulling a loop of yarn through, then inserting the right needle through the back loop of the same stitch, pulling a second loop of yarn through, then sliding the stitches off of the left needle.

Knit the last stitch on needle 1, the first stitch on needle 2, KFB the second stitch, then knit to the second-last stitch on needle 3. KFB that stitch, knit the last stitch on needle 3, the first stitch on needle 4, KFB the next stitch and knit to the end of the needle.

Next, knit a round normally. This loosens the stitches that you just made, while slowing the growth of the toe slightly.

For the third round, knit all the stitches on needles 1 and 2, knit to the second-last stitch of needle 3, KFB, knit the last stitch and the first on needle 4, KFB the second and knit to the end of needle 4.

The fourth round is another one of knit stitch. Note: The above pattern is for the left foot. For the right foot, the increases on the third row are on the second-last stitch of needle 1 and the second stitch of needle 2 instead.

Work out how many stitches go around the ball of the foot by taking your measurement and multiplying it by your gauge. In my case, my gauge working in the round is 3 stitches / cm and the ball of my foot is 16 cm in circumference.

16 cm * 3 stitches / cm == 48 stitches

Knit the above four rounds until your toe has grown to the size measured around the instep, rounded to the nearest even number of stitches. If the yarn is stretchy, round down, if not, round up.

The toe will have a slightly more oblique slant on the side with twice as many increases (in this case, the left). It's a slight difference, but means that, when on the foot, the stitches will stretch more naturally across your toes.

The body of the foot

With the toe done, knit 3 centimetres of knit stitch (stockingette).

Now, if the instep is smaller than the ball of the foot, decrease by knitting the second and third last stitches of one needle together and knitting an SSK on the second and third stitches of the next, both on the opposite side to where you did twice as many increases. Repeat this until you have the right number of stitches for the instep. SSK: Insert the right needle into the first stitch on the left as if to knit, then slip it. Repeat for the second. Insert the left needle through the front loops of the two slipped stitches, wrap the yarn around the right needle and pull it through, slipping the two stitches off the needle. (Video)

In my case, my instep is 15 cm around and my gauge is 3 stitches/cm, so I calculate the number of stitches for the instep as:

15 cm * 3 stitches / cm == 45 stitches

As that's an odd number and I'm using somewhat stretchy yarn, I round down to 44 stitches. To calculate the number of rounds I need to decrease on, I subtract the instep stitches from the stitches around the ball of my foot and divide by 2.

48 stitches around the ball - 44 stitches around the instep == 4 stitches
4 stitches / 2 decreases/round == 2 rounds of decreases

So, on the next two rounds, this being a left sock, I knit the second and third-last stitches of needle 1 together and I SSK the second and third stitches on needle 2. Were it a right sock, I would knit the second and third-last stitches of needle 3 together and SSK the second and third stitches of needle 4.

Repeat this for as many rounds as needed to get to the instep size, then continue knitting stockingette.

Shaping a gusseted heel

Begin the heel when the sock is between 60% and 70% the length of the foot, shorter for stretchy yarn, longer for more solid. The heel that I'm using is an adaptation of the pattern found here.

My foot is 22 cm long and I'm using stretchy yarn, so I calculate it as follows:

22 cm * 0.6 == 13.2 cm

Knowing that I have 4 rows of stockingette per centimetre, I knit 13 centimetres of foot, plus one row.

In the last few rows before shaping the heel, I increase back to the size of the ball of the foot, knitting front and back on the second-last stitch of needle 1 and doing another KFB on the second stitch of needle 2. Were this a right sock, I would do the same with needles 3 and 4 instead.

I want twice as many rows to increase as to decrease, so I do the two KFBs, followed by a round of knit stitch, twice, ending on a round of knit stitch.

Now, place all of the top stitches, plus 2/3 of the bottom stitches, onto another needle or a piece of scrap yarn.

In my case, the sock is 48 stitches around, so I place the 24 top stitches onto a needle, as well as 1/3 the bottom stitches, divided equally between left and right sides.

24 bottom stitches * 2 / 3 == 16
24 bottom stitches - 16 stitches remaining == 8 stitches set aside
8 stitches set aside / 2 == 4 stitches from each side

The stitches set aside are left alone while you knit the heel flap as flat stockingette (knit the right side, purl the wrong side). Continue knitting that until the sock plus heel flap is 90% or 95% the length of the foot.

In my case, with a 22 cm foot and stretchy yarn, I want the sock to total:

22 cm * 0.9 == 19.8 cm

Once the heel flap is the proper length, starting on the right side, knit all but the two last stitches, turn your work and purl all but the two last stitches. On the next row, knit all but the four last stitches, turn your work and purl all but the four last stitches.

Repeat this until only half the stitches on the heel flap are live, then, starting on a knit row, knit the live stitches, knitting the last live stitch together with the first one that was left alone. Finish the row by knitting the second stitch that was left alone.

On the next row, purl the live stitches, purling the last one together with the first one left alone. Purl the second one left alone. Repeat these two steps until all the stitches on the heel flap are live again. At this point, you will be on a knit row. Knit back across these to position the yarn for picking up the stitches.

Next, divide the stitches on the heel flap onto two double pointed needles and, starting on the side with the yarn, pick up the stitches along the side of the heel flap by lifting the stitch with another needle and knitting into it with the double pointed needle holding the yarn. (Video of picking up stitches)

Repeat this for every row of the heel flap plus one stitch beside your saved stitches. This ensures that there's no hole where the heel meets the top of the sock.

With a second double-pointed needle, knit half of the saved stitches. With a third, knit the other half and pick up along the other side of the heel flap. Finally, on the fourth needle, knit the remaining half of the heel flap. You should have two needles with few stitches on them and two needles with quite a lot, having picked up along the heel flap.

Divide the stitches among your four double-pointed needles so that the two that meet at the back of the heel have the heel stitches, the heel flap stitches and four of the stitches from the front of the sock.

From now on, the back of the sock is the starting point. The first needle is the left-hand heel needle, the second and third are the two on the front and the fourth is the other heel needle.

Knit all but 6 stitches on the first double-pointed needle, knit two stitches together, place a marker after that stitch, then knit the next four. Continue to knit across the top of the sock, placing a marker after the first four stitches on the fourth double-pointed needle. SSK the next two stitches, then continue to knit.

Continue knitting in the round, knitting the two before the first marker together and SSKing the two after the second marker, until you have the same number of stitches as for the instep.

Ankle and calf

Knit one round after you finish shaping the heel. Now, you want to increase or decrease to the circumference of the ankle.

To decrease, knit one on the first needle, then SSK, knit the rest of the stitches until the third-last stitch on the fourth needle and knit two together, knitting the final stitch. Repeat this until you have the correct number of stitches for the ankle.

To increase, knit one on the first needle, knit the second stitch front and back, then knit the rest of the stitches until the second-last stitch on the fourth needle, KFB and knit the last stitch. Knit the next round normally and repeat these two rounds until you have the correct number of stitches for the ankle.

Continue to knit until you have 5 centimetres less than the desired length.

If you decreased for the ankle, knit half the stitches on each needle, then knit the next stitch front and back, repeating this for each needle, followed by a round of knit stitch, until you have the same number of stitches as you did for the ball of the foot.

The number of stitches around the calf should be a number evenly divisible by four.

Knit 5 centimetres of ribbing. The easiest way is to knit two stitches and purl the next two, repeating this around the cuff of the sock for 5 centimetres.

For these socks, I went with a variant ribbing pattern, where on one row, I knit two, placed the yarn in front of the work as if to purl, slipped two stitches purlwise, then placed the yarn at the back of the work again, repeating that for one round, then knitting the next.

When you have enough ribbing, cast off very loosely.

Clojure code, round 2

2008-02-05T16:03:00.000-05:00

;;; A simple application that acts as a very primitive time server and ;;; client. (import '(java.net BindException ServerSocket Socket) '(java.lang.reflect InvocationTargetException) '(java.util Date) '(java.io InputStream OutputStream StringWriter) '(java.util.concurrent Executors)) ;;; Globals (def DEFAULT-PORT 51345) ;; A structure defining the data in a unit test, as well as a global ;; list of all the tests. (defstruct unit-test :name :description :function :after) (def ALL-TESTS {}) ;;; Testing functions ;; Unit test accessors (def unit-test-name (accessor unit-test :name)) (def unit-test-description (accessor unit-test :description)) (def unit-test-function (accessor unit-test :function)) (def unit-test-after (accessor unit-test :after)) ;; Add a new test structure to the list of unit tests, or replaces the ;; existing test of that name if it already exists. (defn unit-tests-add [test] (def ALL-TESTS (assoc ALL-TESTS (unit-test-name test) test))) ;; Takes an unsorted and a sorted list of unit tests and moves / sorts ;; some of the tests in the unsorted list over to the sorted list (defn unit-test-list-helper [unsorted sorted] (let [nils (filter (comp not unit-test-after) unsorted) alls (filter (comp (appl eql? :all) unit-test-after) unsorted) others (filter (fn [t] (not-any? (fn [x] (eql? (unit-test-after t) (unit-test-after x))) (concat nils alls))) unsorted) in-sorted (fn [t] (some (fn [x] (eql? (unit-test-name x) (unit-test-after t))) (concat nils sorted)))] (cond (first nils) (list (concat alls (filter (complement in-sorted) others)) (concat nils (filter in-sorted others))) (== (count alls) (count unsorted)) (list nil (concat sorted alls)) :t (list (concat alls (filter (complement in-sorted) others)) (concat sorted (filter in-sorted others)))))) ;; Returns a list of unit tests, sorted according to their :after ;; statement. (defn unit-test-list ([] (unit-test-list (vals ALL-TESTS) nil)) ([unsorted sorted] (if (not (first unsorted)) (map (fn [test] (list (unit-test-description test) (unit-test-function test))) sorted) (let [helped-list (unit-test-list-helper unsorted sorted) unsorted (first helped-list) sorted (second helped-list)] (recur unsorted sorted))))) ;; Run all the defined tests (defn test-all [] (let [test-out (new StringWriter)] (loop [test-list (unit-test-list) failure false] (let [test (first test-list) test-list (rest test-list)] (cond (not test) (do (if failure (println "---\nSome tests failed.") (println "---\nAll tests passed.")) (list failure test-out)) :t (let [test-result (time (binding [*out* test-out] (eval (list (second test)))))] (println "Testing" (first test) ":" test-result) (recur test-list (when (or (not test-result) failure) true)))))))) ;; Define a testing function. Same syntax as defn, only with a ;; description string and a list of extras (may be empty) (defmacro deftest [fname description extras & fdecl] `(do (defn ~fname ~@fdecl) (unit-tests-add (struct unit-test :name (name '~fname) :description ~description :function ~fname ~@extras)) #'~fname)) ;;; Utility functions (defn current-time [] (str (new Date))) ;; Generate an array of bytes from a string, suitable for passing to a ;; write() method. ;; ;; Has a unit test. (defn byte-arr-from-string [#^String str] (. str (getBytes))) (deftest test-byte-array-from-string "string->byte array conversion" () ([] (test-byte-array-from-string (current-time))) ([str] (let [barr (byte-arr-from-string str) bseq (map (comp char (appl aget barr)) (range (alength barr))) chseq (map char str)] (and (== (alength barr) (count bseq) (count chseq)) (some true? (map eql? bseq chseq)))))) ;; Takes a sequence of bytes (not a byte array, but a sequence) and ;; concatenates it into a string. (defn string-from-byte-sequence [coll] (reduce strcat (map char coll))) ;; Gets the integer length of a string. Returns nil for "" and nil. (defn #^Int length-of-string [#^String string] (if string (. string (length)) 0)) ;;; Listener functions ;;; These control the server ;; Make a new listener. Returns that listener or nil if no listener ;; can be created on that port. ;; ;; Has a unit test. (defn #^ServerSocket listener-new ([] (listener-new DEFAULT-PORT)) ([port] (try (new ServerSocket port) (catch BindException except (println "Address is already in use."))))) (deftest test-listener-new "creating a listener" () ([] (test-listener-new DEFAULT-PORT)) ([port] (with-open listener (listener-new port) (when listener true)))) ;; Logical predicates for whether a listener is open or closed. ;; ;; Have unit tests. (defn #^Boolean listener-closed? [#^ServerSocket listener] (. listener (isClosed))) (def listener-open? (complement listener-closed?)) (deftest test-listener-predicates "listener open?/closed? predicates" (:after "test-listener-new") ([] (test-listener-predicates DEFAULT-PORT)) ([port] (let [listener (listener-new port) tests (and (not (listener-closed? listener)) (listener-open? listener))] (. listener (close)) (and tests (not (listener-open? listener)) (listener-closed? listener))))) ;; Wait for a connection to a given listener the return the socket to ;; that connection. ;; ;; NOTE: This function is blocking. (defn #^Socket listener-wait [#^ServerSocket listener] (. listener (accept))) ;; Closes a given listener. Really only for use from the REPL or from ;; unit tests, as actual code should be using (with-open ...) ;; ;; Has a unit test. (defn listener-close [#^ServerSocket listener] (. listener (close))) (deftest test-listener-close "closing a listener" (:after "test-listener-predicates") ([] (let [listener (listener-new DEFAULT-PORT)] (and (listener-open? listener) (not (listener-close listener)) (listener-closed? listener))))) ;; Takes a socket, sends the current time through it, closes the ;; socket's output and returns the socket. (defn #^Socket listener-send [#^Socket lsocket] (.. lsocket (getOutputStream) (write (byte-arr-from-string (current-time)))) (.. lsocket (getOutputStream) (close)) lsocket) ;; Launches the supplied listener and then loops, waiting for ;; connections. ;; ;; NOTE: Because it depends of (listener-wait ...), which is blocking, ;; this is qlso blocking. (defn listener-run [listener port] (loop [socket nil] (if (listener-closed? listener) listener (do (when socket (. (listener-send socket) (close))) (recur (listener-wait listener)))))) ;; Launches a listener as a background thread. Returns that listener. (defn listener-run-in-background ([] (listener-run-in-background DEFAULT-PORT)) ([port] (let [listener (listener-new port) exec (. Executors (newSingleThreadExecutor)) run (appl listener-run listener port)] (when listener (. exec (submit run))) listener))) ;;; Connection functions ;;; These control the client. ;; Makes a new connection to a listener. Times out in 5 seconds if no ;; connection is made. ;; ;; Has a unit test. (defn connection-new ([] (connection-new DEFAULT-PORT)) ([port] (connection-new "127.0.0.1" port)) ([address port] (try (doto (new Socket address port) (setSoTimeout 5000)) (catch InvocationTargetException except (println (strcat "Could not connect to " address " on port " port)))))) (deftest test-connection-new "creating a connection" (:after "test-listener-close") ([] (let [listener (listener-run-in-background) connection (connection-new)] (and (connection-new) (not (listener-close listener)) (not (connection-new)))))) ;; Reads data from a supplied socket as an array of bytes. Stops on ;; timeout and when receiving the -1 end of stream byte. (defn connection-read [#^Socket conn] (let [instream (. conn (getInputStream)) reader (fn [] (try (. instream (read)) (catch InvocationTargetException except -1)))] (loop [bytes nil current-byte (reader)] (if (== current-byte -1) bytes (recur (concat bytes (list current-byte)) (reader)))))) ;; Closes an open socket. (defn connection-close [#^Socket conn] (. conn (close))) ;; Opens a connection to a listener, reads out a byte sequence, then ;; returns that sequence as a string. ;; ;; Has a unit test. (defn connection-run ([] (connection-run DEFAULT-PORT)) ([port] (let [conn (connection-new port) str (when conn (string-from-byte-sequence (connection-read conn)))] (when conn (connection-close conn)) str))) (deftest test-connection-run "running a complete connection" (:after :all) ([] (let [listener (listener-run-in-background) result (connection-run)] (and (listener-open? listener) (not (listener-close listener)) (listener-closed? listener) (<= 0 (length-of-string result)) (== 0 (length-of-string (connection-run)))))))

Clojure: continuing to look

2008-02-05T15:55:00.000-05:00

My second look at Clojure

Casting and complements

As demonstrated to me, (. listener (isClosed)) returns a fresh boolean, rather than Boolean.TRUE or Boolean.FALSE. So, while the symbol looks fine in the REPL, when passed to (if (. listener (isClosed)) ...), it is always true.

So, I made the following pair of complementary¹ utility functions:

(defn #^Boolean listener-closed? [#^ServerSocket listener] (. listener (isClosed))) def listener-open? (complement listener-closed?))

complement takes a function as its argument and returns a function that does exactly the same thing as that function, only with its return value inverted. Thus, (complement listener-closed?) returns a function that does exactly what listener-closed? does and then returns the opposite value.

In this case, it saves me from having to repeat listener-closed?'s details. That way, if I ever need to change listener-closed, in order, say, to handle more than simple Java ServerSockets, I can do so without having to make any changes to listener-open?.

Note, though, that listener-open? is a def and not a defn. That's because defn is a macro that wraps an fn around its body. complement, on the other hand, returns an fn, so I only need to bind it to a Var.

I then amended listener-run so that it uses this explicit cast rather than just taking what ``ServerSocket gives it.

(defn listener-run [listener port] (loop [socket nil] (if (listener-closed? listener) listener (do (when socket (. (listener-send socket) (close))) (recur (listener-wait listener))))))

So, while I'm assigning casting, I should likely take care of the warnings on loading this file, if for no reason other than that they're starting to get a little excessively verbose.

user=> (load-file "/clojure/src/sockets.clj") Reflection warning, line: 17 - call to getBytes can't be resolved. Reflection warning, line: 53 - call to accept can't be resolved. Reflection warning, line: 56 - call to close can't be resolved. Reflection warning, line: 397 - call to getOutputStream can't be resolved. Reflection warning, line: 396 - call to write can't be resolved. Reflection warning, line: 397 - call to getOutputStream can't be resolved. Reflection warning, line: 396 - call to close can't be resolved. Reflection warning, line: 70 - call to close can't be resolved. Reflection warning, line: 78 - call to submit can't be resolved. Reflection warning, line: 88 - call to java.net.Socket ctor can't be resolved. Reflection warning, line: 97 - call to getInputStream can't be resolved. Reflection warning, line: 98 - call to read can't be resolved. Reflection warning, line: 109 - call to close can't be resolved. #<Var: user/connection-run>

Simply, I went through and placed type hints on functions that call out to Java in order to ensure that their methods resolve on load.

Since I also noticed a few new things while reformatting the Clojure manual into a single texinfo document for my own use, I'll also change my use of (. var (toString) to use Clojure's str function:

(str x) Returns x.toString(). (str nil) returns ""

Building a testing framework

All of those casts are fairly minor refactorings that just make the statement of what a listener and connection are to the compiler. On the other hand, the bug that I started with this time would have been caught by the other thing I've been putting off. Namely, actual an testing framework, rather than just eyeballing results in the REPL.

Now, I could go find JUnit, install that, and then see how well Clojure links into it. But, well, that sounds like a new project¹.

So, instead, what I'll look at is defining what it is that my functions are supposed to be doing, write a few tests, then add a framework to glue those tests together.

Ugliness ensues...

Which ended up looking like:

(def DEFAULT-PORT 51345) (defn test-byte-array-from-string ([] (test-byte-array-from-string (current-time))) ([str] (let [barr (byte-arr-from-string str) bseq (map (comp char (appl aget barr)) (range (alength barr))) chseq (map char str)] (and (== (alength barr) (count bseq) (count chseq)) (nil? (first (filter false? (map eql? bseq chseq)))))))) (defn test-listener-new ([] (test-listener-new DEFAULT-PORT)) ([port] (with-open listener (listener-new port) (when listener true)))) (defn test-listener-predicates [])

(defn test-all [] (loop [test-list '(("string->byte array conversion" test-byte-array-from-string) ("creating a listener", test-listener-new) ("listener open?/closed? predicates" test-listener-predicates)) failure false] (let [this-test (first test-list) test-list (rest test-list)] (if (not this-test) (if failure (println "---\nSome tests failed.") (println "---\nAll tests succeeded.")) (let [test-result (time (eval (rest this-test)))] (println "Testing" (first this-test) ":" test-result (if (first test-list) "\n" "")) (recur test-list (when (or failure (not test-result))

And, running it, I get:

user=> (test-all) "Elapsed time: 8.437664 msecs" Testing string->byte array conversion : true "Elapsed time: 68.38411 msecs" Testing creating a listener : true "Elapsed time: 2.831645 msecs" Testing listener open?/closed? predicates : nil --- Some tests failed. nil

So, it works. (Noting that I deliberately built one of those tests to fail, to make sure that the testing function catches failure.) Except that, well, how best to put it... Eww. I know that so far this is only 150 lines, but I'm not going to remember to update the list of tests down in the testing function every time I make a new one. So, I'll run test-all, realise that my new test is missing, then go back and add the new test.

Adding structure...

No, I think what I need here is for the test itself to hold its own payload. Sounds like a reasonable time to add in a struct, really. A struct is just a defined hashmap taking keywords as arguments. For starters, I'll build one that just duplicates the list that I have below:

(defstruct unit-test :string :function)

Well, that does let me wrap tests into structures, but doesn't exactly let me manipulate them:

user=> (struct unit-test :string "creating a listener" :function test-listener-new) {:function user.test_listener_new@d5eb7, :string "creating a listener"}

What I'm going to need is a way to build these tests and load them when I load the file, otherwise I'm just making the process of eyeballing my results in the REPL harder on myself.

After a bit of thought, I settle on this as my test list:

(defstruct unit-test :name :string :function) (def ALL-TESTS {}) (defn unit-tests-add [test] (def ALL-TESTS (assoc ALL-TESTS (get test :name) test))) (defn unit-test-new ([description function] (unit-tests-add (struct unit-test :name function :string description :function function))))

Nothing special at the moment. Just a simple structure defining a unit test as a name, description and function. Now, let's use that one one of the three rudimentary tests that I already have:

(defn test-listener-new ([] (test-listener-new DEFAULT-PORT)) ([port] (with-open listener (listener-new port) (when listener true)))) (unit-test-new "creating a listener" test-listener-new)

All right. Improving a tiny bit. Except that, after evaluating the file twice, I still have:

user=> ALL-TESTS {user.test_listener_new@1a0d866 {:function user.test_listener_new@1a0d866, :string "creating a listener", :name user.test_listener_new@1a0d866}, user.test_listener_new@3f96ee {:function user.test_listener_new@3f96ee, :string "creating a listener", :name user.test_listener_new@3f96ee}}

Makes sense, really. My new instance of test-listener-new has a new symbol. So, even though I know that they're the same function, there's no good reason for the compiler to.

What I really want, though, is to make defining a new test as simple as adding a new (albeit specialised) function, rather than playing around with remembering to call unit-test-new after. So, rather than trying to get unit-test-new to parse out the name of the function, I'll rethink my approach.

What do I actually want? Well, the function, name and description added to my set of tests seamlessiy. I think it's fair to assume that before I'm done, I'll want a few more qualifiers on tests as well. Since I don't know what those are, I won't add them to my structure yet.

Admitting that I need macros...

However, I'm in a conundrum here, because I don't want to add logic to my test definition every time I think of something new. Really, this is starting to sound like a good argument for a macro.

I'll start by just imitating what defn does. Because if I can't imitate that in a macro, the rest of this is going to start smelling like week-old moose.

(defmacro deftest [fname description extras & fdecl] `(defn ~fname ~@fdecl))

So that takes a function name, a descriptive string and a bunch of unspecified "extras" and then proceeds to ignore the second two before shoving the function name and the declaration into a defn. The backtick (`) marks the following list as a template, meaning that the defn won't be resolved right away.

The ~ and ~@ macro characters inside that template mean, respectively, to unquote fname (so that I'm not defining a function called user/fname) and to resolve fdecl into the sequence of values which it represents.

I have to do this because the & rest argument to an fn is considered to be a sequence. Were I simply passing it as an unquoted symbol, I'd end up with an extra set of parentheses around the function body when it's evaluated, which I don't want.

So, having rationalised every jot and tittle of what I just did, I'm going to dump test-listener-new - that is to say, a working function - into deftest via macroexpand-1 and see if it generates a plausible defn. (macroexpand-1 because I don't really want to see the full expanion of defn's internals, just this first layer that I added.

And that resulting definition looks just like the definition of test-listener-new, only on one line and with defn as a qualified namespace/symbol pair.

Seeing as that worked, it means that all I should have to do is wrap defining the function in a do and add, as the second half, a function call adding the test to my list of tests.

(defmacro deftest [fname description extras & fdecl] `(do (defn ~fname ~@fdecl) (unit-tests-add (struct unit-test :name (name '~fname) :string ~description :function ~fname ~@extras))))

Which, when expanded, gats me:

user=> (macroexpand-1 '(deftest test-listener-new "creating a listener" '() ([] (test-listener-new DEFAULT-PORT)) ([port] (with-open listener (listener-new port) (when listener true))))) (do (clojure/defn test-listener-new ([] (test-listener-new DEFAULT-PORT)) ([port] (with-open listener (listener-new port) (when listener true)))) (user/unit-tests-add (clojure/struct user/unit-test :name (clojure/name (quote test-listener-new)) :string "creating a listener" :function test-listener-new quote ())))

Which counts as an almost-but-not-quite. Because I'm quoting the empty list in the arguments that I'm passing to deftest, it expands to (quote ()), which is then unquoted to quote (). Which isn't exactly what I want.

On the other hand, the working syntax:

user=> (macroexpand-1 '(deftest test-listener-new "creating a listener" () ([] (test-listener-new DEFAULT-PORT)) ([port] (with-open listener (listener-new port) (when listener true))))) (do (clojure/defn test-listener-new ([] (test-listener-new DEFAULT-PORT)) ([port] (with-open listener (listener-new port) (when listener true)))) (user/unit-tests-add (clojure/struct user/unit-test :name (clojure/name (quote test-listener-new)) :string "creating a listener" :function test-listener-new)))

It works fine, and saves me a spurious quote. Furthermore, values in a struct default to nil, so if I add more bits to the unit-test structure later, I don't have to cope with accounting for them in any way other than doing nothing for nil, and my previously-defined tests will still work.

Applying these new tests.

So, I modify my test to use this macro and get:

(deftest test-listener-new "creating a listener" () ([] (test-listener-new DEFAULT-PORT)) ([port] (with-open listener (listener-new port) (when listener true))))

And evaluating that adds it to ALL-TESTS.

user=> ALL-TESTS {"test-listener-new" {:function user.test_listener_new@1dcc2a3, :string "creating a listener", :returns nil, :name "test-listener-new"}}

I could have put the extras (that part that I have a suspicion that I'll need, but don't know why yet,) at the end of the deftest macro. However, that means that I'd likely forget to add them. Also, it means that I have to make more changes to an existing test function than simply adding two pieces of information.

On the other hand, because it returns the result of unit-tests-add, deftest isn't quite a drop-in replacement to defn yet. Time to get it to return the function instead.

So, knowing the function name, and establishing the reasonable qualification that the function has been defined (which it should have been, as defining it is a part of the macro, what I want to return is the Var named by fname. Which leads me to:

(defmacro deftest [fname description extras & fdecl] `(do (defn ~fname ~@fdecl) (unit-tests-add (struct unit-test :name (name '~fname) :string ~description :function ~fname ~@extras)) #'~fname))

Which, when I evaluate my test, returns the test function, ensuring that this is now a wrapper around defn that actually returns the right values.

That done, I can finally get to converting my remaining two tests and the test-all function. The tests are easy:

(deftest test-byte-array-from-string "string->byte array conversion" () ([] (test-byte-array-from-string (current-time))) ([str] (let [barr (byte-arr-from-string str) bseq (map (comp char (appl aget barr)) (range (alength barr))) chseq (map char str)] (and (== (alength barr) (count bseq) (count chseq)) (nil? (first (filter false? (map eql? bseq chseq)))))))) (deftest test-listener-predicates "listener open?/closed? predicates" () [])

To make the testing function work, it needs to be able to get at my vector of tests and to examine the tests.

(def unit-test-name (accessor unit-test :name)) (def unit-test-string (accessor unit-test :string)) (def unit-test-function (accessor unit-test :function))

A side-trip into accessors...

Here, however, I'm noticing that I'm duplicating what's already written once in my struct, just to have accessors. Also, :string is a daft name.

So, I'll make :string into :description and generate my accessors based directly off of the structure.

(map (fn [key] (def (sym (name *current-namespace*) (strcat "unit-test-" (name key))) key)) (keys unit-test))

And testing that:

user=> clojure.lang.Compiler$CompilerException: REPL:1026: Second argument to def must be a Symbol at clojure.lang.Compiler.analyzeSeq(Compiler.java:3065) at clojure.lang.Compiler.analyze(Compiler.java:3011) at clojure.lang.Compiler.analyze(Compiler.java:2986) at clojure.lang.Compiler.access$400(Compiler.java:38) at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:2740) at clojure.lang.Compiler$FnMethod.parse(Compiler.java:2626) at clojure.lang.Compiler$FnMethod.access$1300(Compiler.java:2540) at clojure.lang.Compiler$FnExpr.parse(Compiler.java:2327) at clojure.lang.Compiler.analyzeSeq(Compiler.java:3056) at clojure.lang.Compiler.analyze(Compiler.java:3011) at clojure.lang.Compiler.analyze(Compiler.java:2986) at clojure.lang.Compiler.access$400(Compiler.java:38) at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:2255) at clojure.lang.Compiler.analyzeSeq(Compiler.java:3060) at clojure.lang.Compiler.analyze(Compiler.java:3011) at clojure.lang.Compiler.analyze(Compiler.java:2986) at clojure.lang.Compiler.eval(Compiler.java:3085) at clojure.lang.Repl.main(Repl.java:59) Caused by: java.lang.Exception: Second argument to def must be a Symbol at clojure.lang.Compiler$DefExpr$Parser.parse(Compiler.java:296) at clojure.lang.Compiler.analyzeSeq(Compiler.java:3058) ... 17 more

Hmm, not so good. Ok, time to poke at what exactly it is that I'm doing wrong here.

user=> (sym "user" "a") user/a user=> (def (sym "a") 1) clojure.lang.Compiler$CompilerException: REPL:1030: Second argument to def must be a Symbol at clojure.lang.Compiler.analyzeSeq(Compiler.java:3065) at clojure.lang.Compiler.analyze(Compiler.java:3011) at clojure.lang.Compiler.analyze(Compiler.java:2986) at clojure.lang.Compiler.eval(Compiler.java:3085) at clojure.lang.Repl.main(Repl.java:59) Caused by: java.lang.Exception: Second argument to def must be a Symbol at clojure.lang.Compiler$DefExpr$Parser.parse(Compiler.java:296) at clojure.lang.Compiler.analyzeSeq(Compiler.java:3058) ... 4 more user=> (instance? clojure.lang.Symbol (sym "user" "a")) clojure.lang.Compiler$CompilerException: REPL:3: Unable to resolve classname: clojure.lang.PersistentList@d165fe67 at clojure.lang.Compiler.analyzeSeq(Compiler.java:3065) at clojure.lang.Compiler.analyze(Compiler.java:3011) at clojure.lang.Compiler.analyze(Compiler.java:2986) at clojure.lang.Compiler.eval(Compiler.java:3085) at clojure.lang.Repl.main(Repl.java:59) Caused by: java.lang.IllegalArgumentException: Unable to resolve classname: clojure.lang.PersistentList@d165fe67 at clojure.lang.Compiler$InstanceExpr$Parser.parse(Compiler.java:1868) at clojure.lang.Compiler.analyzeSeq(Compiler.java:3058) ... 4 more

So, it appears that both def and instance? are seeing the unexpanded list before it gets turned into a symbol. Curiouser and curiouser.

user=> (map (fn [key] (let [my-sym (sym (name *current-namespace*) (strcat "unit-test-" (name key)))] my-sym)) (keys unit-test)) (user/unit-test-name user/unit-test-function user/unit-test-description) user=> (map (fn [key] (let [my-sym (sym (name *current-namespace*) (strcat "unit-test-" (name key)))] (def my-sym 1))) (keys unit-test)) (#<Var: user/my-sym> #<Var: user/my-sym> #<Var: user/my-sym>)

Ok, so I redefined my-sym three times. After I got as far as:

(map (fn [li] (def ~@li)) (map (fn [key] (list (sym (name *current-namespace*) (strcat "unit-test-" (name key))) (accessor unit-test key))) (keys unit-test)))

I've decided that I'll no longer make a big deal out of building my accessors from my structure, as I seem to have a merry clash between what def wants and what I know how to offer to it.

I'll admit that this failure irks me, but I'm going to leave it as something to poke at later, as I'm getting confused as to operator and macro precedence.

Sobeit:

(def unit-test-name (accessor unit-test :name)) (def unit-test-description (accessor unit-test :description)) (def unit-test-function (accessor unit-test :function))

Giving up and going back to the test function.

Now, on to making the test function actually work with this structure, rather than playing with trying to be clever with generating accessors.

(defn test-all [] (loop [test-list (vals ALL-TESTS) failure false] (let [this-test (first test-list) test-list (rest test-list)] (if (not this-test) (if failure (println "---\nSome tests failed.") (println "---\nAll tests succeeded.")) (let [test-result (time (unit-test-function this-test))] (println "Testing" (unit-test-description this-test) ":" test-result (if (first test-list) "\n" "")) (recur test-list (when (or failure (not test-result))

However, testing that, it shows only that the unit test functions exist. Nothing more.

user=> (test-all) "Elapsed time: 0.181588 msecs" Testing creating a listener : user.test_listener_new@1878144 "Elapsed time: 0.028216 msecs" Testing string->byte array conversion : user.test_byte_array_from_string@137d090 "Elapsed time: 0.019277 msecs" Testing listener open?/closed? predicates : user.test_listener_predicates@15db314 --- All tests succeeded. nil

I need it to evaluate the returned function, preferably with the capacity to insert arguments as needed. Hmm.

(defn test-all [] (loop [test-list (map (fn [x] (list (unit-test-description x) (unit-test-function x))) (vals ALL-TESTS)) failure false] (let [test (first test-list) test-list (rest test-list)] (if (not test) (if failure (println "---\nSome tests failed.") (println "---\nAll tests passed.")) (let [test-result (time (eval (list (second test))))] (println "Testing" (first test) ":" test-result) (recur test-list (when (or (not test-result) failure) true)))))))

Will leave that as a working test framework for the moment and actually move back to writing tests.

(deftest test-listener-predicates "listener open?/closed? predicates" () ([] (test-listener-predicates DEFAULT-PORT)) ([port] (let [listener (listener-new port) tests (and (not (listener-closed? listener)) (listener-open? listener))] (. listener (close)) (and tests (not (listener-open? listener)) (listener-closed? listener))))) (deftest test-listener-close "closing a listener" () ([] (let [listener (listener-new DEFAULT-PORT)] (and (listener-open? listener) (not (listener-close listener)) (listener-closed? listener))))) (deftest test-connection-new "creating a connection" () ([] (let [listener (listener-run-in-background) connection (connection-new)] (and (connection-new) (not (listener-close listener)) (not (connection-new)))))) (deftest test-connection-run "running a complete connection" () ([] (let [listener (listener-run-in-background) result (connection-run)] (and (listener-open? listener) (not (listener-close listener)) (listener-closed? listener) (> 0 (. result (length))) (== 0 (. (connection-run) (length)))))))

So there's my complete set of tests to date. Which I can then without changing the test-all that I had already defined.

user=> (test-all) "Elapsed time: 11.890338 msecs" Testing creating a listener : true Could not connect to 127.0.0.1 on port 51345 "Elapsed time: 1194.393981 msecs" Testing creating a connection : true "Elapsed time: 4.478223 msecs" Testing string->byte array conversion : true "Elapsed time: 35.4397 msecs" Testing running a complete connection : false "Elapsed time: 11.453132 msecs" Testing closing a listener : true "Elapsed time: 13.804548 msecs" Testing listener open?/closed? predicates : true --- Some tests failed. nil

Ok, so I notice two things here. One is that every test works except what should be the last one. The other is that, well, it isn't the last one. In fact, the tests are in no particular order when, in fact, they should have a certain level of ordering in them.

Setting up after statements...

Sounds like I finally have a use for that extras argument that, until now, I haven't been using.

(defstruct unit-test :name :description :function :after) (def unit-test-after (accessor unit-test :after)) (deftest test-listener-predicates "listener open?/closed? predicates" (:after "test-listener-new") ([] (test-listener-predicates DEFAULT-PORT)) ([port] (let [listener (listener-new port) tests (and (not (listener-closed? listener)) (listener-open? listener))] (. listener (close)) (and tests (not (listener-open? listener)) (listener-closed? listener))))) (deftest test-listener-close "closing a listener" (:after "test-listener-predicates") ([] (let [listener (listener-new DEFAULT-PORT)] (and (listener-open? listener) (not (listener-close listener)) (listener-closed? listener))))) (deftest test-connection-new "creating a connection" (:after "test-listener-close") ([] (let [listener (listener-run-in-background) connection (connection-new)] (and (connection-new) (not (listener-close listener)) (not (connection-new)))))) (deftest test-connection-run "running a complete connection" (:after :all) ([] (let [listener (listener-run-in-background) result (connection-run)] (and (listener-open? listener) (not (listener-close listener)) (listener-closed? listener) (> 0 (. result (length))) (== 0 (. (connection-run) (length)))))))

There. Now tests that should have sequence can have them. But this means that I need to actually extract my function for generating the list of unit tests and make it respect the rules that I just added.

(defn unit-test-list ([] (map (fn [x] (list (unit-test-description x) (unit-test-function x))) (vals ALL-TESTS))))

Which gives me the same result for test-all. Now, to sort this data rather than just dropping it in a list.

(defn unit-test-list-helper [unsorted sorted] (let [test (first unsorted) unsorted (rest unsorted)] (cond (or (not (unit-test-after test)) (and (eql? :all (unit-test-after test)) (not unsorted)) (first (filter (fn [x] (eql? (unit-test-name x) (unit-test-after test))) unsorted))) (list unsorted (concat sorted (list test))) :t (list (concat unsorted (list test)) sorted))))

So what that's supposed to do is run through the unsorted list, adding things to the sorted list when they either have no after statement or their after statement is met.

Now, I'll bind that helper into a loop:

(defn unit-test-list ([] (cond (rest (filter (fn [test] (eql? :all (unit-test-after test))))) (do (println "Error: Only one test may be marked as" "coming after all others.") []) :t (unit-test-list (vals ALL-TESTS) (list)))) ([unsorted sorted] (if (not (first unsorted)) (map (fn [test] (list (unit-test-description test) (unit-test-function test))) sorted) (let [helped-list (unit-test-list-helper unsorted sorted) unsorted (first helped-list) sorted (second helped-list)] (recur unsorted sorted)))))

And running it:

user=> (unit-test-list (vals ALL-TESTS) nil)

It hangs.

Stepping through the helper function in a long series of examinations tells me that it's hanging at the last step. Then, after changing it to reflect my actual intent (I typoed sorted as unsorted. I start to think about whether this is actually a good way to build a list. After all, it will take up to the factorial of the length of the list of tests to actually build it.

I poke with comparators for a bit and find that my somewhat fuzzy requirements don't exactly meet comparator's exacting demands². Then I decide to revisit my iffy list function, but move blocks of tests at a time.

(defn unit-test-list-helper [unsorted sorted] (let [nils (filter (comp not unit-test-after) unsorted) alls (filter (comp (appl eql? :all) unit-test-after) unsorted) others (filter (fn [t] (not-any? (fn [x] (eql? (unit-test-after t) (unit-test-after x))) (concat nils alls))) unsorted) in-sorted (fn [t] (some (fn [x] (eql? (unit-test-name x) (unit-test-after t))) (concat nils sorted)))] (cond (first nils) (list (concat alls (filter (complement in-sorted) others)) (concat nils (filter in-sorted others))) (== (count alls) (count unsorted)) (list nil (concat sorted alls)) :t (list (concat alls (filter (complement in-sorted) others)) (concat sorted (filter in-sorted others))))))

This one takes the unsorted list, breaks it into three parts, and moves what it can over to sorted at each step. It sorts my list of functions in three steps, rather than twenty. However, those steps are relatively expensive. I might as well compare the two functions' performances while I've them both on my screen.

user=> (do (dotimes x 10 (time (unit-test-list-old))) (println "----------") (dotimes x 10 (time (unit-test-list)))) "Elapsed time: 3.240635 msecs" "Elapsed time: 0.478832 msecs" "Elapsed time: 0.459835 msecs" "Elapsed time: 0.713778 msecs" "Elapsed time: 0.462908 msecs" "Elapsed time: 0.450616 msecs" "Elapsed time: 0.433575 msecs" "Elapsed time: 0.506489 msecs" "Elapsed time: 0.453689 msecs" "Elapsed time: 5.946846 msecs" ---------- "Elapsed time: 8.428725 msecs" "Elapsed time: 1.182832 msecs" "Elapsed time: 1.732902 msecs" "Elapsed time: 1.130031 msecs" "Elapsed time: 1.121931 msecs" "Elapsed time: 1.130869 msecs" "Elapsed time: 1.137016 msecs" "Elapsed time: 1.390959 msecs" "Elapsed time: 1.163555 msecs" "Elapsed time: 1.132546 msecs"

Interesting. The old one actually did better than the new. Now, I could count their operations, but it would be more fun to get the REPL testing this for me.

user=> (dotimes x 100 (eval `(deftest ~(gensym) "generated fake test" (:after nil) [] true)))

There. Made a hundred test macros with no structure to them. gensym just makes a symbol with a guaranteed-unique name. That ensures that, in generating these tests, I don't have to first come up with my own means to make unique nmes.

Now to try again:

user=> (do (dotimes x 10 (time (unit-test-list-old))) (println "----------") (dotimes x 10 (time (unit-test-list)))) "Elapsed time: 108.745944 msecs" "Elapsed time: 44.778876 msecs" "Elapsed time: 38.915002 msecs" "Elapsed time: 47.836247 msecs" "Elapsed time: 41.699434 msecs" "Elapsed time: 40.098393 msecs" "Elapsed time: 41.917059 msecs" "Elapsed time: 40.536157 msecs" "Elapsed time: 39.040995 msecs" "Elapsed time: 39.437135 msecs" ---------- "Elapsed time: 21.603863 msecs" "Elapsed time: 17.80338 msecs" "Elapsed time: 17.744992 msecs" "Elapsed time: 19.674568 msecs" "Elapsed time: 13.584687 msecs" "Elapsed time: 19.662834 msecs" "Elapsed time: 17.736891 msecs" "Elapsed time: 18.160967 msecs" "Elapsed time: 13.494174 msecs" "Elapsed time: 16.875608 msecs"

A gap is starting to form, I think. What about adding some test macros that do rely on other tests?

user=> (import '(java.util Random)) nil user=> (def RNG (new Random)) #<Var: user/RNG> user=> (. RNG (nextInt)) Reflection warning, line: 948 - call to nextInt can't be resolved. 1410779829 user=> `(deftest ~(gensym) "z" (:after nil) [] true)) user=> `(deftest ~(gensym) "z" (:after nil) [] true) (user/deftest G__2659 "z" (:after nil) [] true) user=> `(deftest ~(gensym) "z" (:after ~(let [keys (keys ALL-TESTS)] (first (drop (rem (. RNG (nextInt)) (count keys)) keys)))) [] true) Reflection warning, line: 951 - call to nextInt can't be resolved. (user/deftest G__2665 "z" (:after "G__2491") [] true) user=> (dotimes x 1000 (eval `(deftest ~(gensym) "z" (:after ~(let [keys (keys ALL-TESTS)] (first (drop (rem (. RNG (nextInt)) (count keys)) keys)))) [] true)))

So, that adds a thousand tests in that do follow a structure, as they all have an after statement this time.

And I run the test again ...

And I go get coffee ...

5 minutes...

It seems to be still running...

Maybe a thousand was a bad seed...

Oh, there it is:

user=> (do (dotimes x 10 (time (unit-test-list-old))) (println "----------") (dotimes x 10 (time (unit-test-list)))) "Elapsed time: 21438.660779 msecs" "Elapsed time: 24259.176236 msecs" "Elapsed time: 20528.833693 msecs" "Elapsed time: 20170.246777 msecs" "Elapsed time: 23642.198659 msecs" "Elapsed time: 20390.313015 msecs" "Elapsed time: 19931.102975 msecs" "Elapsed time: 22232.556093 msecs" "Elapsed time: 21692.653954 msecs" "Elapsed time: 20286.386119 msecs" ---------- "Elapsed time: 5885.515896 msecs" "Elapsed time: 7089.577027 msecs" "Elapsed time: 6119.980053 msecs" "Elapsed time: 6135.949681 msecs" "Elapsed time: 6105.716229 msecs" "Elapsed time: 6035.137985 msecs" "Elapsed time: 6120.137894 msecs" "Elapsed time: 5858.970903 msecs" "Elapsed time: 5882.34203 msecs" "Elapsed time: 6120.462796 msecs"

So, definitely, as the structure gets more complicated, the slightly more sensible approach is pulling ahead.

But just for fun, as I'm going to bed:

user=> #<Var: user/ALL-TESTS> user=> (dotimes x 100 (eval `(deftest ~(gensym) "generated fake test" (:after nil) [] true))) nil user=> (dotimes x 100000 (eval `(deftest ~(gensym) "generated fake test" (:after ~(let [keys (keys ALL-TESTS)] (first (drop (rem (. RNG (nextInt)) (count keys)) keys)))) [] true))) Reflection warning, line: 957 - call to nextInt can't be resolved. (do (dotimes x 10 (time (unit-test-list-old))) (println "----------") (dotimes x 10 (time (unit-test-list))))

Ok, somewhere before generating a 100000-length tree of tests, it crashed, due to a lack of symbols. Fair enough. That test was just waving things around to see if I can.

clojure.lang.Compiler$CompilerException: REPL:749: PermGen space user=> (count (keys ALL-TESTS)) 42577

Revisiting generating my accessors.

However, going back one step, all that silly mucking around with gensyms and defines gives me an idea:

(map (fn [key] (eval `(def ~(sym (name *current-namespace*) (strcat "unit-test-" (name key))) (accessor unit-test ~key)))) (keys unit-test))

There. Now I know that, should I later decide to add more keys to that structure, the accessors will be there and waiting for me. Except that, in a freshly-loaded REPL (with this file loaded via load-file rather than dumping it in, I find that my accessors are no longer loaded.

All right, I'll take the hint, stop picking at this, and leave them as normal functions rather than generated ones.

Revisiting actually running the tests

So, I try running all my tests again. And the test function (which I haven't changed since I started that little digression into sorting my tests) still does what it's told, albeit with a tiny wart and a failed test.

user=> (test-all) "Elapsed time: 306.707138 msecs" Testing creating a listener : true "Elapsed time: 216.1839 msecs" Testing string->byte array conversion : true "Elapsed time: 11.579405 msecs" Testing listener open?/closed? predicates : true "Elapsed time: 10.170846 msecs" Testing closing a listener : true Could not connect to 127.0.0.1 on port 51345 "Elapsed time: 1064.018472 msecs" Testing creating a connection : true "Elapsed time: 46.276552 msecs" Testing running a complete connection : false --- Some tests failed.

And here's the offending test. I'll step through it one bit at a time and see what's going wrong in actuality.

(deftest test-connection-run "running a complete connection" (:after :all) ([] (let [listener (listener-run-in-background) result (connection-run)] (and (listener-open? listener) (not (listener-close listener)) (listener-closed? listener) (> 0 (. result (length))) (== 0 (. (connection-run) (length))))))) user=> (def listener (listener-run-in-background)) #<Var: user/listener> user=> (def result (connection-run)) #<Var: user/result> user=> (listener-open? listener) true user=> (listener-close listener) nil user=> (listener-closed? listener) true user=> (> 0 (. result (length))) Reflection warning, line: 306 - call to length can't be resolved. false user=> (. result (length)) Reflection warning, line: 307 - call to length can't be resolved. 28 user=> (> (. result (length)) 0) Reflection warning, line: 308 - call to length can't be resolved. true

Ok, so I messed up the ordering of greater-than and less-than. I find that I do this a lot in Lisp, because of how I read (> 0 ... mentally. Namely, as a single function, testing if what follows is greater than the integer that I've put there. Or, in other words, exactly backwards.

So, change that to (> (. result (length)) 0) and re-run the test function:

user=> (test-all) java.lang.NullPointerException at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:24) at user.test_connection_run.invoke(Unknown Source) at clojure.lang.AFn.applyToHelper(AFn.java:171) at clojure.lang.AFn.applyTo(AFn.java:164) at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213) at clojure.lang.Compiler.eval(Compiler.java:3086) at clojure.eval.invoke(boot.clj:635) at user.test_all.invoke(Unknown Source) at clojure.lang.AFn.applyToHelper(AFn.java:171) at clojure.lang.AFn.applyTo(AFn.java:164) at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213) at clojure.lang.Compiler.eval(Compiler.java:3086) at clojure.lang.Repl.main(Repl.java:59) "Elapsed time: 9.756547 msecs" Testing creating a listener : true "Elapsed time: 4.988344 msecs" Testing string->byte array conversion : true "Elapsed time: 26.513984 msecs" Testing listener open?/closed? predicates : true "Elapsed time: 10.239011 msecs" Testing closing a listener : true Could not connect to 127.0.0.1 on port 51345 "Elapsed time: 997.681828 msecs" Testing creating a connection : true Could not connect to 127.0.0.1 on port 51345

Erk. All right, that's getting a bit odd. Changing it back gets me the previous, broken test, but that's not exactly helpful. Looking at that dump, I can see that, first off, the error happened in test-connection-run, but, secondly, it happened when trying to invoke an instance method.

Further, on that line, the only one I've been changing, there is indeed an instance method. However, on a successful connection, even one with no data, it should return an empty string.

The answer here lies with the next line.

user=> (== 0 (. (connection-run) (length))) Reflection warning, line: 894 - call to length can't be resolved. java.lang.NullPointerException at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:24) at clojure.lang.Compiler$InstanceMethodExpr.eval(Compiler.java:950) at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2212) at clojure.lang.Compiler.eval(Compiler.java:3086) at clojure.lang.Repl.main(Repl.java:59) Could not connect to 127.0.0.1 on port 51345

Before, the failure at the successful length was causing and to stop and return false, in a "short-circuiting" behaviour that's fairly normal³.

Now, what that reminds me is that (connection-run) is going to return nil when it can't open a connection at all. So thus, I'm calling (length) on nil: (. nil (length)) which, deservedly, throws an error. So, in this case, the reflection warnings about length not being resolved were helping me rather than nagging.

(defn #^Int length-of-string [#^String string] (if string (. string (length)) 0)) user=> (length-of-string "asdf") 4 user=> (length-of-string nil) 0 (deftest test-connection-run "running a complete connection" (:after :all) ([] (let [listener (listener-run-in-background) result (connection-run)] (and (listener-open? listener) (not (listener-close listener)) (listener-closed? listener) (<= 0 (length-of-string result)) (== 0 (length-of-string (connection-run))))))) user=> (test-all) "Elapsed time: 75.76773 msecs" Testing creating a listener : true "Elapsed time: 3.349029 msecs" Testing string->byte array conversion : true "Elapsed time: 9.635583 msecs" Testing listener open?/closed? predicates : true "Elapsed time: 9.517131 msecs" Testing closing a listener : true Could not connect to 127.0.0.1 on port 51345 "Elapsed time: 944.630393 msecs" Testing creating a connection : true Could not connect to 127.0.0.1 on port 51345 "Elapsed time: 1001.659708 msecs" Testing running a complete connection : true --- All tests passed.

And, testing all of that, it does indeed work out the way it ought to, except for the aforementioned wart. To point it out, "Could not connect to 127.0.0.1 on port 51345" would appear to indicate an error, whereas I know that it's actually a valid part of both the tests for which it appears.

So, here's the problem: those errors are useful rather than throwing an exception when connecting manually, but they give the wrong indication (to me at least) when connecting automatically.

So, what I'd like is to shove any messages generated into something other than standard output, where I can inspect them if I so choose.

(defn test-all [] (let [test-out (new StringWriter)] (loop [test-list (unit-test-list) failure false] (let [test (first test-list) test-list (rest test-list)] (cond (not test) (do (if failure (println "---\nSome tests failed.") (println "---\nAll tests passed.")) (list failure test-out)) :t (let [test-result (time (binding [*out* test-out] (eval (list (second test)))))] (println "Testing" (first test) ":" test-result) (recur test-list (when (or (not test-result) failure) true))))))))

So, what that does is, before doing anything, creates a StringWriter outside the loop. Then, when it comes time to perform the test, temporarily re-binds out (the variable telling Clojure where to print to) to that StringWriter.

This means, now, that I can return the success/failure of the set of tests, as well as any error messages, as a list.

user=> (def test-results (test-all)) "Elapsed time: 8.808941 msecs" Testing creating a listener : true "Elapsed time: 3.364394 msecs" Testing string->byte array conversion : true "Elapsed time: 8.342401 msecs" Testing listener open?/closed? predicates : true "Elapsed time: 63.791348 msecs" Testing closing a listener : true "Elapsed time: 982.201928 msecs" Testing creating a connection : true "Elapsed time: 1002.468191 msecs" Testing running a complete connection : true --- All tests passed. #<Var: user/test-results> user=> test-results (nil Could not connect to 127.0.0.1 on port 51345 Could not connect to 127.0.0.1 on port 51345 )

Afterthoughts

So, now I have a very basic (albeit ostensibly extensible) testing framework, as well as a setup that generates that framework automatically.

So, now a few (code-dump-free) thoughts as I actually make sure that my in-file commentary is up to date.

Mistakes

What are the current downsides to this system and this approach? Well, for starters, the test framework only recognises nil and true. This means that, when an early test manages to hang and throw an exception, it brings down the test framework as well.

In laziness, I chose to use return values in order to not have to play too much with caught/thrown errors, and instead passed truth/falsity around. Not necessarily the best approach, but I haven't decided whether throwing/catching is better in this case⁵.

Also, I'll be the first to admit that I'm handling errors in a bit of a cheap way, and one that, were I actually turning this into a real component, rather than something that I'm building as the "build one to throw away" of my learning process, I'd have started to question at about the point that I was wrapping a binding around println in order to isolate my error handling away from my testing function.

Further, the 100 lines defining my simple testing framework should really be extracted from this client-server application. They're only loosely attached to each other, so I should be able to give them their own namespaces with minimal hassle. (I'll wait for the next Clojure release to do that, though, because I know I'll have to change a bunch in order to handle upcoming changes, and would prefer to discuss namespaces with myself once the new system is in.)

Lessons

On contrast, what have I learned? Well, for one, do not try to generate 17 000 lines of Lisp at once via an elisp command. Or, at least, save your document first. (I lost all of my poking at comparators via that gaffe.)

Also, that macros really aren't especially terrifying. Believe me or don't, but deftest was the first Lisp macro that I've ever written. I'm still tripping over order of operations and how forms get expanded and names get resolved, especially when dealing with things like def and load_file, but I do think that that falls simultaneously with me trying to be too clever and not actually understanding what it is that I'm doing.

As I write more, I keep finding new boot.clj functions that let me write a verbose expression a lot more succinctly and clearly. My new ones are with-open, some, not-any? and anything beginning with sort.

I'm finding that having the manual as a single, indexed document is making me try harder to use Clojure's functions for doing something, rather than rolling my own off of Java's library.

As I've found before, having a single unified environment in which to write code, then evaluate it selectively is a great asset. It means, for me at least, that I spend less time having to mentally page-swap as I try to remember what I was going to do next.

There's some caveats to that, though: Number one is that my current environment state doesn't necessarily match what I have written down. I found myself, this session, killing by REPL on occasion just to make sure that it was matching my work as exactly as possible.

Also, it makes me lazy about making sure that, e.g., functions are declared before another function references them. Which works just fine, right up until I try to load the file and realise that my selective evaluation has (yet again) created a file that needs minor reordering in order to load.

But that's a trivial irritation. And far outweighed by being able to write this (as HTML-heavy Markdown), manipulate the actual source of this program and run the REPL, all in the same environment, and all sharing data⁶.

Future / Goals

Well, beyond the nebulous goals I dropped at the end of my initial look, I now have a few more.

Cease using nil as an error value.
Move to separate test / application namespaces.
Look at existing functions that do the same thing (the closes and length-of-string come to mind) and push them into multimethods instead.

(Yes, the long-term geal here is to meander through all 21 chapters of the manual as I find needs.)
Explore wrapping an extant Java unit testing facility in Clojure rather than rolling my own (a nice learning exercise but not exactly a valid solution to anything beyond making myself try out the language.

Ok, it sounds like a fine idea for a project, but I'm trying not to let my focus wander all over the place, so I'll keep this experiment/application self-contained. Back
To clarify my failure to use comparators a little bit, a comparator is an object/function that grabs some items, compares them, and replies which one is greater.

user=> (sort (comparator >) [5 4 8 4 3 8 4 6]) (8 8 6 5 4 4 4 3) user=> (sort (comparator (fn [x y] (println x y (> x y)) >)) [5 4 8 4 3 8 4 6]) 5 4 true 4 8 false 8 4 true 3 8 false 8 4 true 4 6 false 4 3 true (5 4 8 4 3 8 4 6) user=> (sort (comparator (fn [x y] (println x y (>= x y)) >)) [5 4 4 4]) 5 4 true 4 4 true 4 4 true (5 4 4 4)

Here's where that falls apart for my purposes, though: when sorting numbers, letters, names, etc., I know the relative ordering of any two. On the other hand, with these tests, unless one of four conditions is met, the ordering of the two items is unknown.
1. :after is nil. The test comes before all others.
2. :after is :all. The test comes after all others.
3. Both tests have the same :after. They are equivalent.
4. One test's :name is the other test's :after. They have to go in that order.
My problem is that most of my comparisons do not meet these standards. Hence my sort functions both having a means to defer looking at an item until a condition is met. Back
This meaning, in a logical condition, to stop evaluating the condition as soon as its truth / falsity becomes known. Here, and in other tests, I was using it as somewhat of a shorthand to say, "If the test fails at any point, the entire test has failed."

On the other hand, using logical predicates (and, or, etc.) as control structure can quickly get unreadable at a glance.

(or (or (and (test1) (test2)) (test3)) (or (and (test4) (test5) (test6)) (and (test6) (test7) (or (test8) (test8a) (test8b))) (test9)))

A contrived example, sure, but I lost the flow of that logical tree midway through writing the test. I also tried to devise it so that the tree structure would bear at least a passing resemblance to something that had been modified to deal with a later condition⁴ Back
(For what it's worth, it says: Perform test1. On success, perform test2. On success of test1 and test2, return success. Otherwise, perform test3. On success of test3, return success. On failure of test3, try test4, test5, test6, stopping if there's failure. And so on, until, if all other tests have failed, return the success/failute of test9.) Back
Yes, I'm aware that the consistent (and possibly even correct) answer is throw/catch. However, for the purposes of a simple demo application, building Throwables to toss around strikes me as an unnecessarily obfuscating the mechanics of what I'm doing. Back
Emacs, 164 characters wide, divided into two columns, with a full column devoted to the REPL, 20 lines to the source and 40 to this text. Which raises a question: With the admonition to not write any function longer than a single screen, with whose screen length?

For the record, the longest function in this example is 28 lines and the shortest is 1 line long. That's likely a direct consequence to my having kept my source scrunched up with only a little window into it⁷. Back
Done via the script:

(save-excursion (goto-char (point-min)) (re-search-forward "^ *$" (point-max) nil) (message (mapconcat (lambda (lc) (format "%d" lc)) (let ((lcount nil)) (while (not (eobp)) (forward-sexp) (setq lcount (append lcount (list (count-lines (save-excursion (backward-sexp) (point)) (point)))))) lcount) " ")))

And that's why Emacs one-liners are scary. Because before you know it, they're actually 15 lines long, all scrunched into an Eval: prompt. Back

My first look at Clojure: Source

2008-02-02T22:42:00.001-05:00

Here's the resulting source code from my experimentation.

(import '(java.net BindException ServerSocket Socket)
        '(java.lang.reflect InvocationTargetException)
        '(java.util Date)
        '(java.io InputStream OutputStream)
        '(java.util.concurrent Executors))


;;; Utility functions

(defn current-time []
  (. (new Date) (toString)))

(defn byte-arr-from-string [str]
  (. str (getBytes)))

(defn test-byte-array-from-string
  ([]
   (test-byte-array-from-string (current-time)))
  ([str]
   (let [barr (byte-arr-from-string str)
         bseq (map (comp char (appl aget barr))
                   (range (alength barr)))
         chseq (map char str)]
     (and (== (alength barr)
              (count bseq)
              (count chseq))
          (nil? (first (filter false?
                               (map eql?
                                    bseq
                                    chseq))))))))

(defn string-from-byte-sequence [coll]
  (reduce strcat
          (map char coll)))


;;; Listener functions
;;; These control the server

(defn listener-new [port]
  (try (new ServerSocket port)
       (catch BindException except
              (println "Address is already in use."))))

(defn listener-wait [listener]
  (. listener (accept)))

(defn listener-close [listener]
  (. listener (close)))

(defn listener-send [lsocket] 
  (.. lsocket
      (getOutputStream)
      (write (byte-arr-from-string (current-time))))
  (.. lsocket (getOutputStream) (close))
  lsocket)

(defn listener-run [listener port]
  (loop [socket nil]
    (if (. listener (isClosed))
      listener
      (do (when socket
            (. (listener-send socket) (close)))
          (recur (listener-wait listener))))))

(defn listener-run-in-background [port]
  (let [listener (listener-new port)
        exec (. Executors (newSingleThreadExecutor))
        run (appl listener-run listener port)]
    (when listener
      (. exec (submit run)))
    listener))

;;; Connection functions
;;; These control the client.

(defn connection-new
  ([port]
   (connection-new "127.0.0.1" port))
  ([address port]
   (try (doto (new Socket address port)
          (setSoTimeout 5000))
        (catch InvocationTargetException except
               (println (strcat "Could not connect to "
                                address
                                " on port "
                                port))))))

(defn connection-read [conn]
  (let [instream (. conn (getInputStream))
        reader (fn [] (try (. instream (read))
                           (catch InvocationTargetException except
                                  -1)))]
    (loop [bytes nil
           current-byte (reader)]
      (if (== current-byte -1)
        bytes
        (recur (concat bytes (list current-byte))
               (reader))))))

(defn connection-close [conn]
  (. conn (close)))

(defn connection-run [port]
  (let [conn (connection-new port)
        str (when conn
              (string-from-byte-sequence (connection-read conn)))]
    (when conn 
      (connection-close conn))
    str))

My first look at Clojure

2008-02-02T19:00:00.002-05:00

Preliminaries

So, Clojure is a pretty little language, a Lisp¹ for the Java Virtual Machine. That, in and of itself, doesn't say much for or against it.

I've been playing with it for a few days now and really finding that I like more than I dislike and that the bits that I found to be difficult were, well, being fixed or changed.²

In the spirit of Dive Into Python, this is the place where I should offer up a snippet of finished code and then walk through it. Unfortunately, only having used Clojure a few days, I'm still in the first romance stage, rather than the stacks of working code stage.

So, rather than offering a block of code and then dissecting it, or alternately doing the thing that seems common in ruminations on functional languages³ and offering up an example like:

(defn fibs
  ([] (fibs 0 1))
  ([a] (fibs a 1))
  ([a b]
   (lazy-cons a (fibs b (+ a b)))))

(defn nthfib [n]
  (first (reverse (take n (fibs)))))

That, while pretty in demonstrating infinite sequences and a simple form of multiple dispatch, does little to offer a reason to actually use the language⁴. Instead, I'm going to try my hand at two things that have frustrated me in the past with Lisp. Namely sockets and threads⁵.

Goals

So, what I want is:

A client that:
1. Connects to a server.
2. Listens and prints out what it hears.
A server that:
1. Listens on a port
2. Creates a new thread on connection.
3. Sends the current ctime to the client.
4. Disconnects.

Or, essentially, the most primitive version of NTP imaginable.

As Clojure runs on the JVM, first step becomes to open up JavaDoc and check what objects I actually want. I'll start by importing ServerSocket and Socket.

user=> (import '(java.net ServerSocket Socket))
nil

The most primitive client and server possible

At this point, all I want to do is make functions to open the connections, so:

(defn listener-new [port]
  (new ServerSocket port))

(defn connection-new
  ([port]
   (connection-new "127.0.0.1" port))
  ([address port]
   (new Socket address port)))

listener-new is a function that binds on a port and returns a Java ServerSocket bound to that port. connection-new is a function that takes either a port (in which case it connects to localhost) or an address and a port and returns a Socket connected to that.

I'll try to explain new terms and forms as I go. The ones introduced here are defn, which creates a function, with the arguments in square brackets, returning whatever the last value in its body is.

So, (defn [port] (new ServerSocket port)) makes a function that takes one argument and then calls the new function, which instantiates a Java object with whatever arguments are passed. It then returns whatever new returns, which, unless there's an error, should be a ServerSocket.

defn can also accept multiple sets of arguments, in order to either do a certain amount of pattern matching on them or, in this case, provide a default value to a function.

The next step, then, is to try each of these and see if they work.

user=> (def serversocket (listener-new 56894))
#<Var: user/serversocket>
user=> serversocket
ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=56894]
user=> (. serversocket (close))
Reflection warning, line: 17 - call to close can't be resolved.
nil

So, the serversocket will open just fine, though I receive a (relatively harmless) type warning when I close it. That can be removed either by ensuring that the close method call ensures (via a type annotation) that what is passed to it is indeed a serversocket and that what is returned by listener-new is also a serversocket.

For the present, though, I don't actually care about this warning. Let's try connection-new.

user=> (def socket (connection-new 56894))
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
        at java.lang.reflect.Constructor.newInstance(Unknown Source)
        at clojure.lang.Reflector.invokeConstructor(Reflector.java:125)
        at user.connection_new.invoke(Unknown Source)
        at user.connection_new.invoke(Unknown Source)
        at clojure.lang.AFn.applyToHelper(AFn.java:173)
        at clojure.lang.AFn.applyTo(AFn.java:164)
        at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213)
        at clojure.lang.Compiler$DefExpr.eval(Compiler.java:253)
        at clojure.lang.Compiler.eval(Compiler.java:3086)
        at clojure.lang.Repl.main(Repl.java:59)
Caused by: java.net.ConnectException: Connection refused: connect
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(Unknown Source)
        at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
        at java.net.PlainSocketImpl.connect(Unknown Source)
        at java.net.SocksSocketImpl.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at java.net.Socket.<init>(Unknown Source)
        at java.net.Socket.<init>(Unknown Source)
        ... 13 more

Oh dear. Well, that didn't work. Though, I suppose having a port open to connect to might just help.

user=> (def ssocket (listener-new 56894))
#<Var: user/ssocket>
user=> (def socket (connection-new 56894))
#<Var: user/socket>
user=> (. socket close)
Reflection warning, line: 45 - reference to field close can't be resolved.
java.lang.IllegalArgumentException: No matching field found
        at clojure.lang.Reflector.getInstanceField(Reflector.java:175)
        at clojure.lang.Compiler$InstanceFieldExpr.eval(Compiler.java:744)
        at clojure.lang.Compiler.eval(Compiler.java:3086)
        at clojure.lang.Repl.main(Repl.java:59)
user=> (. socket (close))
Reflection warning, line: 53 - call to close can't be resolved.
nil
user=> (. ssocket (close))
Reflection warning, line: 54 - call to close can't be resolved.
nil

And that does it. The error in that block was because I referred to close as a field: close, rather than a method: (close). On the other hand, this doesn't do anything interesting. It just makes and closes the listener and the connection.

Now, the . operator. I've used it a few times so far, so I should clarify what it does. It looks up the second value within the first one. Thus, (. socket (close)) looks for a close method inside the instance socket, then calls it.

Next, I need to add behaviour to the server to detect connection and send the ctime, as well as making the client recognise that data is being passed to it.

Adding data

(import '(java.net ServerSocket Socket)
        '(java.util Date))

(defn current-time []
  (. (new Date) (toString)))

(defn listener-new [port]
  (new ServerSocket port))

(defn listener-wait [listener]
  (. listener (accept)))

(defn listener-send [lsocket]
  (.. lsocket (getOutputStream) (write (current-time)))
  lsocket)

(defn listener-close [listener]
  (. listener (close)))

(defn listener-run [port]
  (let [listener (listener-new port)]
    (listener-send (listener-wait listener))
    (listener-close)))

So now I've added a function that dumps the current time as a string, as well as functions to accept a connection, send that current time and disconnect.

Trying it out, I get:

user=> (current-time)
"Sat Feb 02 12:16:19 EST 2008"
user=> (listener-run 51245)

And then it hangs. Unsurprisingly, really, as it's blocking at (. listener (accept)), which waits for a connection before returning the socket referring to that connection. While this would be a good place to make the thread that I was talking about earlier, for the moment, I just want to see if this actually works. So, what I'll do is start the server in a separate REPL⁶.

So, in the one REPL, I create the server again:

user=> (listener-run 51345)

And in the other, I connect to it:

user=> (connection-new 51345)
Socket[addr=/127.0.0.1,port=51345,localport=1668]

Which results in this in the first REPL.

java.lang.IllegalArgumentException: No matching method found: write
        at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:59)
        at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:26)
        at user.listener_send.invoke(sockets.clj:15)
        at user.listener_run.invoke(sockets.clj:23)
        at clojure.lang.AFn.applyToHelper(AFn.java:173)
        at clojure.lang.AFn.applyTo(AFn.java:164)
        at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213)
        at clojure.lang.Compiler.eval(Compiler.java:3086)
        at clojure.lang.Repl.main(Repl.java:59)

Debugging data transmission

Now, despite this being an error, it does show that the listener made it to listener-send, as that's where write is called. So, I know that it's accepting network connections. Unfortunately, it's crashing when it receives them.

So, my next step is to look at what that command actually does:

user=> (macroexpand '(.. lsocket (getOutputStream) (write (current-time))))
(. (. lsocket (getOutputStream)) (write (current-time)))

The macroexpand function takes a macro passed to it and evaluates it (as well as macros inside that one, recursively). So I know that write is being called on the outputstream of the socket. For debugging purposes, I'll rewrite listener-send procedurally.

(defn listener-send [lsocket]
  (println lsocket)
  (let [outputstream (. lsocket (getOutputStream))]
    (println outputstream (current-time))
    (. outputstream (write (current-time)))
    lsocket))

This is the same function, only it's spitting out its state at each stage.

user=> (listener-run 51345)
java.lang.IllegalArgumentException: No matching method found: write
        at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:59)
        at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:26)
        at user.listener_send.invoke(sockets.clj:18)
        at user.listener_run.invoke(sockets.clj:26)
        at clojure.lang.AFn.applyToHelper(AFn.java:173)
        at clojure.lang.AFn.applyTo(AFn.java:164)
        at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213)
        at clojure.lang.Compiler.eval(Compiler.java:3086)
        at clojure.lang.Repl.main(Repl.java:59)
Socket[addr=/127.0.0.1,port=1789,localport=51345]
java.net.SocketOutputStream@26dbec Sat Feb 02 13:08:22 EST 2008

This time, with output, I can see that I indeed get an OutputStream. Time to play with it and see what it's doing.

user=> (import '(java.io InputStream OutputStream)
nil
user=> (def li (listener-new 51345))
#<Var: user/li>
user=> (def sock (listener-wait li))
#<Var: user/sock>
user=> (def by (. (current-time) (getBytes)))
Reflection warning, line: 46 - call to getBytes can't be resolved.
#<Var: user/by>
user=> (map (comp char (appl aget by)) (range (alength by)))
(\S \a \t \space \F \e \b \space \0 \2 \space \1 \3 \: \4 \9 \: \4 \4 \space \E \S \T \space \2 \0 \0 \8)
user=> (def ostream (. sock (getOutputStream)))
Reflection warning, line: 48 - call to getOutputStream can't be resolved.
#<Var: user/ostream>
user=> (. ostream (write by))
Reflection warning, line: 49 - call to write can't be resolved.
nil
user=> (listener-close li)
nil

So, there's the catch. When I pass current-time directly to the OutputStream's write method, what's expected is a write( String s ). What actually exists is write( Byte[] barr ).

The map done on by is worth looking at, though. I used it to find out what exactly by was set to, it being a Java array. So, first off, I get a list of all numbers from 0 to the length of by: (range (alength by)). That's the indices to the array. Then (comp char (appl aget by))⁷ is called on each of those.

Converting to and from byte arrays.

This actually raises a valid assertion. When creating a byte array from a string, the character values in the byte array should be the same as those in the string.

So, I'll create a function (that I'll use in a minute anyway) that makes a Java byte array out of a string. And, because I'm starting to get more cautious after my repeated bugs, I'll make a simple test to assert that its results are the same as the string from which it was made.

(defn byte-arr-from-string [str]
  (. str (getBytes)))

(defn test-byte-array-from-string
  ([]
   (test-byte-array-from-string (current-time)))
  ([str]
   (let [barr (byte-arr-from-string str)
         bseq (map (comp char (appl aget barr))
                   (range (alength barr)))
         chseq (map char str)]
     (and (== (alength barr)
              (count bseq)
              (count chseq))
          (== 0
              (count (filter false?
                             (map eql?
                                  bseq
                                  chseq))))))))

There. A String to Byte[] converter and a unit test. Now, to run the test:

user=> (test-byte-array-from-string)
true
user=> (test-byte-array-from-string "qwertyuiop")
true

And the test indicates that there is no loss per se in converting the string to a byte array. That done, I'll add the conversion to a byte array into listener-send. If that works, I'll collapse the expanded debuggy listener-send back down.

(defn listener-send [lsocket]
  (println lsocket)
  (let [outputstream (. lsocket (getOutputStream))]
    (println outputstream
             (current-time)
             (instance? outputstream java.io.OutputStream))
    (. outputstream (write (byte-arr-from-string (current-time))))
    lsocket))

And now a further round of testing:

user=> (listener-run 51345)
Socket[addr=/127.0.0.1,port=2564,localport=51345]
java.net.SocketOutputStream@88e2dd Sat Feb 02 15:23:04 EST 2008 true
nil

So, the server says (more or less) that it did its steps and nothing went wrong. Let's check this from the other side:

user=> (def conn (connection-new 51345))
#<Var: user/conn>
user=> (def ins (. conn (getInputStream)))
Reflection warning, line: 69 - call to getInputStream can't be resolved.
#<Var: user/ins>
user=> ins
java.net.SocketInputStream@453807
user=> (. ins (read))
Reflection warning, line: 71 - call to read can't be resolved.
83
user=> (. ins (read))
Reflection warning, line: 72 - call to read can't be resolved.
97

And, at the other end, I get the bytes that (I think) I sent. Time to wrap up my client connection and get it to read the entire input stream.

Reading the transmitted data

(defn string-from-byte-sequence [coll]
  (reduce strcat coll))

(defn connection-new
  ([port]
   (connection-new "127.0.0.1" port))
  ([address port]
   (new Socket address port)))

(defn connection-read [conn]
  (let [instream (. conn (getInputStream))]
    (loop [bytes nil
           current-byte (. instream (read))]
      (if (== current-byte -1)
        bytes
        (recur (conj bytes current-byte)
               (. instream (read)))))))

(defn connection-close [conn]
  (. conn (close)))

(defn connection-run [port]
  (let [conn (connection-new port)
        str (string-from-byte-sequence (connection-read conn))]
    (connection-close conn)
    str))

So, what this is trying to do is open a connection, read from it until the number -1 is found and then returning the resulting sequence of bytes.

Why -1? Because the JavaDoc for InputStream::Read) says:

The value byte is returned as an int in the range 0 to 255. If no byte is available because the end of the stream has been reached, the value -1 is returned. This method blocks until input data is available, the end of the stream is detected, or an exception is thrown.

So I'm trying -1 first, as by the time I'm reading, listener-close should have been called, closing the socket.

Bugs with reading the data

user=> (connection-run 51348)
"5648485032848369325048584949585449325048329810170321169783"

So, it reads fine, but it appears that something went wrong in translating the byte sequence into a string again. Looking at string-from-byte-sequence, it concatenates the string values of the items in the collection.

(defn string-from-byte-sequence [coll]
  (reduce strcat coll))

So, I'll try concatenating two numbers:

user=> (strcat 64 65)
"6465"

Right, not what I was intending at all. So, before I concatenate the numbers, I'll need to convert them to characters.

user=> (strcat (char 64) (char 65))
"@A"

That worked a lot better. And tells me what I need to do to fix the function:

(defn string-from-byte-sequence [coll]
  (reduce strcat
          (map char coll)))

Because map is lazy, it means that simply using (strcat (map char coll)) gets me the string value of map's returned FnSeq instead of the concatenated characters. The reduce acts to apply strcat to every element of the entire sequence.

And, connecting to a new server, I get:

user=> (connection-run 51349)
"8002 TSE 15:13:61 20 beF taS"

Whoops, almost it! Only backwards. Checking the byte sequence as it comes in, I find that it is indeed reversed when returned from connection-read. So, I test what connection-read does to build the byte sequence:

user=> (conj (conj nil 1) 2)
(2 1)

So I check the documentation for conj and indeed:

Conj[oin]. Returns a new collection with the item 'added'. (conj nil item) returns (item). The 'addition' may happen at different 'places' depending on the concrete type.

An amendment to connection-read later:

(defn connection-read [conn]
  (let [instream (. conn (getInputStream))]
    (loop [bytes nil
           current-byte (. instream (read))]
      (if (== current-byte -1)
        bytes
        (recur (concat bytes (list current-byte))
               (. instream (read)))))))

And I finally get the desired result:

user=> (connection-run 51354)
"Sat Feb 02 16:44:09 EST 2008"

Bugfixing the primitive NTP server

So, what I now have is a server that waits for a connection on a given port, then sends the time and a client that makes a connection and reads the time. What I don't have is a bug-free or elegant version of this.

Time for me to run down the list of bugs and desires that I know of at the moment:

The server fails to clean up its port correctly, meaning that throughout this testing, I've been incrementing ports any time I choose to keep the server's REPL open.
The client will hang, waiting for data in its stream, when fed a port that is open but not responding.
The server is single-threaded still. This is the only one of my six test-application goals left undone. But making it not block the REPL while waiting for input will make testing far easier.
No Java exceptions are currently handled, despite their being thrown.
Ten reflection warnings when I load the file into my REPL.
test-byte-array-from-string is the only test attached. The behaviour of all the functions should be testable.
It would be nice (though not useful in this case) to represent the data stream as a lazy sequence, rather than expecting that it all be here.

So, I'll address these in order, starting with the server port.

The server fails to clean up its port...

I'll start by looking at what listener-run actually does.

(defn listener-run [port]
  (let [listener (listener-new port)]
    (listener-close (listener-send (listener-wait listener)))
    listener))

It binds listener to a new ServerSocket listening on port.
It passes that ServerSocket to listener-wait, which calls the accept method, returning the opened socket.
That socket is passed to listener-send, which gets its OutputStream and writes the current time to the stream, returning the socket.
That socket is passed on to listener-close, which closes it.
The listener is returned.

And thatfourth step would likely be the problem here. Rather than closing the listener, it's closing the socket acquired from accept, which means that rather than five, I only have four goals complete.

So, I'll try:

(defn listener-run [port]
  (let [listener (listener-new port)]
    (listener-send (listener-wait listener))
    (listener-close listener)
    listener))

But that has the net result that now, the server still holds onto that address, but the client hangs, waiting for data. A little bit of prodding later and I have this:

(defn listener-send [lsocket] 
  (.. lsocket
      (getOutputStream)
      (write (byte-arr-from-string (current-time))))
  (.. lsocket (getOutputStream) (close))
  lsocket)

(defn listener-run [port]
  (let [listener (listener-new port)]
    (. (listener-send (listener-wait listener)) (close))
    (listener-close listener)
    listener))

Which successfully sends the time across the (loopback) network, then releases the port.

The client will hang...

Not exactly true. I wrote that forgetting that I had random ServerSockets poking around. In fact, what happens when the client is passed a closed port is:

user=> (connection-new 1500)
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
        at java.lang.reflect.Constructor.newInstance(Unknown Source)
        at clojure.lang.Reflector.invokeConstructor(Reflector.java:125)
        at user.connection_new.invoke(Unknown Source)
        at user.connection_new.invoke(Unknown Source)
        at user.connection_run.invoke(Unknown Source)
        at clojure.lang.AFn.applyToHelper(AFn.java:173)
        at clojure.lang.AFn.applyTo(AFn.java:164)
        at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213)
        at clojure.lang.Compiler.eval(Compiler.java:3086)
        at clojure.lang.Repl.main(Repl.java:59)
Caused by: java.net.ConnectException: Connection refused: connect
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(Unknown Source)
        at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
        at java.net.PlainSocketImpl.connect(Unknown Source)
        at java.net.SocksSocketImpl.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at java.net.Socket.<init>(Unknown Source)
        at java.net.Socket.<init>(Unknown Source)
        ... 13 more

Conclusion: I need to not only handle connection timeouts but also refused connections. As this happens in connection-new, which only has one instruction, I need to wrap the new socket in a try/catch block.

(defn connection-new
  ([port]
   (connection-new "127.0.0.1" port))
  ([address port]
   (try (new Socket address port)
        (catch InvocationTargetException except
               (println (strcat "Could not connect to "
                                address
                                " on port "
                                port))))))

(defn connection-run [port]
  (let [conn (connection-new port)
        str (when conn (string-from-byte-sequence (connection-read conn)))]
    (when conn 
      (connection-close conn))
    str))

So now trying to connect to an invalid port spits an error message and returns nil. On the other hand, it still hangs when pushed to connect to a port that doesn't behave like it expects (send data, close socket).

user=> (connection-new 1280)
Could not connect to 127.0.0.1 on port 1280
nil

The answer, then, is to give the client a timeout after which it assumes that no data is coming.

(defn connection-new
  ([port]
   (connection-new "127.0.0.1" port))
  ([address port]
   (try (doto (new Socket address port)
          (setSoTimeout 5000))
        (catch InvocationTargetException except
               (println (strcat "Could not connect to "
                                address
                                " on port "
                                port))))))

(defn connection-read [conn]
  (let [instream (. conn (getInputStream))
        reader (fn [] (try (. instream (read))
                           (catch InvocationTargetException except
                                  -1)))]
    (loop [bytes nil
           current-byte (reader)]
      (if (== current-byte -1)
        bytes
        (recur (concat bytes (list current-byte))
               (reader))))))

So now, if nothing comes in in five seconds, connection-read signals an error and flags the current byte as -1, signalling an end of the transmission.

doto, which I used to set the socket timeout, takes an instance, evaluates the following body on it, then returns the changed instance. I used it in this case because setSoTimeout returns void while changing the object's state.

user=> (connection-run 80)
""

The server is single-threaded...

Oh bugger. I really should have put this lower on the list. I did not want to deal with threading today.

So, that leaves me the question: What should the server do? For starters, I'll make its instances persistant. That way, with a single server open, I can connect repeatedly without restarting the server each time.

(defn listener-run
  ([port]
   (listener-run (listener-new port)
                 port))
  ([listener port]
   (loop [socket nil]
     (when socket
       (. (listener-send socket) (close)))
     (recur (listener-wait listener)))
   listener))

However, what I want is for that same function to be pushed into the background. So, for starters, I'll give the loop an exit condition:

(defn listener-run [listener port]
  (loop [socket nil]
    (if (. listener (isClosed))
      listener
      (do (when socket
            (. (listener-send socket) (close)))
          (recur (listener-wait listener))))))

Now, if I happen to have captured the listener, I can kill the server process by closing the listener. This isn't useful yet, as the server will still loop infinitely when it's launched.

To make sure that I'm not holding on to inaccessable listeners, I also removed my earlier convenience form that created a listener for me.

So, time to push listener-run into its own execution thread.

(defn listener-run-in-background [port]
  (let [listener (listener-new port)]
    (listener-run listener port)
    listener))

There's a starting form. Though all it does at the moment is run the listener in the same way as the bits I removed above, I'll now poke at adding Java's concurrency bits into it.

(defn listener-run-in-background [port]
  (let [listener (listener-new port)
        exec (. Executors (newSingleThreadExecutor))
        run (appl listener-run listener port)]
    (. exec (submit run))
    listener))

So, that's a first attempt. It creates a pool of one thread, wraps running the server in an anonymous function and punts the function off to the thread pool.

And testing it:

user=> (def li (listener-run-in-background 51345))
#<Var: user/li>
user=> li
ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=51345]
user=> (connection-run 51345)
"Sat Feb 02 20:49:15 EST 2008"
user=> (connection-run 51345)
"Sat Feb 02 20:49:16 EST 2008"
user=> (connection-run 51345)
"Sat Feb 02 20:49:17 EST 2008"
user=> (. li (close))
Reflection warning, line: 560 - call to close can't be resolved.
nil
user=> li
ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=51345]
user=> (. li (isClosed))
Reflection warning, line: 562 - call to isClosed can't be resolved.
true
user=> (connection-run 51345)
Could not connect to 127.0.0.1 on port 51345
nil

You know, I was expecting that to not work. That really shouldn't have been that easy. To be fair, this doesn't do anything with that thread pool that I created. So I'll shut it down once I'm done running the server.

(defn listener-run-in-background [port]
  (let [listener (listener-new port)
        exec (. Executors (newSingleThreadExecutor))
        run (fn [] (do (listener-run listener port)
                       (. exec (shutdown))))]
    (. exec (submit run))
    (list exec listener)))

user=> (connection-run 51345)
""
user=> li
(java.util.concurrent.Executors$FinalizableDelegatedExecutorService@19bb25a ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=51345])
user=> (connection-run 51345)
""
user=> (. (first li) (isShutdown))
Reflection warning, line: 114 - call to isShutdown can't be resolved.
true

And that, on the other hand, fails miserably, as do my next six attempts. Then it finally occurs that, since in my original version, the executor is empty and unreferenced, all I'm really doing, as I try to bind exec inside itself with increasingly complex combinations of let, do, appl and fn, is creating circular references, and that I'd really just be a lot better off letting that ExecutorService fall on the floor and assuming that, because it can't be reached, it'll get picked up by the garbage collection⁸.

No Java exceptions are currently handled...

Well, I dealt with two of the three exceptions that I'm seeing routinely. Better catch the other one, the BindException when trying to launch two servers at the same port.

(defn listener-new [port]
  (try (new ServerSocket port)
       (catch BindException except
              (println "Address is already in use."))))

And checking that with an open server:

user=> (def li (listener-run-in-background 51345))
#<Var: user/li>
user=> (def li (listener-run-in-background 51345))
Address is already in use.
#<Var: user/li>
user=> li
nil

Afterthoughts

I'll count that one as done and take a look back at what needs doing. Type/reflection warnings, unit tests and cleaning my hacky "just get it done" code. Really, not bad for an afternoon's learning.

Further things to play with, from this tangent at least, seem to be: serialization: seeing just what I can push as an array of bytes, poking at java.lang.Concurrent and seeing how it reacts.

But as someone who's written more Lisp than Java in his life, I have to say, Clojure had me at the (.) function. Lisp plus library support is making this a very neat language to learn, and unlike my previous forays into Common Lisp, I haven't hit a wall where I found myself needing to reimplement something common⁹.

Also, despite fairly routinely using (basic) Java objects in this exercise, I never felt like I was having to coerce things to fit a foreign function interface. For that matter, this approach felt more or less like the right way to use objects: as a very rich system of types, rather than as a way to control program structure and flow.

It's not Common Lisp and it's not Scheme. It's its own dialect. Back
To wit:
- Namespace examination and the capacity to work in multiple files without having to think too hard about how imports interact. (Fixed in the latest version with namespaces becoming first-class citizens.)
- Metadata being applicable to everything, not just symbols and sequences. (On the agenda to be added.) Back
Clojure counts, for all intents and purposes, as a functional language. It's certainly less strict than Haskell, but on a par with the ML family in terms of being functional. Back
That naive implementation ran out of heap space at 100000 numbers. Likely binding it to a var would help.

After testing, apparently not.Back
I'm aware that there's lisps with good socket implementations and ones with good threading implementations. It's just that, for me at least, I've never found the two things in the same place at the same time when I needed them. Back
REPL: Read Evaluate Print Loop. A shell in which you interact with your program and the language.

In theory, I could have written all of this within the Clojure REPL. In practice, it's far simpler to write functions in a separate file and then send them to the REPL.

This means that, when I do something silly like causing the REPL to hang due to waiting for a network connection, I can just reload my file of functions.

Also, it does mean that, at the end of the day, I have my functions in useable and modifiable form, rather than as java bytecote that will be lost when I close the REPL. Back
This is an admittedly Haskellish way to build a function. In this case, its results are directly equivalent to the anonymous function (fn [x] (char (aget by x))).

To dissect this a little more, appl takes a function and at least one argument, then returns a new function that takes the rest of the original function's arguments. In other words, it applies the arguments given to it partially.

This means that my (appl aget by) creates a new function equivalent to (fn [a] (aget by a)).

comp, on the other hand, is function composition. For f(a) and g(b) f .comp. g is equivalent to f( g( b ) ) or (f (g a)). In other words, it takes functions and makes a new function that applies them from right to left to its arguments.

Were I expanding out this function completely, on expanding appl, it becomes: (comp char (fn [a] (aget by a))) and then on composing char to it, (fn [b] (char ((fn [a] (aget by a)) b))).

In this case, I used comp and appl because I found the resulting form easier to write and apply. By contrast, there are many circumstances where I'd prefer a simple anonymous function. Back
In my defense here, I should point out that I was reading the C++ Frequently Questioned Answers last week, and it may have been channeling a few bad memories. Back
To be fair, one of those involved the aforementioned choice between getting sockets or threads, before throwing up my hands in dismay.Back

wet pixels

Spiraling Twist Hat

Casting on

Brim

Crown

Decreasing

Golden Spiral Blanket - One square metre

Twirling twist socks - One foot

Woken by socks

Sierpinski/Koch Scarf - Halfway there

Mutations

Braid-pattern pullover -- Prelude

Ravelry Hack #1 - Counting downloads

Installation

Knitting is an acceptable Lisp

Swatch, knit thyself

Why I like circular knitting

Shadow Check Socks - Pattern

Requirements

Measurements

Instructions

Casting On

Heel Stitch Toe

Knitting the body of the sock

Knitting a gusseted heel

Heel Stitch

Turning the heel

Picking up from the heel flap

Gusseting

Seed Rib

Ankle

Calf

Embedded Moss Stitch Ribbing

Shadow Check Socks - One down!

Stitch Patterns - Charts

Seed Rib

Mistake Stitch Rib

Embedded Moss Stitch Rib

Stitch patterns

Shadow Check Socks -- Time to start the heel flap

Upper pattern

Shadow Check - Right Side

Shadow Check - Wrong Side

Dogwood Cabled Hat -- Pattern Amendments

Simple Textured Socks -- Rip number 3

Heel Stitch Toe

Simple Textured Socks

Shadow check (circular)

Heel Stitch

Two-Piece Knotwork Hat

Requirements

Instructions

Brim

Casting on

Grafting

Body

Picking up

Knitting the body

Reducing

Finishing

Chart

Brim

Body

Spiral-stitch hat

Two-Piece Knotwork Hat -- Ripping

Picking up a knit selvedge

Preparation

Picking Up

Equalizing Sizes

Increasing

Decreasing

Actually changing the number of stitches

Two-Piece Knotwork Hat -- Grafting and Failure

Knit Stitch

Purl Stitch

Two-piece Knotwork Hat -- Brim 1

Dogwood Cabled Hat

Joy of Cables Hat -- Completed extension

Joy of Cables Hat -- extension

Broad Street Mittens - Sleeve part 2