Ashton Kemerling

AK

The Dangers of Partisanship

Anyone who knows me personally knows that I am a huge Clojure fan. I could go on and on about why it’s the best; the regularity, the macros, ClojureScript, core.logic, etc. etc.

But last week I attended LambdaJam, which was awesome by the way, and I came back with a different opinion. No, I don’t think Haskell or Erlang tops out Clojure, I’m still probably going to reach for Leiningen for all my personal projects. But I learned that it’s a good idea to keep an eye on other languages, since no language will ever be so good that you never need to touch anything else.

In particular I learned the joy of the APL based languages, especially J. I learned that the F# people have a novel way of dealing with statically typing external resources via type providers, and I learned that Haskell can indeed be tight and clean despite what all those monad tutorials imply. I will probably take each of these subjects apart in separate posts in the future, I’m still trying to pick up bits and pieces of my thoughts after several intense key notes overloaded my brain.

What’s the take away? That functional languages are a lot more diverse than most OOP/procedural languages, and that there’s a lot that can be learned from them. So instead of picking your favorite and only learning that, you really should try to learn several to cover your bases.

Stealing Terminology

One of my favorite tricks is to borrow terminology from other walks in life. This is particularly important for me, since it’s all too easy for engineers to end up living and talking in pure engineering speak. Looking to other careers for the correct words to describe something allows one to express ideas that might not be easily communicated otherwise.

If you look carefully, you can see examples of this in engineering blog posts. People talk about fighting entropy, or the metaphysics of data. These are terms that did not originally have any engineering context whatsoever, being brought in to express ideas and concepts about another subject.

My most recent borrowed term is Force Multiplier. A force multiplier is anything that allows an group or individual to be more effective without an increase in effort. It’s normally a military term referencing how better weather forecasts, equipment, or intelligence can allow a unit to accomplish much more without increasing effort/losses. I like to use it in reference to things like unit tests and automation, that make me more effective as an engineer without costing me more time. If I don’t have to babysit a build waiting to type in the next step, then I can get back to work while a CI server works for me. I’m instantly more effective without actually spending more time at work. With a comprehensive suite of unit tests I can spend less time worrying and validating that my refactoring has damaged the quality of the system, thus I can get more work done in the same period of time.

I’m sure there are far more examples of this than I’ve listed, but they’re always fun to find and analyze.

Fast Cheap Good

In the distant past (1950’s or so), project managers and engineers came up with what is known as the project management triangle: fast, cheap, or good; pick two.

While software engineering can be very different from mechanical, it does at least share the same project management setup. Quality software designed cheaply will be late, cheap software released early will be poor in quality, and quality software released on time will be expensive. These differences come from the quality and number (thus cost) of the managers and engineers, the choice of methodologies, scope of features, and internal organizational setups.

What is different is the fact that software engineers aren’t limited by physics the way that our mechanical brethren are. With few exceptions for high performance computing, the limitation of most software projects is the imagination and effort of its engineers, not hard limits in manufacturing technologies or physics. Combine this with a fad-heavy market for programming methods (scrum! extreme! agile! pair!), and it can be very tempting to assume that we can find the perfect balance with the correct management processes and the right methodology.

This is false, of course. Management and methodology is about dealing with the communication overhead when enough people are working on an project. The pipe dream of management and methodology is for a group of N producers to produce N times more than one person alone. This is of course rubbish, as the Mythical Man Month demonstrated handily, it’s simply impossible to manage or process your way to good, cheap, and fast.

So what’s the point of it all then? Why don’t we just go back to waterfall? Because the point of agile, scrum, pair programming and friends is not to get us all the way to good, cheap, and fast. The point is to go from choosing one of three, to choosing two of three. A poorly managed, low discipline team can only choose one of good, cheap, and fast; and this is of course worthless. Cheap software that’s late will probably still run out of VC and be beaten by the competition. Bad and cheap software will struggle to take over the market, and software that’s both bad and late probably shouldn’t be written. However a well managed, well disciplined team can survey the market, measure the competition, and knowingly choose what compromises they wish to make in speed, quality, and cost. Poorly managed teams blunder into one of the choices, usually cheap and bad, and end up having very little control over their own fate.

Stop Surprising Me Ubuntu

There are almost as many design principles as there are articles about them. But my two favorites are the Principle of Least Surprise, and You Ain’t Gonna Need It.

The Principle of Least Surprise, is one of the design principles of the language Ruby, specifically in the context of “the principle of least surprise after you learn Ruby very well.” And all good systems have the property that they do not surprise experienced users, but great systems do not even surprise inexperienced users. This is what Apple has been doing for years, trying to make their products mimic familiar scenarios and objects from the real world, so that users can simply pick up the product and use it.

You Ain’t Gonna Need It is actually a programming principle, stating that you should never add features under the assumption that someone might need it someday. Chances are that users either won’t need it, won’t want it, or will end up preferring something significantly different than what you provided. But I think this principle can be simplified down to “provide the minimum set of options and features to allow users to complete the task”. Simpler products are easier to learn, easier to master, and easier to create and test. The only extra work is in designing it properly.

With this in mind, let’s turn towards Ubuntu. I’m hardly a Linux novice, having used Gentoo, Fedora, Ubuntu, Debian, Mint, openSuse, and others before eventually converting to Mac OSX. I had to upgrade my spare Ubuntu box this week from 12.04 all the way to 13.04, and found a few gripes.

First of all, there’s the Amazon search feature they added sometime in the 11.X releases. It slows the search box down, which is the main way that applications are opened, and clutters up the results. Besides being a transparent money grab, I have a hard time imagining someone trying to open up their music client and suddenly deciding that they need to buy some cheese crackers right now. It all seems like quite a bit of distraction in order to enable a few users that want to order from Amazon without having to fire up Chrome first.

But the part that really angered me was when I tried to turn this “feature” off. I couldn’t find any settings within the dock itself (right clicking does nothing, it seems), and I couldn’t find any reference to it in the system settings panel either. A quick trip to Google, which I feel shouldn’t have been necessary, told me that I could turn this search off in the Privacy settings, for some reason. As annoying as this was, all would be well if the upgrade to 12.10 didn’t turn it back on again. Why would you do that to me, Ubuntu? I clearly stated that I had no interest in this mis-feature by hunting it down and disabling it, what purpose does re-enabling it have other than grab a bit more money from Amazon and anger me?

It’s these little mis-steps and more that make or break the user experience of a product. I’m fairly tech savvy, and am comfortable digging through StackExchange and various forums to find some solutions to my problems, but that’s not the way that the average customer works. For those who aren’t comfortable looking up answers or editing configuration files, these little details will quickly scare them away to an operating system that’s less surprising and simpler. And these are the users that Canonical must attract if they’re hoping to turn Ubuntu into a mainstream Operating System.

Introducing Butler.el

I’m a big fan of reducing barriers to increase adoption and compliance. One of the largest examples for me in the past few months was when I tried out Eclipse for Python Development. The most important plugins for me (other than Python and Git) were the Jenkins and Jira plugins. Quick access to my tasks and instantaneous in-editor feedback about job status increased the likelihood that I would respond to failed builds or fill out tickets correctly.

I eventually decided that Eclipse was overkill for Python development, and way too heavy for a Macbook Air. But in my switch back to Emacs I particularly missed my Jira and Jenkins plugins. This is why I’m writing to announce the 0.1.0 release of Butler. Butler allows for you to view the jobs on your Jenkins servers, and trigger new jobs without switching out of Emacs.

At this point Butler’s feature set is modest: view, refresh, and trigger. These are just the groundwork functions for a larger feature set in the coming weeks, including console output, visual indication of running jobs, watching jobs, build queue, etc. I have high hopes that in a few months this plugin will be as mature as its eclipse counterpart.

If you have any bugs or feature requests, please add them to the Github issue tracker.

A New Addiction

Well, I have officially started collecting watches. My first decent purchase was a few months ago, but I’m counting it as a collection now because I have more pieces than is functionally required, which arguably is 1 in the cell phone era.

Why collect watches? Well, first of all they’re modern Mens jewelry. Mens fashion has changed remarkably little in the century, especially compared to Womens’. So with suits that may stay “in style” for 2-6 years depending on the cut, men only have a few places to go to express their style: shoes, watches, ties, and cuff links.

Further still, very few men under the age of 25 (my group at the moment) wear suits. I usually wear a button up or t-shirt to work, and rarely wear any pants more fancy than jeans. This leaves me shoes and watches. This of course also limits my shoe choices a bit, as does the weather in Chicago and my commuting method of choice (bike).

So, that leaves the lonely watch as my main fashion item. What I particularly love about watches is their intricacy, especially mechanical ones. What I love even more about them is how functional yet counter-functional they can be. Watches do one thing quite well: deal with the passing of time. But at the same time, they’re hardly required in the era of LED signs, car clocks, and cell phones. Furthermore, many have far more complications (features beyond basic date & time) than most people will use. How many people actually time things with their chronographs? How many divers watches rated to 200m go futher underwater than 8 feet? Not many.

At this point, I have two pieces in my collection which I will photograph and post later:

  • A quartz Fossil chronograph, with leather band.
  • An Invicta Pro Diver Automatic, in stainless steel.

I enjoy both quite a bit. The Fossil has a surprising amount of sentimental value for me already, as it was my Valentine’s day gift this year. But I’m also growing to love the Invicta, it feels more serious with its weight, and the hands glow to a degree I’d never seen before.

Next step? Time to start hitting up estate sales and thrift shops in look of a hidden treasure.

Binding vs. Assignment

A coworker of mine was recently running into problems with the following snippet of Python code, and turned to another functionally oriented developer and myself for help.

Odd Functions
1
2
3
4
5
6
7
funs = []
for i in ["a" "b"]:
    def internal_function():
        return i
    funs.append(internal_function)
print funs[0]()
print funs[1]()

At first glance, one would expect this to print “a” then “b”. But much to my surprise (and my coworkers frustration), it was returning “b” both times.

My first instinct was that Python was closing properly, but was messing up the namespacing. So I asked him to use lambdas instead, creating this:

Lambdas
1
2
3
4
5
funs = []
for i in ["a", "b"]:
    funs.append(lambda : i)
print funs[0]()
print funs[1]()

Same result! So by this point we know that its not namespacing that is causing the problems, as you can’t accidentally assign two lambdas the same name. So I began to suspect that the use of “i” was the issue. Perhaps we’re closing around i, which is a mutable variable, and Python can’t anticipate that we want a copy instead of the actual reference. Since i is mutable, both closures are having what i means changed from underneath them after closure creation time. (Coincidentally, Python 2.7.3 keeps i around even after the loop is completed, which I think is odd).

So the solution here would be to use a temporary variable. Any variable first created inside a loop should theoretically only have a lifespan of one iteration through the loop. So let’s try the following:

Temporary Variables
1
2
3
4
5
6
funs = []
for i in ["a", "b"]:
    temp = i
    funs.append(lambda : temp)
print funs[0]()
print funs[1]()

Same result, again! So clearly python is mishandling variables by a large amount. So what’s going on here? My fellow functional coworker was finally able to explain: the real problem here is that Python conflates binding and assignment. Binding is used to create a new variable within the current context, while assignment can only change the value of a given variable within the narrowest bound scope.

Why is this an issue? Well, Python uses the same symbol “=”, to specify binding and assignment, while most other languages use two syntactic rules. For example Go apparently uses “:=” to bind, and “=” to assign. Java uses type declarations to denote bindings, and Clojure/Haskell more or less only have bindings underneath. Unfortunately there’s an upper limit to how intelligent an imperative compiler can get, especially in interpreted languages where the compiler must be fairly quick. So Python doesn’t do the leg work to realize that temp is only used inside the loop, and binds it only once outside the loop. The side effect of this is that anyone who closes over any local variable inside the loop will be surprised when it is mutated by further iterations, and said variables will be left laying around after the loop is done.

So how do we get out of this? One obvious option is to pick a language that understands the difference. Short of that, use a separate function to force Python’s hand.

Ugly Fix
1
2
3
4
5
6
7
8
9
def create_lambda(x):
    return lambda : x

funs = []
for i in ["a", "b"]:
    funs.append(create_lambda(i))

funs[0]()
funs[1]()

Which unfortunately is pretty ugly. Other languages will get you out of this bind though. Clojure has no concept of assigning variables, only binding. So the only way to create mutable state in Clojure is through successive binds, or via the special Software Transactional Memory functions, which are specially designed for concurrency.

Nice and Clean
1
2
3
4
5
6
7
(def funs
     (for [i ["a" "b"]]
         (fn []
             i)))

(println ((first funs)))
(println ((second funs))

And according to my Go enthused coworker, this also works:

1
2
3
4
5
6
7
8
9
10
11
12
13
func main() {
    for i:=0; i<4; i++ {
        tmp := i

        funs[i] = func () int {
            return tmp

            }
    }
    fmt.Println(funs[0]())
    fmt.Println(funs[1]())
    fmt.Println(funs[2]())
}

If you are currently working on a Python project, or are a Python enthusiast, the only hope is to be careful, and hope that Python 3’s compiler might be more intelligent.

Size Matters

About 6 months ago I started to look for a new language for hobby projects. I wanted something practical, yet fun. I managed to limit the criteria down to:

  • Functional
  • Good community
  • Growing acceptance

This automatically eliminates a lot common languages, like Python, Ruby, and Java. After my Common Lisp job, I knew that a sane compilation system/target were an absolute must, so I looked at Scala first. On the surface Scala seemed a real winner: growing fan base, increasing commercial adoption, functional, and targeting the JVM. But while trying out Scala I realized that I had missed a huge criteria: language size.

There’s only so much information that a programmer can keep in their mind at a time. Every programmer needs to split their internal memory in at least 4 different ways, if not more. There are language details to remember, library or API interfaces to recall, domain knowledge to retain, and possibly coordination or social issues to be concerned about. Once this internal memory is exceeded details end up being forgotten, and recalling forgotten information involves an expensive round trip to documentation, code, or the internet.

There’s no escaping the need to recall language details, at least until someone invents a syntax free programming languages; similarly it’s near impossible to write an application of any size without relying on libraries or APIs. Domain knowledge is often expensive to acquire and beneficial to have. And social issues cannot be resolved by programming language choice alone.

So the real question is, what are you going to skimp on? Are you going to pay the price of memory “cache misses”? Are you going to forget portions of your language, or certain libraries? Or are you going to segregate domain knowledge among the team so that everyone knows their bit only?

The real world answer is probably a bit of each. But I contend that reduction of language features in memory is the least damaging of all the outcomes. I doubt it’s controversial to say that domain knowledge should be shared widely among developers, and should be as much in memory as possible; programmers who understand what they’re programming for tend to program better on average. Similarly it’s hard to eliminate a decently written library: that functionality has to come from somewhere, and it’ll either be an external library, an internal one, or duplicated code.

But language reduction is an interesting idea. Programmers long ago got away with previous few features, yet managed some amazing things. Don’t believe me? Consider how small the Scheme spec is compared to say, Java. It reduces the need for fancy and hard to learn IDEs, allows the programmer to get more done between looking up function or class definitions, and increases the speed with which an engineer can fully understand a piece of code.

It was actually this feature that finally drove me away from Scala to Clojure. After a few hours of Scala work I was turned off by the size of the language, and got tired of saying “Why did it do that?” Clojure on the other hand is relatively small, only a dozen “special forms” which have to be hard coded into the language, and then a 3-4 dozen core functions. This means that I can both keep the entire language in memory, and retain a large amount of library and domain knowledge to get work done.

Octopress

I’m currently in the process of changing my website over from Wordpress.com to Octopress on Github pages. There are two things driving me from Wordpress.com: control and price.

Wordpress is fine if you are either a big PHP afficianado, or are afraid of web programming in general. It’s quite easy to install themes and plugins without any knowledge of what this means. However, this only is true if you’re self-hosting Wordpress.org (the project) and not using Wordpress.com (the commercial website using Wordpress.org). On Wordpress.org you’re stuck using a limited number of (mostly for pay) themes and plugins, and are restricted from inserting HTML, JS, or CSS into your website.

So if you want Google Analytics, you’ll need to use a third party CDN who can inject the Google Analytics JS into the DOM. No thanks. If you want any social networking integration, you better hope that Wordpress.com supports it (sorry, app.net users!). And if you want pictures or other media, you’ll have to manually control the links to a third party system or hope that you don’t exceed the cheap data storage tier on wordpress.com.

So what’s so great about Octopress?

  1. Free or cheap hosting wherever.
  2. Easy dev preview.
  3. Version controlling.

Octopress uses Jekyll, which makes it really easy to separate content from presentation, and produces purely static webpages that can be tossed onto any webhost. It also has great integration for thid party systems, and adding your own is only a hop, skip, and a bit of JS away. Local development is super straightforward with the rake generate and rake preview commands, which produce static HTML and a testing webserver, respectively. And everything is stored in Git, so upgrading is only a matter of merging (!), and you can host on Github by pushing to the correct repo (or use rake deploy). And since your side is code, you don’t have to count on flimsy undo support, merely revert commits as needed to go back in history.

So far the only hiccup is moving all of the comments over from Worpress to Disqus. All of the docs on this I’ve seen are out of date, I’ll report if I get this working.

Java Doesn’t Deserve Half the Shit We Give It

It’s very popular among programmers to diss Java on Reddit and Hacker News. But Java really isn’t that bad of a language, and I really think there are other reasons why people hate it.

While we like to pretend that programming language choice is purely about practicality, there is a very large social aspect to it. Nobody wants to admit to liking Cobol or VBA, and being at the forefront of a new, exciting language can bring a type of prestige in programming circles. Thus to some degree, our choice of programming language reflects on how we wish to see ourselves. Haskell programmers think of themselves as highly skilled mathematicians, python users think of themselves as massively productive, etc.

Personally, I think the easiest way to sum up the design decisions behind Java is thusly: You have a large number of average programmers, how can you get them to design large and complicated systems reliably. This is not the most exciting idea in the world, but it’s thoroughly realistic. Your average programmer is, well, average. While this may be reason enough to not bother with Java in your hobby projects, I don’t really see this as a reason to heap scorn upon it. It may not be exciting, but Java really isn’t that bad.