Posts tagged programming

Literate Jenks natural breaks and how the idea of code is lost f

On writing a fresh implementation of an algorithm when the original description is locked by copyright and unavailable. Most available implementations are translations of an old Fortran implementation, and they lose something as they go.

6programming, copyright,

Clojurescript: up and running f

My current reading.


Greg Wilson on “What we actually know about software development and why we believe it’s true.” via programming is terrible

6programming, craft,

Talk: Advancing Distributed Systems, by Eric Brewer f


He begins with a useful history lesson, putting the development of distributed systems into perspective from two distinct views—databases and operating systems—showing NoSQL and SQL have a longer history than most imagine. He also offers some promising directions in which they can learn from each other.

A fascinating lecture on computer science, built upon his experience teaching a combined internals course on operating system & relational databases. I was lucky enough to catch Eric’s keynote live, streamed from the RICON conference (thanks Basho et al). (And the Slides are online too)

Source: programmingisterrible


keep-alive-agent f

This destroyed sockets bug kept coworker Danny & I occupied through the start of the week. This simple keep-alive agent is our workaround for it. If you happen to need a socket-reusing agent for node.js, it might prove useful to you.

6node.js, programming,

Developer diary: Error handling

As you know, I hate boilerplate. It’s what enrages me about Java. Anything that a compiler/language/whatever can predict and generate automatically should be predicted, out of the way of the programmer. (This is why Java IDEs enrage me even more.) The node.js world has some particularly voluminous boilerplate in the error handling that typically opens every async callback. Javascript doesn’t provide any language features to help with this and the community is resistant to standardizing on any third-party modules.

My Sooper Dooper Sekrit Projekt has a lot of it and it’s starting to annoy me. I am going to adopt a new pattern to help.

Here’s an example of the problem. Look at the blocks of error handling in putTogetherSomeData():

Now imagine many functions with the same pattern repeated over and over: every call to couch.get() must be followed by nearly identical error handling code. This boilerplate is a problem for several reasons. It’s a surface for bugs; the error handling is required for application safety, but nothing demands its presence. It clutters the source and makes it harder to read and see what’s going on. As with all boilerplate, spotting minor differences in the error handling from case to case can be difficult. It adds to programmer cognitive load.

Here we’ve introduced some improvements.

We start by wrapping the db get functions. The wrapper functions take three parameters: the input originally required by the getters, a success callback, and a failure/error callback. The wrapper functions include all their own error handling. They look at err and status codes and call their error handlers instead of their success callbacks when appropriate.

This allows putTogetherSomeData() to flow better. It first defines the error handler function that it uses to clean up after one of its operations fails. Then it calls the wrapper functions. The success functions don’t bother with any predictable error handling at all, because they simply aren’t called in conditions that count as pure errors.

The error handler is similar to an exception handler, in that you can define it at the level where it makes most sense to handle the error. Lower levels can just toss it on up. But unlike exceptions and err params, you have to define that function somewhere; you can’t just forget to write the error handling.

Weaknesses: One hand-written wrapper per call. Doesn’t help functions that throw exceptions.

Advantages: If you’re like me, you’ve already wrapped many of your third-party db driver calls to do things like log.

Here we’ve auto-wrapped predictable functions.

Weaknesses: Not every function will be auto-wrappable, because the application might have different needs on different invocations of a third-party call. Not enough data is making it back to the error handler. That callback extraction code is iself boilerplate that needs to be refactored out. I like the assertions, though!

Other approaches to the problem

Promises. See q, a popular implementation of promises.

6node.js, programming, project, developer diary, medium,


[text: “who the fuck indents with spaces, you should be jettisoned into space”, photograph of the deep reaches of space, where no man has gone before]

[HD Version]

Source: theprofoundprogrammer



[text: “I don’t know how to explain source control without waving the gun”, photograph of a feathery waterfall pouring between two rocky outcroppings beneath a worn stone bridge, photo credit Richard Gaywood]

[HD Version]

I love these too much.

Source: theprofoundprogrammer


Developer diary recommences

Earlier this year I blogged a lot about a web app project I was working on that used mostly Ruby stack: Padrino (sinatra + stuff) on top of Riak + Redis for persistence + ElasticSearch for search + Davis.js + my own Backbone-ish model with ajax persistence. I reached a point where I had just implemented some markdown extensions in my javascript preview library & realized that I’d have to re-implement them in Ruby. At that moment I stopped and pondered the virtues of the single-language stack.

And then I got a job in San Francisco. That plus commuting has been hell on personal projects. I resurrected it recently, though, because I discovered a massive itch that needed scratching that was similar to (but not identical to!) the previous project. Thus Project Two, Electric Boogaloo was born.

The stack isn’t quite the same:

javascript + node.js
couchdb for document persistence via cradle (this project is much more document-centric than the other one; this could easily be swapped out for riak if necessary)
redis for fast indexes (am using couch views as little as possible, since all the operational stories discuss their suckage)
redback for some higher-level data structures built on redis
my own model schema + persistence layer (actually separate!)
express.js for serving web pages
blade for templating
ElasticSearch for full-text search via my own very thin wrapper
and a few smaller packages, like my fork of marked for github-flavored markdown, prettydate so javascript can have strftime() like a civilized language, and so on.

I also finally broke down and started using the async flow control library. If there’s anything I hate, it’s boilerplate. And node.js callback error-handling is boilerplate at its repetitive code-cluttering worst.

I haven’t done anything on the client-side that needs a single-page app just yet. But if I do, I’ll use davis.js again, again with my own non-backbone model library. Backbone made me write reams of boilerplate. You know what I think about that.

Javascript is such a dangerous and loose language that I’ve been trying to maintain some strict discipline with myself about the code. Not just formatting (which I’ve always had strong opinions about) and not just running linters. I’m trying to stay modular with small files and small pieces that work together and aren’t interdependent. Any single piece should be something I could swap out.

This week I’ve been wiring up ElasticSearch. My next programming session should have documents being indexed on their way into the system. The one after that will start implementing a search API that is intended to be consumed by two kinds of web pages (a general “search stuff” text box that tries to guess what you mean, and a tweaky search form, the kind with a zillion checkboxes) plus the externally-focused json api I’m planning. I’ve been trying to write design documents for myself as I go on this project. Not just for myself! My plan is to open it up as soon as I have a site running with alpha-level features.

As always, it was raving frustration with a really-badly-written and -performing existing site that motivated me to start this one. Niche audience, though. No idea how I’m going to make it self-sustaining given the user base. Yes, I haven’t told you what it is yet.

6javascript, programming, project, medium,


Server’s down, the HD remake.

Source: theprofoundprogrammer


Mother Jones on the brogramming thing f

Sounds like Path has a real prize of an exec in Van Horn. Seriously, why is that guy allowed to speak in public representing them? He’s a serial offender.

The writer unfortunately gets the Siri search for Planned Parenthood story 100% wrong, as did the original complainer. (The vast majority of Siri searches are algorithmic. It can only find Planned Parenthood under terms that PP chose to advertise itself with. It didn’t happen to choose those terms.)

The term I like to use for myself is “software engineer”. I think “programmer” is also a pretty awesome word. I am also completely okay with “geek” and with “roomful of geeks”. Those would be my people.

6culture, programming,

Programmers Being Dicks f

Now following this, which also includes examples of programmers being dicks and then fixing it, with excellent pull request comments like this one.

Though I did like the “I expected you to just CNAME this to Hacker News” comment.


jquery -> ender; some notes.

Yesterday I migrated my project from jquery to Ender. Ender isn’t a library itself, but instead a tool for building javascript libraries from components. It offers something close to the jquery feature set at a much smaller size. If you don’t need everything that jquery does, or if you don’t need the massive backwards compatibility with broken browsers it gives you, ender might be a win.

I had the problem of making my client-side router (davis.js), my client-side templating, and some jquery plugins work.

First, davis.js. The minimal ender build for this needs events, selectors, a dom manipulator, and a dom ready check, aka their “jeesh” package:

ender build bonzo bean qwery domready

I encountered two minor problems. First, davis.js explicitly looks for jquery, not for anything pretending to be jquery in the infamous $. So before you construct your app, tell Davis to use the Ender $:

Davis.$ = $;
var myapp = new Davis.App();
// etc

Second, Bean, the ender events library, doesn’t appear to stop the propagation of delegated events the way jquery does. So Davis needs to do so explicitly in its delegates:

var handler = function (targetExtractor) {
    return function (event) {
      if (differentOrigin(this)) return true
      var request = new Davis.Request ($(this)));
      return false;

Davis should now work.

That was the easy part of the migration. Far more annoying was migrating from jquery’s ajax api to reqwest's.

Next, jquery plugins. Rewrite them. They were not invented by you, so they’re probably awful. Okay, that’s a little unrealistic for some things. If you’re using Bootstrap and its plugins, the ender rewrite will be handy. Note that I couldn’t make its meta-package work. I had to build a library with individual plugins one at a time using ender add.

The other thing I did was rewrite ICanHaz to use the faster mote.js instead of the default mustache javascript implementation, as well as to be agnostic about jquery, zepto, and ender. This isn’t enough code to deserve a github repo, so have it in a gist.

6javascript, programming, project,