Seeking a New Clojure Engagment

My long-running Clojure project at Aviso is ramping down now and I'm on the hunt for a new engagement ... preferably in Clojure.

I'm really thankful to Aviso for giving me the opportunity to spin up my real-world Clojure (and ClojureScript!) skills over the last two and a half years. I'm definitely in a position now where using Clojure feels like "cheating", compared to working in other languages I know and use. I know I have to work much harder in Java to accomplish the same goals.

I'm interested in either a long term engagement, or a full time position; in either case, it should be in Portland, Oregon, or support remote work.

I'm also very interested in providing Clojure training; I had a little preview of this recently at Uberconf, where Neal Ford and I presented an Introduction to Clojure. I'll be using any upcoming downtime to create new Clojure training materials.

I'm also open to short-term "bootstrapping" engagements, to work with a small team interested in getting into the Clojure world. This could be a mix of training, mentoring, and prototyping.

Here's a few of our accomplishments, many of which have been open-sourced:

• Created the pretty library, which improves Clojure's dreadful exception output. It has been downloaded 227K times to date, and is also incorporated into the Boot build tool.
• Created Rook -- sane, smart, fast, Clojure web services
• Created an amazing web application asset pipeline: Twixt
• Built test tooling that leverages Docker to start, stop, and maintain Docker containers (containing PostgreSQL databases).

Even with all the powerful developer features built into Tapestry, I feel even more productive in Clojure and ClojureScript. I'd like to share that with your team, in either a short-term or long-term relationship. Contact me!

Postgres, JDBC, Time Zones

I've been banging my head against the wall, trying to get things to work correctly with JDBC and PostgreSQL ... ah, time zones, every programmer's nemesis.

Does this seem familiar?

• Parse "1984-03-02" to a date, say #inst "1984-03-02T00:00:00.000-00:00"
• Insert that date into the database, as a PostgreSQL date column
• Read the value back from the database
• Print it out and get #inst "1984-03-01T08:00:00Z"
• Curse!

A little bit of digging shows that this is a pretty common problem unless both the client and the server are running in the UTC time zone. Now, it goes without saying that you are eligible for institutionalization unless you are running your servers in UTC, and that goes triple for your data store ... but the client? That really shouldn't matter.

Except it does; unless you use the full version of PreparedStatement.setTimestamp(int, Timestamp, Calendar), the PostgreSQL driver uses ... whatever the default client time zone is. Really, that's in the JDBC specification. So much for repeatable behavior!

My solution uses a dash of Joda Time (via clj-time):

	(ns setup
	(:import
	[java.util Date Calendar TimeZone]
	[java.sql PreparedStatement])
	(:require
	[clj-time.coerce :as coerce]
	[clojure.java.jdbc :as jdbc]))
	(extend-type Date
	jdbc/ISQLParameter
	(set-parameter [val ^PreparedStatement stmt ix]
	(let [cal (Calendar/getInstance (TimeZone/getTimeZone "UTC"))]
	(.setTimestamp stmt ix (coerce/to-timestamp val) cal))))

view raw setup.clj hosted with ❤ by GitHub

This hooks into the clojure.java.jdbc library and extends the logic related to how a PreparedStatement is initialized for execution. It ensures the date or timestamp is interpreted in UTC.

core.async article on Medium

I've just started writing on medium.com, here's my first article, about a clever design I came up with involving core.async.

Better Abstractions With core.async.

Subtle nicety on defrecord

I just stumbled on a rather nice feature of defrecord that I've missed in the past; perhaps you have as well?

I've been using Stuart Sierra's component library to manage startup of the server I'm building.

A big part of component is the use of records. You define a record that stores your own data and have the record extend component/Lifecycle; this is provided to the component library and it will assoc in any dependencies before invoking the start method.

	(defrecord ConnectionManager [execution-mode active-connections databases]
	component/Lifecycle
	(start [component] ...)
	(stop [component] ...))

view raw gistfile1.clj hosted with ❤ by GitHub

Now, in practice, the values of the record's fields, execution-mode, active-connection, etc., don't change (at least, the component library doesn't change them). So the easiest thing is to just reference those symbols in the body of the start or stop method.

But what about other cases, where the field values may change? Do you need to to access the values via keyword? That is, if the field values are changed and you use the symbols, do you get the current value, or the value as passed to the record's constructor?

	(defprotocol State (state [this]))
	(defrecord StateImpl [state-value]
	State
	(state [this] (:state-value this)))

view raw gistfile1.clj hosted with ❤ by GitHub

That will, of course, work ... but it's actually not necessary. From the defrecord documentation:

Note that method bodies are not closures, the local environment includes only the named fields, and those fields can be accessed directly.

Our answer is right there in this typically too-terse description. The method bodies look like closures over the initial values of the fields ... but they are not; they are translated into references to the actual fields.

Here's a simpler version of StateImpl, with a simple experiment to prove its behavior:

	(defprotocol State (state [this]))
	(defrecord StateImpl [state-value]
	State
	(state [this] state-value))
	(-> (StateImpl. 42) state)
	=> 42
	(-> (StateImpl. 42) (assoc :state-value 99) state)
	=> 99

view raw gistfile1.txt hosted with ❤ by GitHub

I haven't looked at the bytecode generated, but it appears that referencing a field in a method body is very similar to accessing a field in a normal Java object; the value retrieved is based on the this-like first parameter. In other words, it's designed to be very natural for Java developers. Even so, this behavior comes as a welcome surprise ... though I'm pretty sure any of the current books on Clojure have this information.

Google Docs Thinks You are Offline? Work around their bug!

... and so suddenly, and without warning, Google Docs stopped responding for me. My spreadsheet (my invoice spreadsheet without which I can't get paid) would load and then immediately tell me "You are offline" (there's an icon for that).

After trying all the obvious things ...

• Clear my cache
• Try it in FireFox instead of Chrome
• Restart my computer
• Try a different Internet connection

... I started hunting around the web. The problem has been around since at least 2012. This guy had a way to fix it that didn't work for me.

However, it did teach me about the Chrome's built in DNS information page where I could see 0.docs.google.com ERR_NAME_NOT_RESOLVED.

First I tried turning the Internal DNS client off (not on) ... that didn't help.

Then I did a hack. First I pinged docs.google.com and got 173.194.33.32 (due to the vagaries of DNS, you might get a different number).

Then I added a line to my /etc/hosts file:

173.194.33.32 0.docs.google.com

... which basically says: "don't use DNS, I've got your IP address right here".

Things seem to be working!

My guess is that this represents some kind of load balancing solution implemented by Google Docs ... and either the 0 server has failed, or its configuration got munged, or this is simply a bug in the Google Docs client-side software.

Gradle: overruling third-party dependencies

We all hate dependency hell, and on any Java project of any size, you'll hit it eventually. My project, for Aviso, is a large Clojure code base, with tons of dependencies on third-party libraries, many of which are Java libraries with their own third-party dependencies. Despite using Gradle as our build tool, we don't get a free pass, we sometimes end up with conflicts.

It often plays out like this: module A as a dependency on library L, which has a transitive dependency on library Q. That's OK, module A has a consistent class path when it builds.

Meanwhile, module B has a dependency on library M, which has a transitive dependency on library Q ... but a different version. That's OK, module B also has a consistent class path when it builds.

However, inside IntelliJ, you see both version of library Q in the "External Libraries" folder of the Project explorer. That's unfortunate and can cause confusion when navigating your code.

Worse yet, in the final application, combining modules A and B, you will be executing one module with a different version of library Q than your tests. That alone makes me a touch nervous.

Fortunately, Gradle provides a quite reasonable way of dealing with this. The hard way would be to just turn off all transitive dependencies. But I consider that throwing out the baby with the bathwater.

Instead, we can selectively override transitive dependency, consistently across all modules. And we can do this in a single place, in our top-level build.gradle:

	def versionOverrides = [
	"asm:asm": "3.3.1",
	"bultitude:bultitude": "0.1.7",
	"commons-codec:commons-codec": "1.7",
	"commons-io:commons-io": "2.4",
	"io.aviso:pretty": "0.1.9-SNAPSHOT",
	"joda-time:joda-time": "2.1",
	"org.clojure:core.incubator": "0.1.1",
	"org.clojure:tools.macro": "0.1.1",
	"org.clojure:tools.namespace": "0.1.1",
	"org.codehaus.groovy:groovy-all": "1.8.6",
	"org.jsoup:jsoup": "1.7.1",
	"org.yaml:snakeyaml": "1.12",
	"slingshot:slingshot": "0.10.3"
	]
	subprojects {
	configurations.all {
	resolutionStrategy.eachDependency { DependencyResolveDetails details ->
	def overrideVersion = versionOverrides[details.requested.group + ":" + details.requested.name]
	if (overrideVersion != null && details.requested.version != overrideVersion) {
	logger.info "Overriding dependency ${details.requested.group}:${details.requested.name} version ${details.requested.version} --> $overrideVersion"
	details.useVersion overrideVersion
	}
	}
	}
	}

view raw build.gradle hosted with ❤ by GitHub

This one small change affects every child project; we have a single place to maintain and resolve these version conflicts and don't have to chase down which module (among the 37 currently in our overall project) is the culprit for introducing a conflict. When we see a conflict, we add a new mapping to versionOverrides and we are done.

This is a huge example of how powerful Gradle's Groovy DSL is; because the build script is also executable code, there's room to put logic in place that simply can't be defined declaratively.

Our change hooks into the dependency resolution logic associated with each Gradle configuration (a configuration is essentially a way of declaring the class path for compiling, testing, or executing Java code).

Gradle kindly exposes a step inside the overall process of analyzing the dependencies; this code hooks into this step. It sees the requested dependency, and if it's in the override map, forces the version number to a specific value. In fact, this mechanism is powerful enough to replace dependencies, but that's beyond our immediate needs.

This is one of the reasons I use Gradle in preference to Maven: Gradle has the tools to cleanly and easily address my specific problems and particular edge-cases.

Tapestry Quicky: ConditionalComment

Here's a quicky I just put together for a client to generate IE conditional comments. This isn't a feature supported by Tapestry's JavaScriptSupport (for libraries; it is support for CSS files).

Fortunately, this is something that comes together in Tapestry in almost no code:

	package com.fivoosh.components;
	import org.apache.tapestry5.BindingConstants;
	import org.apache.tapestry5.MarkupWriter;
	import org.apache.tapestry5.annotations.Parameter;
	/**
	* A component that wraps its body in an IE conditional comment.
	*/
	public class ConditionalComment {
	@Parameter(required = true, allowNull = false, defaultPrefix = BindingConstants.LITERAL)
	String condition;
	void beginRender(MarkupWriter writer) {
	writer.writeRaw(String.format("<!--[if %s]>", condition));
	}
	void afterRender(MarkupWriter writer) {
	writer.writeRaw("<![endif]-->");
	}
	}

view raw ConditionalComment.java hosted with ❤ by GitHub

This is what it looks like in a template:

	<t:conditionalcomment condition="lte IE 9">
	<t:remove>
	Fix IE8+9 lack of support for html5 elements and media queries
	</t:remove>
	<script src="${context:js/html5shiv.js}" type="text/javascript"/>
	<script src="${context:js/respond.js}" type="text/javascript"/>
	<script src="${context:js/media.match.js}" type="text/javascript"/>
	</t:conditionalcomment>

view raw Layout.tml hosted with ❤ by GitHub

And here's the markup it produces:

	<!--[if lte IE 9]>
	<script type="text/javascript" src="/assets/ctx/z79f6b0bd/js/html5shiv.js"></script>
	<script type="text/javascript" src="/assets/ctx/zf8f53e99/js/respond.js"></script>
	<script type="text/javascript" src="/assets/ctx/z8aa0bee8/js/media.match.js"></script>
	<![endif]-->

view raw Page HTML hosted with ❤ by GitHub

And that's the whole point ... the URLs for the referenced JavaScript libraries are now Tapestry assets, and will have access to Tapestry's asset pipeline as unique resources, including GZip compression and minimization.

It is always important to me that Tapestry not get in the way: frameworks that lock you down and prevent you from accomplishing your goals should not be used. Tapestry has a long history, and there are certainly many areas that could have been implemented differently, or simply not implemented at all, but it's nice that most edge cases, like this one, have a simple solution.