Tapestry Training -- From The Source

Let me help you get your team up to speed in Tapestry ... fast. Visit howardlewisship.com for details on training, mentoring and support!

Monday, September 29, 2014

Postgres, JDBC, Time Zones

I've been banging my head against the wall, trying to get things to work correctly with JDBC and PostgreSQL ... ah, time zones, every programmer's nemesis.

Does this seem familiar?

  • Parse "1984-03-02" to a date, say #inst "1984-03-02T00:00:00.000-00:00"
  • Insert that date into the database, as a PostgreSQL date column
  • Read the value back from the database
  • Print it out and get #inst "1984-03-01T08:00:00Z"
  • Curse!

A little bit of digging shows that this is a pretty common problem unless both the client and the server are running in the UTC time zone. Now, it goes without saying that you are eligible for institutionalization unless you are running your servers in UTC, and that goes triple for your data store ... but the client? That really shouldn't matter.

Except it does; unless you use the full version of PreparedStatement.setTimestamp(int, Timestamp, Calendar), the PostgreSQL driver uses ... whatever the default client time zone is. Really, that's in the JDBC specification. So much for repeatable behavior!

My solution uses a dash of Joda Time (via clj-time):

This hooks into the clojure.java.jdbc library and extends the logic related to how a PreparedStatement is initialized for execution. It ensures the date or timestamp is interpreted in UTC.

Thursday, July 24, 2014

core.async article on Medium

I've just started writing on medium.com, here's my first article, about a clever design I came up with involving core.async.

Better Abstractions With core.async.

Wednesday, June 04, 2014

Subtle nicety on defrecord

I just stumbled on a rather nice feature of defrecord that I've missed in the past; perhaps you have as well?

I've been using Stuart Sierra's component library to manage startup of the server I'm building.

A big part of component is the use of records. You define a record that stores your own data and have the record extend component/Lifecycle; this is provided to the component library and it will assoc in any dependencies before invoking the start method.

Now, in practice, the values of the record's fields, execution-mode, active-connection, etc., don't change (at least, the component library doesn't change them). So the easiest thing is to just reference those symbols in the body of the start or stop method.

But what about other cases, where the field values may change? Do you need to to access the values via keyword? That is, if the field values are changed and you use the symbols, do you get the current value, or the value as passed to the record's constructor?

That will, of course, work ... but it's actually not necessary. From the defrecord documentation:

Note that method bodies are not closures, the local environment includes only the named fields, and those fields can be accessed directly.

Our answer is right there in this typically too-terse description. The method bodies look like closures over the initial values of the fields ... but they are not; they are translated into references to the actual fields.

Here's a simpler version of StateImpl, with a simple experiment to prove its behavior:

I haven't looked at the bytecode generated, but it appears that referencing a field in a method body is very similar to accessing a field in a normal Java object; the value retrieved is based on the this-like first parameter. In other words, it's designed to be very natural for Java developers. Even so, this behavior comes as a welcome surprise ... though I'm pretty sure any of the current books on Clojure have this information.

Friday, March 28, 2014

Google Docs Thinks You Offiline? Work around their bug!

... and so suddenly, and without warning, Google Docs stopped responding for me. My spreadsheet (my invoice spreadsheet without which I can't get paid) would load and then immediately tell me "You are offline" (there's an icon for that).

After trying all the obvious things ...

  • Clear my cache
  • Try it in FireFox instead of Chrome
  • Restart my computer
  • Try a different Internet connection

... I started hunting around the web. The problem has been around since at least 2012. This guy had a way to fix it that didn't work for me.

However, it did teach me about the Chrome's built in DNS information page where I could see 0.docs.google.com ERR_NAME_NOT_RESOLVED.

First I tried turning the Internal DNS client off (not on) ... that didn't help.

Then I did a hack. First I pinged docs.google.com and got (due to the vagaries of DNS, you might get a different number).

Then I added a line to my /etc/hosts file: 0.docs.google.com

... which basically says: "don't use DNS, I've got your IP address right here".

Things seem to be working!

My guess is that this represents some kind of load balancing solution implemented by Google Docs ... and either the 0 server has failed, or its configuration got munged, or this is simply a bug in the Google Docs client-side software.

Thursday, February 20, 2014

Gradle: overruling third-party dependencies

We all hate dependency hell, and on any Java project of any size, you'll hit it eventually. My project, for Aviso, is a large Clojure code base, with tons of dependencies on third-party libraries, many of which are Java libraries with their own third-party dependencies. Despite using Gradle as our build tool, we don't get a free pass, we sometimes end up with conflicts.

It often plays out like this: module A as a dependency on library L, which has a transitive dependency on library Q. That's OK, module A has a consistent class path when it builds.

Meanwhile, module B has a dependency on library M, which has a transitive dependency on library Q ... but a different version. That's OK, module B also has a consistent class path when it builds.

However, inside IntelliJ, you see both version of library Q in the "External Libraries" folder of the Project explorer. That's unfortunate and can cause confusion when navigating your code.

Worse yet, in the final application, combining modules A and B, you will be executing one module with a different version of library Q than your tests. That alone makes me a touch nervous.

Fortunately, Gradle provides a quite reasonable way of dealing with this. The hard way would be to just turn off all transitive dependencies. But I consider that throwing out the baby with the bathwater.

Instead, we can selectively override transitive dependency, consistently across all modules. And we can do this in a single place, in our top-level build.gradle:

This one small change affects every child project; we have a single place to maintain and resolve these version conflicts and don't have to chase down which module (among the 37 currently in our overall project) is the culprit for introducing a conflict. When we see a conflict, we add a new mapping to versionOverrides and we are done.

This is a huge example of how powerful Gradle's Groovy DSL is; because the build script is also executable code, there's room to put logic in place that simply can't be defined declaratively.

Our change hooks into the dependency resolution logic associated with each Gradle configuration (a configuration is essentially a way of declaring the class path for compiling, testing, or executing Java code).

Gradle kindly exposes a step inside the overall process of analyzing the dependencies; this code hooks into this step. It sees the requested dependency, and if it's in the override map, forces the version number to a specific value. In fact, this mechanism is powerful enough to replace dependencies, but that's beyond our immediate needs.

This is one of the reasons I use Gradle in preference to Maven: Gradle has the tools to cleanly and easily address my specific problems and particular edge-cases.

Friday, December 06, 2013

Tapestry Quicky: ConditionalComment

Here's a quicky I just put together for a client to generate IE conditional comments. This isn't a feature supported by Tapestry's JavaScriptSupport (for libraries; it is support for CSS files).

Fortunately, this is something that comes together in Tapestry in almost no code:

This is what it looks like in a template:

And here's the markup it produces:

And that's the whole point ... the URLs for the referenced JavaScript libraries are now Tapestry assets, and will have access to Tapestry's asset pipeline as unique resources, including GZip compression and minimization.

It is always important to me that Tapestry not get in the way: frameworks that lock you down and prevent you from accomplishing your goals should not be used. Tapestry has a long history, and there are certainly many areas that could have been implemented differently, or simply not implemented at all, but it's nice that most edge cases, like this one, have a simple solution.

Monday, November 11, 2013

Improving Clojure Feedback : Stack Traces

Perhaps no subject in software development is as important to me as feedback, which to my mind, is the language, library, or framework providing information back to the developer when things go wrong. A language, library, or framework that provides poor feedback is creating barriers to its own adoption.

There's a possibly apocryphal legend about early FORTRAN compilers. Back in the day (thankfully, even earlier than my personal back in the day), you would code up your FORTRAN code on punch cards and deliver them to an operator. At some convenient time (for the operator), the compiler would be started and fed the stack of cards. Assuming the cards had not been bent, folded, spindled, or mutilated ... and that the program itself contained no errors, you would get a compiled program somewhere. If anything failed, you would simply get the error message "FAIL". Nothing else.

That's bad feedback.

It's important to feel that you are in control of your own tools; that's part of the love/hate relationship so many developers have with their IDEs: the IDEs certainly enable a lot of productivity ... but what you always end up remembering is when the IDE gets in the way of some really critical, really difficult stuff. Or when it crashes mysteriously at a critical moment. Or when it just won't dosomething and won't tell you why not.

Likewise, any tool (or language, library, or framework) that provides bad feedback, especially when things go wrong, is building a perception that it is hard to use overall. We're a fickle bunch ... we'll be off to find the Next Big Thing instead.

Which brings us to my favorite programming language, Clojure. When things are going right in Clojure they are going very, veryright. However, when things go wrong in Clojure it can be very painful.

First of all, Clojure's basic mechanism for any kind of error is to simply throw an exception. Since Clojure is a Lisp, the difference between compile time and runtime is not so easily distinguished ... a running Clojure program is often loading source and compiling code as well as running the application.

And what happens to exceptions? They are thrown in one piece of code and, more often than not, caught, wrapped, and re-thrown in containing stack frames. Ultimately, this whole stack of exceptions gets spewed out to the console.

This isn't quite bad as that FORTRAN compiler; however, even on my big 30" cinema display, a Clojure compilation failure always sends me scrolling backwards in my terminal window to see the actual error and (with more than a pinch of luck) the file name and line number containing the error.

It's much worse when it comes to runtime failures in my own code; it is likely that the exceptions will be nested even deeper than the number of levels intituted by the Clojure compiler and REPL. Beyond that is the issue of laziness ... program execution goes a little topsy-turvey from your code due to laziness, as what code is executing is typically driven by what particular bit of lazily-computed data is needed at any particular instant. That takes a little getting used to for new Clojure developers.

And so, deep in your code, you passed a string when a keyword was expected, or you passed a map when a seq was expected, or any number of other similar situations ... and now you are faced with an exception and its stack trace.

Here's the ugly not-so-secret of Clojure: it's a light and tasty frosting on top of a cake of sawdust and rusty gears: Java. Every Clojure function must be converted into a Java class in order to execute within the JVM. Java places somewhat arbitrary limits on the class and method names: Where Clojure loves dashes in function names, Java doesn't allow them. Likewise, Clojure loves all kind of other punctuation, with functions named "+" or "string?", that Java forbids as well.

Clojure manglesthe Clojure namespace and function names to fit into the Java world, and nothing in the exception reporting chain does anything to make the result any prettier. Here's an example:

All the information is available ... but you have to work to get at it: You need to mentally de-mangle the Java class names back to Clojure namespaces and function names. You have to ignore a lot of stack frames that are really the infrastructure supporting Clojure on top of Java. You have to assemble the meaningful stack trace (the last and deepest one) from the parts it extends off of the prior stack traces. You have to ignore the numeric ids tacked on to the end of names to help ensure uniqueness.

This is not great feedback.

What if we could provide just a little bit better feedback? What if we could do a lot of that name-demangling automatically, and present the same data in a more helpful way? That's what I've been working on as part of the io.aviso:pretty library for Clojure.

Pretty can be used to format that same exception just a little bit nicer:

The exact format is still in-flux because it is currently so wide; fortunate, the interesting stuff is always to the left!

This same approach appears in the related io.aviso:twixt library, modified to take advantage of how much more richly data can be presented in a web page:

However, this is just the start. Providing truly useful feedback is more than just putting a patch on exception reporting. It requires attention to detail throughout the tool. This has been the case for Apache Tapestry, where a significant amount of the framework is about detecting and reporting errors in a useful way.

Twixt has a set of utilities to address this as well; a way of tracking what the application is doing so that exceptions can be reported. The end result is an "operations trace" that can nest to an arbitrary depth and is used to reconstruct how the application arrived at the point of failure.

This extra layer of code can pay big dividends when things go wrong; the nesting of operations can be critical to understanding the underlying problem; the operations trace combines the very local scope of the failure with a larger scope that explains why the failed code was invoked in the first place.

So I'm very excited by the possibilities, but to provide truly universal, useful feedback in Clojure may not be something that can be effectively added in from the outside: it has to be a priority for the core Clojure developers, and bred into the fabric of the Clojure compiler and runtime. That's something I'd love to see ... or help with.