The following two code snippets are equivalent, differing only on what the client code ends up looking like, but yielding in all other ways similar results:

The code above can be refactored to the pimp-my-lib style which follows, and vice versa:

In both cases, the imported LibraryCode.scala provides a callable method which requires state in the form of a Thing object. I’m assuming the Thing type is also defined external to the client code — probably in a third-party library.

The first pattern, which to the best of my knowledge doesn’t have a name, you have a method in the local scope which accepts an implicit parameter, and effectively allows you to pass a single in-scope object to multiple methods without needing to repeat yourself. I’m going to dub this the “quiet-my-scope pattern“, because it reduces  the visual chatter that would otherwise be caused by passing the same object to multiple method calls — and I’ll just hope that name sticks.

The second is the famous pimp-my-lib pattern, which is well-known throughout Scalaland to be used to “add” methods to externally-defined classes.

Using implicits we have these two API patterns to choose from — they are equivalent, in so far as an implementation of either one can be mechanically transformed in to the other, only style and convenience being a differentiator between them. You might find it useful to keep around this alternative to the pimp-my-lib pattern, as in some APIs it will yield more readable, more convenient code.

Try it out: the next time you find yourself augmenting an existing class with methods using the pimp-my-lib pattern, take 5 minutes to refactor your solution to methods with implicit parameters (using the sample Thing code above as a guide). You won’t lose much time in the exercise, and you may find you’ll end up with better code.

Similarly, the next time you find yourself defining a method with an implicit parameter, take 5 minutes to work through what a refactored solution with a pimp-my-lib style method would look like. You may like the result quite a bit more than what you started with.

I’ve recently learned how to use two complimentary static techniques for controlling how many times methods are called in an API: Phantom Types, and Scala type constraints.

Why would you want to control the number of times a method is called? Consider, for example, the common Builder Object pattern. I don’t mean the classic GoF Builder Pattern — I use builders all the time, and I don’t at all recognize this class diagram describing what a Builder is supposed to be. By Builder I mean an object with a fluid API whose job is to collect up state for the purpose of creating one or more other objects, which is pretty common in Java and many Scala DSLs.

Rafael Ferriera wrote a piece about builders and Phantom Types I’m going to use as a starting point for introduction of the topic. I’ll start with his introduction of a ScotchBuilder: a Scala object that knows how to take a proper gentleman’s order for a scotch.

Let me quote Raphael to introduce the domain:

So, let’s say you want to order a shot of scotch. You’ll need to ask for a few things: the brand of the whiskey, how it should be prepared (neat, on the rocks or with water) and if you want it doubled. Unless, of course, you are a pretentious snob, in that case you’ll probably also ask for a specific kind of glass, brand and temperature of the water and who knows what else. Limiting the snobbery to the kind of glass, here is one way to represent the order in scala.

Code block

sealed abstract class Preparation  /* This is one way of coding enum-like things in scala */
case object Neat extends Preparation
case object OnTheRocks extends Preparation
case object WithWater extends Preparation
 
sealed abstract class Glass
case object Short extends Glass
case object Tall extends Glass
case object Tulip extends Glass
 
case class OrderOfScotch(val brand:String, val mode:Preparation, val isDouble:Boolean, val glass:Option[Glass])

You can imagine providing a ScotchBuilder class to generate immutable OrderOfScotch objects with a fluid API. Below is a first-pass at such a ScotchBuilder, which is your typical fluid implementation. Its nice, but we can do better. (Code, again, taken originally from Raphael’s post, modulo changing from stateful to stateless and fixing a couple of his typos)

There are two unattractive features with this Builder  that we are going to clean up:

1. a client can re-invoke the same setter methods over and over
2. the client can also completely forget to call other methods that should be called

Check this out, where I am able to make a non-sense scotch order:

Notice I was able to specify the brand twice, and the size as both “double” and “single”. Quite ambiguous what I meant there. Considering the price difference between a single Cragganmore (cheap) and a double Macallan 25 (expensive!), that’s maybe an ambiguity we’d like to stamp out of the system.

I’m going to now show you how to use both Phantom Types and type constraints to ensure at compile time certain Builder methods are invoked:

• with at-most-once semantics. E.g., withGlass should be called zero or one time by client code for a single ScotchBuilder instance.
• with exactly-once semantics. E.g., withBrand, withMode, and isDouble each need to be called exactly once.
• and, by the way, you can use this same technique to define one-or-more-times semantics. (Though I’m not going to go that far in this article. If you grok at-most-once and exactly-once, you will be able to figure out one-or-more-times.)

These techniques are not limited to just fluid Builder APIs. Pretty much any API where you wanted to constraint the call semantics can employ Phantom Types and type constraints to achieve the desired call semantics. This is going to apply to objects that traditionally walk through a lifecycle. For example, any API where there are init() and destroy(). The Builder under consideration also has a lifecycle: several configuration methods must be called, and then finally the build method gets invoked.

Continuing the Builder example, first I’m going to stop the build method from working if there are any builder values that haven’t been set correctly. That is, using Phantom Types and type constraints, I’m going to set things up so that compiler won’t even compile code that attempts to build a scotch when all the builder parameters have not been set. After that I’m going to stop the individual withXYZ methods from being called more than once using the same technique.

Here’s how I’m going to do make the build method uncallable except when the Builder has been fully configured: I’m going to add to the ScotchBuilder class one type parameter per method whose calls we want to track. The type parameters are going to track whether each of the withXYZ() methods have been called or not; the classes Zero and Once are defined to represent these two states. I’m then going to constrain the build method to only be callable if the appropriate type parameters are bound to the Once type.

So I’m adding 4 type parameters, each able to be bound to the type Zero or Once. So instead of having 1 ScotchBuilder class, I actually am defining 16. That is, 16 different permutations of the possible bindings to the 4 type parameters. The build method will then be constraining to be callable on ScotchBuilder[Once, Once, Once, _] (one of 2 specific bindings).

This first chunk of code adds the Zero and Once types to track the number of times the individual withXYZ() methods are called below. Note that the case class copy method allows you to specify type parameters, which I’m using in the implementation of each withXYZ method:

And below is the build method, with type constraints guaranteeing the build method can only be called on instances of type ScotchBuilder[Once, Once, Once, _].

I use the =:= type class to guarantee this constraint on the ScotchBuilder type parameters. An implicit value of =:=[A, B] only exists when A == B. (For a deeper explanation of the =:= type constraining object, check out this blog post by Debasish Ghosh). I’ve further created a type alias IsOnce[T] = =:=[T, Once], which allows me to apply =:= as a type class.

The upshot of all of this is that any attempt to invoke build on a ScotchBuilder not matching ScotchBuilder[Once, Once, Once, _] simply cannot be compiled. You literally cannot compile code that improperly uses a ScotchBuilder to build a order of scotch!

Note that in the code above we never actually create an instance of Zero or Once — these type parameter bindings are purely for compile-time bookkeeping. Hence the term Phantom Types, because these types are never instantiated nor participate at runtime.

We can even do a little better with the builder above by constraining the withXYZ methods to have exactly-once or at-most-once call semantics, as appropriate. This is going to make the compiler fail at the point where the API is being misused — i.e., where a withXYZ method is being used the second time for a single ScotchBuilkder instance. So it’ll be a lot easier when using this API to figure out what you did wrong. Here is the final version of ScotchBuilder:

All this extra type bookkeeping is well worth it for me because it means I don’t have to write a bunch more test code. I never need to worry or test for cases where client is is misusing my ScotchBuilder: it simply is not possible to do. And there’s never a need to write regression tests against a case which is impossible.

Great ACM Queue article comparing SQL and NoSql systems. Interesting conclusions:

• While SQL and NoSql models are very different, there are layers for querying functionally that make them more-or-less equivalent. That is, the key/value model of NoSql systems is a functional dual of Codd’s relational model.
• Introduction of the term coSQL to refer to the key/value-based functional dual of SQL described as the model backing for “the most common noSQL databases” (Hadoop, Cassandra, Riak, Neo4J, as well as others mentioned explicitly as common NoSql systems)
• Putting forth a “data-model formalization model” coupled with the idea of a monadicly-based query language as central themes that will develop within the coSQL world, and will drive the industry to a more economically efficient configuration.

Who knows if the predictions will come true, but the coSQL model is a great tool for comparing SQL and NoSql technologies on the axes of theoretical functionality and capability. I should have mentioned first the authors: Erik Meijer and Gavin Biermann. These guys are math-oriented to the extreme. We can trust in their development of coSQL as a dual of SQL, and in their conclusions about the theoretical comparisons of coSQL and SQL systems. Practical comparisons are a completely different matter, of course.

Just reviewed a gem from Programming Scala. Of all the Fibonacci implementations I’ve seen, my new favorite is below. Its 1 statement long, and there’s not a recursive function in sight:

If you have not seen that zip trick before, follow me on a little explanation. The code defines a Stream — a non-strict iterable — that begins with two literal values “0″ and “1″. The stream then continues the Fibonacci sequence by zipping the sequence to itself. More specifically, to its own tail sequence.

This figure illustrates what the zipper is creating.

The tail value of the initial sequence is just the sequence starting with the literal value “1″. The zipper creates Pairs out of the each member of the sequence pairwise joined with the next member of the sequence.

The first pair is (0, 1).

The second pair is then (1, 0 + 1) = (1, 1).

The third pair is then (1, 1 + 1) = (1, 2).

The forth pair is then (2, 1 + 2) = (2, 3). And so on.

The coolest part is of course the complete lack of apparent recursion. The whole sequence is lazily evaluated, so the Stream takes up little space — one new Stream object per element in the sequence.

Generalizing

We can generalize this stream-zipping technique. When the value of a Stream element n can be calculated from the previous k sequence members, we can use a k-ary version of this technique. That is, if the stream theStream member n can be defined by some function s:

def theSeq(n) = s(theSeq(n-1), theSeq(n-2), …, theSeq(n-k))

We can define the stream in a single statement thus:

• Explicitly define the first k-1 stream members
• For all other members, perform k-1 zips to create a TupleK of the previous k sequence elements.
• A single closure then defines the next element from this Tuple.

Here, for example, is a rolling average of the last 4 members of a Stream[Double]:

You can use Iterator.sliding(n) to get a similar effect. And that does work on infinite, non-strict streams. Personally, I just though this technique was so cool, and it does have the benefit of strongly-typed tuples. (Iterator.sliding() simple provides more Streams. Try it out if you’re curious.)

I’m laboring under an extremely waterfall project right now. A client asked for a set of REST-style services, but the client’s making me write approx. 200 pages of documentation before writing the code.

(Most of the documentation could be replaced by a far smaller, and more logically accurate, Operation State Modeling description. But that’s not what I’m here to talk about…)

Writing the REST-style interface, something pretty interesting occurred to me. I’ll need the casual reader to put on his thinking cap before moving on here, so please do that if you haven’t already. I’ll wait…

What is a Future?

Let’s start with what a future is: To review, as the linked document says, a future is “…a place-holder for the undetermined result of a (concurrent) computation…”. Let’s say there’s a value your program needs, but the computation or retrieval of that value takes a significant amount of time or resources. Instead of synchronizing (waiting) for the value to be computed, you could spin off an asynchronous process to compute/retrieve the value; alternatively, you could leave a placeholder object that only starts the complex computation/retrieval on demand. Either way, your program leaves in place a placeholder object know as a future.

When your program eventually comes back to the future object asking for the value, the the future either a) hands it back immediately if it has a cached copy; or b) the main program synchronizes (blocks) until the background process completes its computation/retrieval.

The object graph produced by Hibernate when you make an object query uses futures to represent unretrieved values. The query result Hibernate produces is an object graph with placeholder Collections representing lazy relationships. The lazy collections initially don’t have any data in them, but instead have just enough data in them to generate a secondary Hibernate query. When the program tries to access a lazy collection’s contents, only then does the Collection query a Hibernate entity manager for the collection of related objects it represents. The placeholder Collection objects are thus futures.

URL/URI References as Lazy Futures

When a REST-style service response contains a link URL/URI reference to another entity, you can think of that link as a kind of future in the same way as a Hibernate lazily populated Collection. One could easily populate a graph of objects from JSON source, replacing URL/URI references with lazy future objects.

In Scala, a Future[X] extends Function1[X], meaning that Future is Function subtype, so you can use a Function1[_] type to represent URL/URI-based references:

From Timefire:

First, let’s just dispense with the details and show you a working example, yes, this is working code:

Code block
   
public void onModuleLoad() {
    $("div").css("color", "red").click(new Function() {
      public void f(Element e) {
        Window.alert("Hello");
        $(e).as(Effects).fadeOut();
      }
    });
  }

If you haven’t seen that before, and if the left side of you brain didn’t just rupture the coronal suture between the frontal and parietal bones of your skull, then get find an absorbent mop and read it again.

Inheriting and using GQuery is as easy as importing in to your module. Of course you need to download and build GQuery, then try it out!

You think you know how to define a for loop, do you? Scala has 4 different kinds.

---

1. Traditional for loop. Loops through the members of an Iteratable. The result of the loop block is Unit.

Not surprisingly, this loop prints out the members of the collection, in order that they are returned by the collection’s iterator. In fact, this loop construct is translated exactly to

---

2. For…Yield Loop. Use the yield keyword within the body of a for loop. The loop block has a result which is an iterator of the yielded results. Thus the for loop can act as the RHS of an assignment.

Each time through this example loop, the result of the loop is yielded out. All the yielded results are collected in to a single iterator. This type of for loop is syntactic sugar for the following statement

That’s right, a simple for…yield loop is just syntactic sugar for Iteratable.map().

---

Interlude: Watch Out for Infinite Iterables

Infinite (or extremely large) iterables, usually implemented as Streams or Ranges, are a bit tricky. For example, it probably wouldn’t surprise you to learn that the following traditional for loop will not terminate (at least not in your lifetime):

But executing this next for…yield loop does not cause an indefinite wait — it works fine! (Go ahead, try it out!)

How can this be so? You need to know that o to 10000000000 results in a Range object, which is a non-strict iterator. That is, it is an iterator that calculates the “next” value as you iterate through it, rather than pre-computing all members up front and storing them in memory when the Range is created.

Iteratable functions that yield a scalar result, like foldLeft() or length() or any catamorphic function, are not safe to use with non-strict iterators, because these methods must iterate over the entire iteratable’s contents to produce a result. The foreach() method falls in to this unsafe category — foreach()‘s return type is Unit, which is considered scalar.

Remember the traditional for loop is just syntactic sugar for a foreach() call, and this explains why looping over the range with a traditional loop causes an indefinite wait.

Iteratable functions that yield results of similar size to the input, such as map() and flatMap(), are perfectly safe to use with non-strict iterators. These methods themselves yield non-strict iterators, and don’t need to iterate over the source iterator’s contents to yield a result.

The for…yield loop is just syntactic sugar for a map() call, and this explains why looping over the Range with a for…yield loop doesn’t cause an indefinite wait.

End of Interlude

---

3. Guarded For…Yield Loop. Throw an if guard in to a for..yield loop, and its translated to a filter-map pipeline (perhaps I should say filter |> map, using the pipe operator).

And as the example implies, this for loop style is non-strict collection safe (because both filter() and map() are non-strict safe).

---

4. Nested For Loop. Nesting iteration over iterators is syntactic sugar for a nested flatMap() call. Stare at the loop below and the form it gets translated in to by the Scala compiler for a little bit, you don’t need my explanation:

Note that nested for loops are also non-strict safe, because filter(), map() and flatMap() are all non-strict safe. Only the traditional for loop is not safe for non-strict use.

Catamorphism is the $20 word for a function that condenses a set of values down to a single value. For example, in SQL the terms grouping function or aggregator function refer to catamorphic functions, which includes SQL functions AVERAGE(), SUM(), and MEDIAN(). All of these SQL functions have in common a pattern that operates on a set of numeric values, and returns a single numeric value.

In functional programming, catamorphisms are embodied in the folds functions. I like Scala, so I’ll discuss some catamorphic function examples in found in the standard Scala library’s Iterable trait.

The Iterable trait in Scala represents the concept of an ordered series of same-typed values. This trait is common to all the collection types (Lists, Seqs, and Streams, and many other “series like” things). The Iterable trait defines several catamorphic functions — functions which reduce the series of values down to a single value.

Perhaps the simplest to start with is the forall() and exists() methods. These methods each take a Boolean test operation, applies it to each member of the Iterable, and returns the logical AND (forall()), or OR (exists) of the test values. Despite the fact that they act differently, both methods have the same type signature, because both are Boolean catamorphisms:

Event more generic are the foldLeft() and foldRight() methods in Iterable. foldRight() is really just a slight tweak on foldLeft(), so I’ll just explain left-hand version and for the right-hand version I’ll just do some… erm… hand-waving.

The foldLeft() method receives an aggregator function used to combine each member of the Iterable successively in to an aggregated value. foldLeft() starts with an initial value, which it combines with the first Iterable value using the aggregator function. The result is then aggregated with the second Iterable value; and then the third, and so on, until the entire Iterable has been reduce down to a single aggregated value. Thus foldLeft() is a catamorphism, as it combines and reduces the series of values represented by an Iterable down to a single value.

foldRight() is the same as foldLeft() — the only difference being which end of the series each starts with. The foldLeft() function starts with the head of the series (traditionally drawn to the left of the tail — hence the name) and proceeds to the coda; foldRight() starts with the series coda and proceeds to the head.

If you think for a second, you can see how forall() and exists() could be implemented using foldLeft():

In addition to forall() and exists() there are other catamorphic convenience functions in Iterable, and as it turns out each of them could be re-implemented using foldLeft() (or foldRight()) as well. For example, mkString() creates a string representation of the Iterable series by combining each member’s toString() value, along with a prefix, a suffix, and a list separator string.

Follow-up: What a great week! We had a few really sharp students, and were able to discuss and learn about all kind of extra material in addition to the materials in the course outline. It was especially rewarding to see the students’ appreciation of the amazing new features of GWT 2.0 — Code Splitting, resource generators and code generators, and we worked out how to make a GWT Widget/jQuery UI bridge: all in addition to the course materials!

We’ve published an in-depth description page, or you can go to visit the ticket purchasing page right away. GWT is an extremely valuable tool to add to your development toolkit. This amazing in-depth instruction will up your personal capitol, and your developer skill-set, quickly!

We’ve got several open seats of the date of this posting, so we’re trying something novel to get those seats sold: every day until Jan. 25, the price of each seat is going down by $300! On Jan. 19, the price was $2,000 for all 5 instructional days (normally $3,750). The next day, Jan. 20, the price went down to $1700, and so on.

If there are still seats open on Jan. 25th, they will sell for just $500 for all 5 instructional days!

You can also purchase fewer than 5 days. If you just want a quick instruction to GWT, attend just days 1 and 2. For more in-depth mastery of GWT, attend days 3, 4, or 5! Its up to you to decide how to increase your personal value and abilities as a developer with this unique set-up.

Location, BTW, is at the Smart-Soft training location in Irvine, CA.

Like most software professionals, I have had a troubling, shameful secret. I had no idea what monad means. More importantly, I had no idea why I should know what monad means. But I had a sneaking suspicion that I should.

Tony Morris is this incredibly knowledgeable and opinionated (in a good way) CS professor who I know because of his swaggering around the Scala-Users mailing list. I love Tony because he takes the time to try explain important CS concepts, as in this video.

The slides in the video are incredibly fuzzy. From Tony’s website, the talk’s slides. [Link updated 8/22/2011 when Tony Morris left comment below about slides being moved. - BDM]