Quietly, Blumenfeld & Maso has been building up a software portfolio for Twitter analytics of the past few months. Last night, we go more than our feet wet with live tracking and reporting for the #UBP10 party from 5 Minutes for Mom (thanks to @momfluential for setting that up.)
Some basic stats to give you a taste of how successful the #UBP10 Twitter party was: 15,704 tweets from 1,204 attendees, with combined reach 1,159,469 followers, and 29,397,282 unique messages in 7 days. Fully half that traffic came during the 2 hour #UBP10 party last night!
Over the next couple weeks we’ll be rolling out more and more of our tracking and visualization tools. (Please contact us for more info if you’re interested in hearing more!)
Twitter hashtag tracking statistics and analytics
Just reviewed a gem from Programming Scala. Of all the Fibonacci implementations I’ve seen, my new favorite is below. Its 1 statement long, and there’s not a recursive function in sight:
lazy val fib: Stream[Int] = Stream.cons(0, Stream.cons(1, fib.zip(fib.tail).map(p => p._1 + p._2)))
If you have not seen that zip trick before, follow me on a little explanation. The code defines a Stream — a non-strict iterable — that begins with two literal values “0″ and “1″. The stream then continues the Fibonacci sequence by zipping the sequence to itself. More specifically, to its own tail sequence.
This figure illustrates what the zipper is creating.
The tail value of the initial sequence is just the sequence starting with the literal value “1″. The zipper creates Pairs out of the each member of the sequence pairwise joined with the next member of the sequence.
The first pair is (0, 1).
The second pair is then (1, 0 + 1) = (1, 1).
The third pair is then (1, 1 + 1) = (1, 2).
The forth pair is then (2, 1 + 2) = (2, 3). And so on.
The coolest part is of course the complete lack of apparent recursion. The whole sequence is lazily evaluated, so the Stream takes up little space — one new Stream object per element in the sequence.
We can generalize this stream-zipping technique. When the value of a Stream element n can be calculated from the previous k sequence members, we can use a k-ary version of this technique. That is, if the stream theStream member n can be defined by some function s:
def theSeq(n) = s(theSeq(n-1), theSeq(n-2), …, theSeq(n-k))
We can define the stream in a single statement thus:
Here, for example, is a rolling average of the last 4 members of a Stream[Double]:
val data: Stream[Double] = ... val padded_data = Stream.fill(4)(0.0) ++ data ++ Stream.fill(4)(0.0) // Note: tail padding not a problem even if data is infinite. /* Here's where the stream is joined to itself. Also, mapping the (((Int,Int),Int),Int) to (Int,Int,Int,Int) for readability. Can't be recursively defined */ def zip4[A](str: Stream[A]): (A,A,A,A) = (str zip str.tail zip str.tail.tail zip str.tail.tail.tail) map { p=> (p._1._1._1, p._1._1._2, p._1._2, p._2)} /* Could be recursively defined in terms of base type Product */ def avg4(p: (Double,Double,Double,Double)): Double = (p._1, p._2, p._3, p._4) / 4 /* Finally, generating the rolling-average stream */ lazy val avg_rolling: Stream[Double] = zip4(padded_data) map (avg4)
You can use Iterator.sliding(n) to get a similar effect. And that does work on infinite, non-strict streams. Personally, I just though this technique was so cool, and it does have the benefit of strongly-typed tuples. (Iterator.sliding() simple provides more Streams. Try it out if you’re curious.)
Super-segue I’d like to take you on for no reason, other than to help you understand what its like to be me some times…
Just read “Programming and fallacies” on Michael Galpin’s blog.
Made me think of the very (very, very) old “Love is a Fallacy” humor writing by Dobie Gillis (book) author Max Shulman (– please read that some time, its such a riot). And when I say “old” I mean the original story was old when the old black-and-white Dobie Gillis TV show was on. In case you aren’t familiar, that was the show that launched the career of “Gilligan’s Island”’s Gilligan star Bob Denver before Gilligan.
But my segue-adled mind doesn’t stop there, because Dobie Gillis is no doubt the best example of modern humor a logical mind can ever read. And who defined modern humor? No shit: Freud. You probably didn’t know he wasn’t famous in his early career not for his mommy-mangled psycho-sexual theories — his PhD thesis in fact was the seminal text on laughter and humor.
That I know all those bizarrely ancient pop cultural references, and you don’t, means that
I’m laboring under an extremely waterfall project right now. A client asked for a set of REST-style services, but the client’s making me write approx. 200 pages of documentation before writing the code.
(Most of the documentation could be replaced by a far smaller, and more logically accurate, Operation State Modeling description. But that’s not what I’m here to talk about…)
Writing the REST-style interface, something pretty interesting occurred to me. I’ll need the casual reader to put on his thinking cap before moving on here, so please do that if you haven’t already. I’ll wait…
Let’s start with what a future is: To review, as the linked document says, a future is “…a place-holder for the undetermined result of a (concurrent) computation…”. Let’s say there’s a value your program needs, but the computation or retrieval of that value takes a significant amount of time or resources. Instead of synchronizing (waiting) for the value to be computed, you could spin off an asynchronous process to compute/retrieve the value; alternatively, you could leave a placeholder object that only starts the complex computation/retrieval on demand. Either way, your program leaves in place a placeholder object know as a future.
When your program eventually comes back to the future object asking for the value, the the future either a) hands it back immediately if it has a cached copy; or b) the main program synchronizes (blocks) until the background process completes its computation/retrieval.
The object graph produced by Hibernate when you make an object query uses futures to represent unretrieved values. The query result Hibernate produces is an object graph with placeholder Collections representing lazy relationships. The lazy collections initially don’t have any data in them, but instead have just enough data in them to generate a secondary Hibernate query. When the program tries to access a lazy collection’s contents, only then does the Collection query a Hibernate entity manager for the collection of related objects it represents. The placeholder Collection objects are thus futures.
When a REST-style service response contains a link URL/URI reference to another entity, you can think of that link as a kind of future in the same way as a Hibernate lazily populated Collection. One could easily populate a graph of objects from JSON source, replacing URL/URI references with lazy future objects.
In Scala, a Future[X] extends Function1[X], meaning that Future is Function subtype, so you can use a Function1[_] type to represent URL/URI-based references:
/** * Invoice can be deserialized from JSON such as: * { * id: "XYZ", * lineItems: "http://somewhere.com/inv/XYZ/lineItems" * // ^^^ URL; serialized representation of * // a List[LineItem] future. * } **/ class Invoice(val id: String, val lineItems: () => List[LineItem], ...) { ... }
From Timefire:
First, let’s just dispense with the details and show you a working example, yes, this is working code: Code block public void onModuleLoad() { $("div").css("color", "red").click(new Function() { public void f(Element e) { Window.alert("Hello"); $(e).as(Effects).fadeOut(); } }); }
First, let’s just dispense with the details and show you a working example, yes, this is working code:
Code block public void onModuleLoad() { $("div").css("color", "red").click(new Function() { public void f(Element e) { Window.alert("Hello"); $(e).as(Effects).fadeOut(); } }); }
public void onModuleLoad() { $("div").css("color", "red").click(new Function() { public void f(Element e) { Window.alert("Hello"); $(e).as(Effects).fadeOut(); } }); }
If you haven’t seen that before, and if the left side of you brain didn’t just rupture the coronal suture between the frontal and parietal bones of your skull, then get find an absorbent mop and read it again.
Inheriting and using GQuery is as easy as importing in to your module. Of course you need to download and build GQuery, then try it out!
<module ...> <inherits name='com.google.gwt.query.Query'/> ... </module>
Response to last week’s GWT 2.0 training was pretty enthusiastic! So much so, that we’re running another week of training: 2.2.2010 through 2.5.2010. If anyone is interested in joining us in Irvine, for all or just part of the time, please contact me (@bmaso on twitter, or use the contact form). We can offer some pretty good prices for the remaining last-minute seats!
In addition to normal GWT, here’s the super cool topics we’ll be covering:
This course is especially good for J2EE developers who don’t feel completely (or even at all) comfortable in JavaScript or the browser environment — you will end your time with us with some amazing “Browser Fu“!
You think you know how to define a for loop, do you? Scala has 4 different kinds.
Unit
for(mbr <- collection) { println(mbr) }
Not surprisingly, this loop prints out the members of the collection, in order that they are returned by the collection’s iterator. In fact, this loop construct is translated exactly to
collection.foreach(mbr => println(mbr))
val transformed = for(mbr <- collection) { yield transform(mbr) }
Each time through this example loop, the result of the loop is yielded out. All the yielded results are collected in to a single iterator. This type of for loop is syntactic sugar for the following statement
val transformed = collection.map(mbr => transform(mbr))
That’s right, a simple for…yield loop is just syntactic sugar for Iteratable.map().
Iteratable.map()
Infinite (or extremely large) iterables, usually implemented as Streams or Ranges, are a bit tricky. For example, it probably wouldn’t surprise you to learn that the following traditional for loop will not terminate (at least not in your lifetime):
Stream
Range
for(i <- 0 to 100000000000) println(i)
But executing this next for…yield loop does not cause an indefinite wait — it works fine! (Go ahead, try it out!)
val numberStrings = for(i <- 0 to 100000000000) yield ("" + i)
How can this be so? You need to know that o to 10000000000 results in a Range object, which is a non-strict iterator. That is, it is an iterator that calculates the “next” value as you iterate through it, rather than pre-computing all members up front and storing them in memory when the Range is created.
o to 10000000000
Iteratable functions that yield a scalar result, like foldLeft() or length() or any catamorphic function, are not safe to use with non-strict iterators, because these methods must iterate over the entire iteratable’s contents to produce a result. The foreach() method falls in to this unsafe category — foreach()’s return type is Unit, which is considered scalar.
foldLeft()
length()
foreach()
Remember the traditional for loop is just syntactic sugar for a foreach() call, and this explains why looping over the range with a traditional loop causes an indefinite wait.
Iteratable functions that yield results of similar size to the input, such as map() and flatMap(), are perfectly safe to use with non-strict iterators. These methods themselves yield non-strict iterators, and don’t need to iterate over the source iterator’s contents to yield a result.
map()
flatMap()
The for…yield loop is just syntactic sugar for a map() call, and this explains why looping over the Range with a for…yield loop doesn’t cause an indefinite wait.
End of Interlude
// Using the guarded for...yield loop syntactic sugar val oddSquares = for(i <- 0 to 10000000000 if i % 2 == 1) yield (i*i) // Exactly the same thing as val oddSquares = (0 to 10000000000).filter(i=>i % 2 == 1).map(i=>i*i)
And as the example implies, this for loop style is non-strict collection safe (because both filter() and map() are non-strict safe).
filter()
// Long-winded nested for loop val pairsThatSumTo100 = for(i <- 0 to 100; j <- i to 100 if i + j == 100) yield Pair(i, j) // Slightly shorter but harder to read raw form that gets compiled val pairsThatSumTo100 = (0 to 100).flatMap(i=>(i to 100).filter(j=>i+j==100).map(j=>Pair(i,j)))
Note that nested for loops are also non-strict safe, because filter(), map() and flatMap() are all non-strict safe. Only the traditional for loop is not safe for non-strict use.
Catamorphism is the $20 word for a function that condenses a set of values down to a single value. For example, in SQL the terms grouping function or aggregator function refer to catamorphic functions, which includes SQL functions AVERAGE(), SUM(), and MEDIAN(). All of these SQL functions have in common a pattern that operates on a set of numeric values, and returns a single numeric value.
AVERAGE()
SUM()
MEDIAN()
In functional programming, catamorphisms are embodied in the folds functions. I like Scala, so I’ll discuss some catamorphic function examples in found in the standard Scala library’s Iterable trait.
The Iterable trait in Scala represents the concept of an ordered series of same-typed values. This trait is common to all the collection types (Lists, Seqs, and Streams, and many other “series like” things). The Iterable trait defines several catamorphic functions — functions which reduce the series of values down to a single value.
Perhaps the simplest to start with is the forall() and exists() methods. These methods each take a Boolean test operation, applies it to each member of the Iterable, and returns the logical AND (forall()), or OR (exists) of the test values. Despite the fact that they act differently, both methods have the same type signature, because both are Boolean catamorphisms:
forall()
exists()
exists
trait Iterable[A] { ... def forall(test: (A) => Boolean): Boolean = ... def find(test: (A) => Boolean): Boolean = ... ... }
Event more generic are the foldLeft() and foldRight() methods in Iterable. foldRight() is really just a slight tweak on foldLeft(), so I’ll just explain left-hand version and for the right-hand version I’ll just do some… erm… hand-waving.
foldRight()
The foldLeft() method receives an aggregator function used to combine each member of the Iterable successively in to an aggregated value. foldLeft() starts with an initial value, which it combines with the first Iterable value using the aggregator function. The result is then aggregated with the second Iterable value; and then the third, and so on, until the entire Iterable has been reduce down to a single aggregated value. Thus foldLeft() is a catamorphism, as it combines and reduces the series of values represented by an Iterable down to a single value.
foldRight() is the same as foldLeft() — the only difference being which end of the series each starts with. The foldLeft() function starts with the head of the series (traditionally drawn to the left of the tail — hence the name) and proceeds to the coda; foldRight() starts with the series coda and proceeds to the head.
trait Iterable[A] { ... def foldLeft[B] (initialValue: B)(aggregatorFunc: (B, A) => B): B = ... def foldRight[B](initialValue: B)(aggregatorFunc: (B, A) => B): B = ... ... }
If you think for a second, you can see how forall() and exists() could be implemented using foldLeft():
trait Iterable[A] { ... def foldLeft[B] (initialValue: B)(aggregatorFunc: (B, A) => B): B = ... ... def forall(test: (A) => Boolean): Boolean = foldLeft(true)(_ && test(_)) def exists(test: (A) => Boolean): Boolean = foldLeft(false)(_ || test(_)) ... }
In addition to forall() and exists() there are other catamorphic convenience functions in Iterable, and as it turns out each of them could be re-implemented using foldLeft() (or foldRight()) as well. For example, mkString() creates a string representation of the Iterable series by combining each member’s toString() value, along with a prefix, a suffix, and a list separator string.
mkString()
toString()
trait Iterable[A] { ... def foldLeft[B] (initialValue: B)(aggregatorFunc: (B, A) => B): B = ... ... def mkString(prefix: String, separator: String, suffix: String): String = foldLeft(prefix)(_ + separator + _) + suffix ... }
Follow-up: What a great week! We had a few really sharp students, and were able to discuss and learn about all kind of extra material in addition to the materials in the course outline. It was especially rewarding to see the students’ appreciation of the amazing new features of GWT 2.0 — Code Splitting, resource generators and code generators, and we worked out how to make a GWT Widget/jQuery UI bridge: all in addition to the course materials!
We’ve published an in-depth description page, or you can go to visit the ticket purchasing page right away. GWT is an extremely valuable tool to add to your development toolkit. This amazing in-depth instruction will up your personal capitol, and your developer skill-set, quickly!
We’ve got several open seats of the date of this posting, so we’re trying something novel to get those seats sold: every day until Jan. 25, the price of each seat is going down by $300! On Jan. 19, the price was $2,000 for all 5 instructional days (normally $3,750). The next day, Jan. 20, the price went down to $1700, and so on.
If there are still seats open on Jan. 25th, they will sell for just $500 for all 5 instructional days!
You can also purchase fewer than 5 days. If you just want a quick instruction to GWT, attend just days 1 and 2. For more in-depth mastery of GWT, attend days 3, 4, or 5! Its up to you to decide how to increase your personal value and abilities as a developer with this unique set-up.
Location, BTW, is at the Smart-Soft training location in Irvine, CA.
Like most software professionals, I have had a troubling, shameful secret. I had no idea what monad means. More importantly, I had no idea why I should know what monad means. But I had a sneaking suspicion that I should.
Tony Morris is this incredibly knowledgeable and opinionated (in a good way) CS professor who I know because of his swaggering around the Scala-Users mailing list. I love Tony because he takes the time to try explain important CS concepts, as in this video.
The slides in the video are incredibly fuzzy. From Tony’s website, the talk’s slides.
