28 September 2009

Posted in Arrangement, Art, Business

Scala: Multi-core for loop

Two of the things that really attract me to Scala are its immutability and threading approaches. It seems to me that both of these make Scala a great candidate for multi-core and cloud-friendly constructs.

Here is what I developed to help me write a multi-threaded iterator (of course, these are only used when the order of execution for each element doesn't matter):

def fire[A,B](expanded:A=>B):Fire[A,B] = new Fire[A,B](expanded);

class Fire[A,B](val expanded:A=>B) {
def forall(list:Iterable[A]):Any = {
var caller = self;
list foreach {
el =>
actor {
caller ! expanded(el)

list map {
el =>
receive {
case result:B => doReceive(result)

protected def doReceive(result:B):Any = {


def andlisten[C](contracted:B=>C):FireAndListen[A,B,C] = new FireAndListen(expanded, contracted)

class FireAndListen[A,B,C](expanded:A=>B, contracted:B=>C) extends Fire[A,B](expanded) {
override def doReceive(result:B):C = contracted(result);

Thus, you get something that looks like this:

fire {
myVar:myType => myMethod(myVar1)
} andlisten { //optional
myVar2 => myMethod2(myVar2) // the type of myVar2 is the return of the previous method. If the return type is Unit, then this section is unnecessary
} forall myList

This is effectively the same as:

myList foreach {

except that it performs better on multi-core computers.
Posted in Arrangement, Art, Business

Scala: Map of Lists, Part II

A couple of posts ago, I talked about an attempt to create sort of a group by method for partitioning an existing list.

Well, I fussed over it for a while longer and came up with this:

Instead, I created an implicit function, and removed the projection part (C to B) to get the following:

implicit def listenhancer[C](list:List[C]) = new {
def groupby[A](keyresolver:C=>A) = {
(list map { keyresolver(_) } removeDuplicates)
map { key => (key, list filter { key equals keyresolver(_) }) }
: _*

Now, I know that some in the Scala community don't like implicit functions. For now, I think that they create very clean-looking code, and so they are still on my "Good Coding Practice" list.

Anyway, one of the things that I noticed about the other is that the keys list had a whole bunch of duplicates, and so the valueresolver method was getting called too many times. So, I added the 'removeDuplicates' call. Second, I decided that, to keep the goal of this method succinct, I wasn't going to support projection unless there was a compelling reason.

So, now, I can just call it like this:

val assetsMap = assets groupby { asset => asset.getName.substring(0, asset.getName.indexOf("_file")) }

Doesn't that look a lot nicer?
Posted in Arrangement, Art, Business

Scala and Testing

In our company, we have a rich and mature Java stack that we draw from for nearly every project. However, I politely told my manager that, for this project, we would be using Scala. I explained that we would still be able to use everything on the Stack, and we would get it done faster and cheaper that way.

So, happily, he gave me a little bit of line.

When it came to writing Unit Tests, we wanted to use EasyMock, an awesome Java library for mocking, stubbing, etc.

Everything worked great, but I ran into one little problem:

If I have the following code:

... do unit testing ...

Then, I get on the replay call:

java.lang.IllegalArgumentException: not a proxy instance
at java.lang.reflect.Proxy.getInvocationHandler(Proxy.java:637)
at org.easymock.EasyMock.getControl(EasyMock.java:1600)
at org.easymock.EasyMock.replay(EasyMock.java:1502)...

This is because I am not using the classextension.EasyMock class. However, if I use that, then Scala complains that "expect" isn't a member of classextension.EasyMock.

So, here is the solution that I came up with:

I use import shortnames to include both EasyMock classes:

import org.easymock.{EasyMock=>EM};
import org.easymock.classextension.{EasyMock=>CEM};

And then in the code, I do the following:

EM expect(myClass myMethod myParam) andReturn myReturn
CEM replay myClass
...unit test stuff...
CEM verify myClass
CEM reset myClass

And it works great!

Has anyone else found a good solution for this?

The next problem that I have run into is method matching when one of the values passed in is an array projection.

Observe the following code:

myClass.myMethod(param1, param2, str split "_" drop 1)

The last parameter returns type Array.Projection. If I have the exact same code in my expect, then this particular parameter is still listed as being unequal.

The only way that I have found so far around this is:

EM expect(myClass.myMethod(EM anyObject[type1], EM anyObject[type2], EM anyObject[type3])) andReturn myReturn

Not a big fan of this strategy, but it works. Any ideas?

25 September 2009

Posted in Arrangement, Art, Business

Scala: Map of Lists

Every developer has had to deal with a Map whose key is of type X and whose value is a list. Additions to the map often happen like this in Java:

List l = map.get(key);
if ( l == null ) {
l = new ArrayList();
map.put(key, l);

Kind of cumbersome, and kind of not thread-safe.

What is to be done?

Well, google-collections tries to take care of the cumbersome-ness with their MultiMap class. The put looks the same as in java.util.Map, but the get returns a List instead.

So, instead of the code above, you get:

map.put(key, value);

Now, if you want thread-safety as well, you call MultiMaps.synchronizedMap, passing your existing MultiMap.

So, I had a specific case that I was working on which was basically a group by function given a list of C. My decision was to create in the end a map with an A for the key and a List of B for the value where A is the type of the group by property and B a projection of C (So, the List could be of C as well).

Here is what I came up with:

def inclAsList[A,B,C](keyresolver:C=>A, valueresolver:A=>List[B], list:List[C]) = {
(list map { el => keyresolver(el) } removeDuplicates)
map { key => (key, valueresolver(key)) }
: _*

So, this is thread-safe. What it does is takes the list of C and applies a function to find the group by properties. Then, iterating through those, it applies a function with each key to get a list of values that match.

"But what about 'put'?" you say. The thing is, once you have immutability built in, you will find that the need to be able to modify a map doesn't come up as often. This example is prime--because Maps and Lists are immutable in Scala, we are nearly there before we know it regarding thread safety.