Part  2 - More IronPython and delegates...

So we have briefly covererd strongly typed delegates and event handlers... But they make the assumption that we know what our arguments are at compile time, what if we dont?



Say for instance you're expression engine has a bag of contextual values being passed around in a dictionary, like this:

Dictionary context = new Dictionary();

context.Add("user", "alex");

context.Add("age", 26);

What if we want to use python to evaluate expressions against that context?... Say something like writing out "my name is alex and my age is 25" - the expression in python is easy enough, we can go:

my name is' + name + ? and my age is ? + str(age)

But how do we marry all these together... lets start exploring... ever noticed that delegates have a DynamicInvoke method with a signature like this?

object DynamicInvoke(params object[] parameters)

Perhaps we can try and use this to our advantage... lets see shall we?

First Attempt

[Test]

[ExpectedException(typeof (ArgumentException), "T must be a concrete delegate, not MulticastDelegate or Delegate")]

public void LooseDelegateAndDynamicInvoke()

{

    PythonEngine engine = new PythonEngine();
    List parameters = new List(new string[] {"name", "age"});
    Delegate func =

        engine.CreateMethod("return ?my name is' + name + ? and my age is ? + str(age)", parameters);

    string result = (string) func.DynamicInvoke("alex", 26);
    Assert.AreEqual("my name is alex and my age is 25", result);

}

Sadly this doesn't work so well, seems the PythonEngine is looking for a concrete delegate... we could try giving it void delegate, but it does type checking on the number of parameters and their type, so we're a little stuck.



At this point I think the best answer is to actually build some code which generates the apropriate delegate type, based on the supplied dictionary, and then passing that onto the python engine, but there is another way... it's a little less elegant, but it's at least amusing:

Round 2

First of let's build a little helper method...

private static string GenerateFunction(string functionName, string[] parameters, string statements)

{

    StringBuilder builder = new StringBuilder();
    builder.AppendFormat("def {0}(", functionName);
    for (int i = 0; i < parameters.length;="">

    {

        if (i > 0) builder.AppendFormat(", ");

        builder.AppendFormat(parameters[i]);

    }
    builder.AppendFormat("):rn    ");
    builder.AppendFormat(statements.Replace("rn", "n").Replace("r", "n").Replace("n", "rn    "));
    return builder.ToString();

}

You can no doubt see where this is going - so given a statement like this:

GenerateFunction("func", new string[] { "user", "age" }, "return user + ' is ' + str(age) + ' years old'")

We'd end up with a string like this:



    def func(user, age):

        return user + ' is ' +
str(age) + ' years old'



And, moving on from there we build a test...

[Test]

public void GeneratingPythonFunction()

{

    Dictionary context = new Dictionary();

    context.Add("user", "alex");

    context.Add("age", 26);
    List parameters = new List(context.Keys);

    List
    PythonEngine engine = new PythonEngine();
    engine.Execute(

        GenerateFunction("func", parameters.ToArray(), "return user + ' is ' + str(age) + ' years old'"));
    PythonFunction func1 = engine.EvaluateAs("func");
    engine.Execute(

        GenerateFunction("func", parameters.ToArray(),

                         "return user + ' is ' + str(age+1) + ' years old next year'"));
    PythonFunction func2 = engine.EvaluateAs("func");
    object result1 = func1.Call(values.ToArray());

    Assert.AreEqual("alex is 26 years old", result1);
    object result2 = func2.Call(values.ToArray());

    Assert.AreEqual("alex is 27 years old next year", result2);

}

Are there problems with this?... well there's a few, unsurprisingly!

• PythonFunction's are not delegates.
• String manipulation is a bit clunky
• Because these are named functions, you probably want to lock against some object when generating the function, to avoid cross-threading issues...

Back to delegates

Aside from the problems it works alright, but what we really want is a delegate... those PythonFunctions don't give you the flexibility to substitute IronRuby in the future now do they?

So first off, lets declare a delegate suitable for our purposes.

[ThereBeDragons("Only use as a last resort")]

public delegate object UntypedDelegate(params object[] parameters);

Aint she a beauty ;o) - full credit for the ThereBeDragons attribute goes to Ayende, though it's probably not really warranted in this situation - now, let's rework the last test to use a delegate instead, a simple anonymous delegate will do the dirty work:

[Test]

public void UntypedDelegateForPythonFunction()

{

    Dictionary context = new Dictionary();

    context.Add("user", "alex");

    context.Add("age", 26);
    List parameters = new List(context.Keys);

    List
    PythonEngine engine = new PythonEngine();
    engine.Execute(GenerateFunction("func",parameters.ToArray(),"return user + ' is ' + str(age) + ' years old'"));
    PythonFunction func1 = engine.EvaluateAs("func");
    UntypedDelegate func1Delegate = delegate(object[] param)

    {

        return func1.Call(param);

    };
    object result1 = func1Delegate(values.ToArray());

    Assert.AreEqual("alex is 26 years old", result1);

}

It's certainly better then passing around PythonFunction instances - though you need to be a little careful... there's a little quirk here, if we were to use:

object result1 = func1Delegate.DynamicInvoke(values.ToArray());

It's going to fail because the wrong number of arguments were supplied (it expects only 1, an array), so our delegate doesn't really behave like it has multiple parameters.. so to dynamically invoke this delegate we'd need to take special care, promoting the arguments into a second array like so:

object result1 = func1Delegate.DynamicInvoke(new object[] { values.ToArray()});

Next Time

When I get bored I'll write a version which doesn't require the clunky generator and post it up...



Next time I'll talk about writing classes which are python-friendly, riveting stuff eh? As you were.

I felt like posting some code snippets... so I was looking around
and thought perhaps some IronPython might be interesting.... Some
of you may find this useful, possibly not.  I tend to
explore functionality using unit tests, so all my code will be
test cases - I might make this part of a small series on looking
at python code with a view to integrating it into an existing
system as a scripting engine.  Lets call the series
"IronPython is your friend"... because it really is.

Part 1 - IronPython and delegates....

So you have downloaded the IronPython runtime dll's, and you
would like to start integrating it into some project your working
on... first off lets create a python engine, and make sure it
goes....

[Test]

public void GetGoing()

{

    PythonEngine engine = new PythonEngine();

    engine.Execute("myString = 'hello'");

    Assert.AreEqual("hello reader", engine.Evaluate("myString + ' reader'"));

}

That was pretty easy; we executed some python code to assign the
value hello to myString, and then evaluated the expression - sweet
you might say - no effort required.



Now, you could write your scripting engine so that it passed the
strings to the engine for evaluation every time you wanted to
evaluate them - but this strings malarkey isn't very pleasant...
what we really want is a decent god fearing .Net delegate, so we
can call this without having to think about the python engine....
Lets check out our first destination, method delegates.

Method Delegates

Given a converter delegate like this:



   
Converterint,
string>



(Which by the way, if you haven't stumbled across the generic
ConverterTInput,TOutput> delegate yet,
effectively you end up with a delegate method of the
definition"TOutput Converter(TInput)" - pretty
handy actually)

 

We could do this:

[Test]

public void StatementsToStronglyTypedDelegate()

{

    PythonEngine engine = new PythonEngine();

    List parameters = new List(new string[] {"age"});

    Converter converter =

        engine.CreateMethod<>>("return str(age+1)", parameters);

    Assert.AreEqual("11", converter(10));

}

Using a strongly typed overload for the engine's
CreateMethod method, we're able to evaluate statements using
our new delegate, which in turn uses our IronPython method we
created earlier... looks promising!

Events

Now delegates are cool,  but it's only half the story for
scripting in an application - depending on your model, you may be
aiming to produce results similar to a microsoft's VBA in
applications like Excel or Word - and for this we need to be able
to assign python code to events on our own classes... we can do
this using delegates of course, or we could let the python (user
code) do this itself, and save some time.... So lets have a look at
the latter:

[Test]

public void HookingToEvents()

{

    PythonEngine engine = new PythonEngine();
    ManualResetEvent waitHandle = new ManualResetEvent(false);

    WebClient client = new WebClient();
    List results = new List();
    engine.Globals["client"] = client;

    engine.Globals["results"] = results;
    engine.Execute(

        @"def storeResult(sender, e):

    results.Add(e.Result)

    e.UserState.Set()
# assign out storeResult function as an event handler for the client class

client.DownloadStringCompleted += storeResult

");
    client.DownloadStringAsync(

        new Uri(

            "http://api.feedburner.com/awareness/1.0/GetFeedData?uri=http://feeds.feedburner.com/BitterCoder"),

        waitHandle);
    Assert.IsTrue(waitHandle.WaitOne(10000, false), "timeout occured after 10 seconds");
    Assert.AreEqual(1, results.Count);
    Assert.IsTrue(results[0].StartsWith("<>

}

     

This test is a bit clunky, but you get the idea... we are:

• Creating a web client, for downloading some xml from a web
  site.
  
• Set the client as a global variable for the default module in
  the python engine.
  
• Execute some python, which creates a function, and assigns
  that function as a handle for the web client's
  DownloadStringCompleted event (take note at how concise the code
  is, this would be a lot uglier in C#)
  
  
• start the async download
  
• Make sure we got a response, and that the python code did
  what it should.
  

All in all, with a little finesse you can recreate a model
similar to VBA, driving your extension points from events - and
this certainly has some merit - obviously you want to create the
code in their own modules, as opposed to using the default module
for the python engine as we are.



We haven't quite finished with delegates yet though, I'll leave
that for part 2 - which I'll post shortly.

It seems like batching is a data access "theme" at the moment (much like Castle, IOC & IronPython are all themes in the NZ Blogspace at the moment... even though none of them could be considered new, I've been using Castle IoC container for 1 & 1/2 years now!) - no sooner do I discover the batching support in the Rhino.Commons library, it appears NHibernate now supports it in the latest beta drop and it's a planned feature for the upcoming version 3 of the enterprise library ... It's interesting that this support never made it's way into the framework itself for version 2.0, it's all but there.



On the flip side, for NHibernate at least, the performance boons of batching are impressive (see below for an image from Ayende's blog where he's done some profiling), I haven't checked Base4.net out in a while, but I wonder if Alex James has had a chance to implement batching support yet? - maybe I should make that my task for the weekend (checking out base4.net again that is heh... not implementing support ;o)

Over the past couple of days I've been working on using DirectShow
and DES (DirectShow Editing Services) for re-encoding mobile video
content suitable for posting on the web... With the goal of
including video editing into our unified "media engine" which the
Syzmk line of products makes use of... as an off-shoot of that work
I decided to refactor the rather ugly code we had for working with
DES over the weekend into something a little less ugly, that hid
most of the details away, say hello to:







(sorry, couldn't resist the lame web 2.0 logo ;o)



A simple little library for doing most of the useful things in DES,
the project is hosted here
up on Codeplex.  It's currently in an
alpha state, at version 0.9.0.0 - and should get feature drops
every week or so until I think it can go into beta.



To give an idea of what I'm up to, it's basically just wrapping up
the various DES components and removing alot of the unpleasantness
(ie. you get to deal with human-friendly units of time like
seconds) .. and seperating the concerns of the timeline, and
rendering that timeline into something useful like audio or video
container (currently that's just WAV, AVI & Windows Media
formats).  It's fully functional, but the interfaces at this
point it isn't very stable as I'm likely to refactor the various
interfaces mercilessly till I get it working how I want, so if you
build something with the library now, it's likely to break in the
future (but probably not in major way, it is after all mirroring
the functionality available in DES).   The added bonus of
developing a set of rich wrappers for a lot of this stuff is that I
can mock away the requirement on DES altogether, which is always a
bonus considering how long the units tests currently take to run
for the Syzmk RMP product.



When I get a chance, I'll put up some code examples to demonstrate
how you might use it... At any rate it'll be going through some dog
fooding in the next month or so while I integrate it with the Syzmk
product, so you'll probably here a little more about in the
future.

After a long hiatus from Blogging, I've returned... hopefully with a vengence, but more likely with a small "vurrrrp" sound... like hot air bubbling up through porridge.



Lets start off with a recap... Spectacles, testacles, wallet and all that jazz.



My name is Alex Henderson, 26 years young, living in Auckland, New Zealand.  I've been working on code since I was knee high, cut my teeth in GW Basic while in primary school, and C++ a few years after that... Now days I tend to spend my hours working in C# code, and my spare time in either .Net and ruby or python when I feel like a change.



For the last year and a half I've been working for a start up company called Seismic Technologies which works in a rather ill-defined space, which I'll probably talk about in posts to come.



The main product I've been working on for Syzmk is the "Rich Media Processor" - it's a message clearing house, geared
towards taking messages with rich content, transforming them, and then distributing them... The messages might be files on a network share, emails, SMS or MMS messages from a phone etc.  It's all about rich content, so a large part of the product deals with transforming the content before furnishing it to line of business
systems - we have been working with companies like XSOL who use our product to use messages from mobile devices to extend their business processes outside of the host organisation.



Before I was working for the Seismic, I was travelling around Asia for 4 months... which was a lot of fun, I hope to visit europe next... Then maybe Japan, Egypt and South America.