Wednesday, February 25, 2009

Binding dynamic types in WPF using IronPython and DLR


What would be covered in this post?

- Creating classes in IronPython and comparing with C# classes. Adding public properties in IronPython class.

- Using classes/interfaces in IronPython class which is in the hosting assembly. For example, if the application hosting the DLR and executing the IronPython class at runtime has an interface IDyn, this article covers on how to use that interface within the Python script.

- Accessing the dynamic type created in the IronPython and making an instance of it.

- Finally, binding the dynamic type to WPF controls.

Pre-Requisites

Before, we get started, I recommend you go through the article I posted previously which shows how to use the IronPython engine within .NET applications. I will be using the same PythonEngine class with a new method added as shown below.

   1: public T GetDynamicType<T>(string scriptFile)


   2:         {


   3:             try


   4:             {


   5:                 var source = engine.CreateScriptSourceFromFile(scriptFile);


   6:                 source.Execute(scope);


   7:                 var SomeClass = engine.GetVariable(scope, "DynObject");


   8:                 var t = engine.Operations.Create(SomeClass);


   9:                 return (T)t;


  10:             }


  11:             catch (Exception ex)


  12:             {


  13:                 Debug.WriteLine(ex);


  14:             }


  15:             return default(T);


  16:         }





In the above method, we see that the engine (refer to the previous article, it would be useful if you want to understand it) creates an instance of ScriptSource object from the script file. Then we pass the current ScriptScope we created earlier and pass it to the Execute() method which would make the class be available to be used. Line 7 shows using GetVariable() on the engine and we pass the scope (which knows where the class we are looking for is) and also the name of the class, which is DynObject. Once we have the class (I assume its the IL, or at least i understood it that way), we use the Create() method on the engine passing the class which would return instance of that class. Then we simply return the instance by casting it to type T (a generic, so we use default(T) which is like null but for generics). So this method basically executes a python script and returns an instance of the class we specify. May be I would just refactor the method to make it look like shown below and which allows the method itself to be more generic than it is.





   1: public T GetDynamicType<T>(string scriptFile, string className)


   2:         {


   3:             try


   4:             {


   5:                 var source = engine.CreateScriptSourceFromFile(scriptFile);


   6:                 source.Execute(scope);


   7:                 var SomeClass = engine.GetVariable(scope, className);


   8:                 return (T)engine.Operations.Create(SomeClass);


   9:             }


  10:             catch (Exception ex)


  11:             {


  12:                 Debug.WriteLine(ex);


  13:             }


  14:             return default(T);


  15:         }




The method is used as shown below.





   1: object o = pe.GetDynamicType<Object>("DynObject.py", "DynObject");




Creating IronPython class with public properties



Consider the C# class shown below.





   1: public class DynObject : IDyn 


   2: {


   3:    public string Name { get; set; }


   4:    public string Age  { get; set; }


   5: }




The class is named DynObject which implements the interface IDyn and which has two public properties Name and Age. The python code for the same is shown below.





   1: class DynObject(IDyn):


   2:   


   3:   def __init__(self): # this is the constructor


   4:     self._name = None # initialized to NULL


   5:     self._age = None # initialized to NULL


   6:     


   7:   def __getName(self):


   8:     return self._name


   9:     


  10:   def __getAge(self):


  11:     return self._age  


  12:     


  13:   def __setName(self,value):


  14:     self._name = value


  15:     


  16:   def __setAge(self,value):


  17:     self._age = value


  18:     


  19:   Name = property( # this is the Name property. use "property"


  20:      fget= __getName, # getter


  21:      fset= __setName  # setter


  22:   )


  23:   


  24:   Age = property(


  25:      fget=__getAge,


  26:      fset=__setAge


  27:   )






The statement followed by # are the comments. Hope they help you relate the classes properly and with that one should be able to write basic classes on their own.



Importing Namespaces in IronPython and setting up default namespaces



Now the problem with the above script when you Execute() it in the method I first discussed is that it would complain about not knowing what IDyn is. So basically we do an import the namespace. But the interface IDyn was created inside a namspace “Dynamics” which is the same application that is executing the python script. Had it been a ‘Debug’ then one could have done the following to import System.Diagnostics.*





   1: import clr


   2: clr.AddReference("System.Diagnostics")


   3: from System.Diagnostics import *


   4: # from System.Diagnostics import Debug 


   5: # if you just want to import Debug class




But the issue I has was to know what assembly should I mention in the AddReference and I was wondering if there was a good way to import some default namespaces into the engine without having the user mention them manually. For that reason we have ScriptRuntime.LoadAssembly(Assembly asmToLoad) which can be used to add reference to assemblies of your choice. In order to support this, I felt that it would be a good idea to load the assemblies before you even generate the ScriptSource object from the script file. For this reason, I modified the PythonEngine (the wrapper I talked about in my previous post on IronPython) constructor and now it looks like as shown.





   1: private PythonEngine(){


   2:             engine = Python.CreateEngine(new Dictionary<string, object>() { 


   3:                 { 


   4:                 "DivisionOptions", PythonDivisionOptions.New 


   5:                 } //using Dictionary Initializer ;)    


   6:             });


   7:             var runtime = engine.Runtime;


   8:             //Load the assemblies into the engine's runtime.


   9:             runtime.LoadAssembly(typeof(Debug).Assembly);


  10:             runtime.LoadAssembly(typeof(string).Assembly);


  11:             runtime.LoadAssembly(typeof(IDyn).Assembly);


  12:             scope = engine.CreateScope();


  13: }




What should be of interest to import default assemblies into the system are in lines 7-11. Then on we can move on to add the imports to the python file as shown. No more AddReference() required.





   1: from System.Diagnostics import Debug


   2: from Dynamics import IDyn


   3:  


   4: class DynObject(IDyn):


   5:   


   6:   def __init__(self):


   7:     Debug.WriteLine("here")


   8:     self._name = "buddi"


   9:     self._age = "22"


  10:     


  11:   def __getName(self):


  12:     return self._name


  13:     


  14:   def __getAge(self):


  15:     return self._age  


  16:     


  17:   def __setName(self,value):


  18:     self._name = value


  19:     


  20:   def __setAge(self,value):


  21:     self._age = value


  22:     


  23:   Name = property(


  24:      fget= __getName,


  25:      fset= __setName


  26:   )


  27:   


  28:   Age = property(


  29:      fget=__getAge,


  30:      fset=__setAge


  31:   )




Note that you still need to do the “from  NAMESPACE import CLASSNAME/*”.



How about integrating the python code in WPF?



The IDyn was just an empty interface which was added to explain how one could use .NET interfaces within IronPython classes. So back to the WPF application, which is simple and straight forward. The XAML code for the Window is shown below.





   1: <Window x:Class="DynamicBinding.Window1"


   2:     xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"


   3:     xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"


   4:     Title="Window1" Height="300" Width="300">


   5:     <StackPanel>


   6:         <TextBlock Text="{Binding Name}" Background="Red" Margin="5" Height="30"/>


   7:         <TextBlock Text="{Binding Age}" Background="Blue" Margin="5" Height="30"/>


   8:     </StackPanel>


   9: </Window>




From the bindings shown above, the Name property is looked for in the DataContext of TextBlock, StackPanel or Window. So in the code-behind we use our PythonEngine to obtain an instance of the dynamic type we created and set the DataContext of the window to the instance we obtained. The code behind is shown below.





   1: using System;


   2: using System.Windows;


   3: using Dynamics;


   4:  


   5: namespace DynamicBinding


   6: {


   7:     /// <summary>


   8:     /// Interaction logic for Window1.xaml


   9:     /// </summary>


  10:     public partial class Window1 : Window


  11:     {


  12:         private PythonEngine pe = PythonEngine.Engine;


  13:  


  14:         public Window1()


  15:         {


  16:             InitializeComponent();


  17:             object o = pe.GetDynamicType<Object>("DynObject.py", "DynObject");


  18:             this.DataContext = o;


  19:         }


  20:     }


  21: }




This ability to bind WPF controls to dynamic types would be a great feature if you would like to generate UI based on the user controlled params. This would be even easier with the dynamic support in C# 4.0 which I hope to see another release during the MIX 2009.



Next I would like to work on generate the python classes from XML which would be provided by the user and even the XAML would be generated from the XML. This would be the basis for my reporting suite which would be powered by Silverlight, DLR and WCF.

Wednesday, February 18, 2009

Make your .NET application extendible using DLR : IronPython

There are different ways to define this – extensibility in applications, adding macro capability to your applications, providing add-in like facility in .NET applications. For me, all of this mean the same – ability to modify your application (data or the UI) at runtime by means of scripts. Like I mentioned in the previous post, I wanted to dig into DLR or Dynamic Language Runtime which is basically a framework that could be used to provide dynamic languages support on the .NET platform. Even though I am not much interested in giving up C# for dynamic languages like Python or Ruby, I was very much keen to add facility to make my applications extensible using these languages (to tell you the truth, I am more comfortable with JavaScript or Groovy than any of these).

Anyway, I worked on an application which displays a datagrid and one could modify that the data being displayed using Python script at runtime. The way I did it was to host IronPython inside C# using the DLR framework.

The article would be a detailed step by step procedure on each and every piece of code that has been written so that it helps the readers get familiar with the following

  1. Using WPF Datagrid and databinding. Also shows Dependency Properties.
  2. How to refresh WPF Datagrid items.
  3. Hosting IronPython inside C# applications.
  4. Passing data to and fro between C# and IronPython script.

Shown below are the screenshots for the application that we will be looking through.

image image image
Application when started First time execution  Subsequent executions

Figure 1 shows the application when it first started. The data that appears in the grid has been generated on the fly for simplicity. The application provides two buttons whose titles are descriptive enough – one for running the script that modifies the data and the other for launching the script in notepad.

Figure 2 is when first the script is applied. Notice the amount of time it takes (2.1 seconds). It looks slow and the majority of the time, I guess, is consumed for initialization of the IronPython engine in the DLR. The python script that was executed looks at the age of the person and classifies them accordingly. Notice that the persons over 60 have their address as “senior citizen at work”. (Address is a pretty lame field for description, anyway I am lazy now to change it).

Figure 3 shows two subsequent executions which has the same effect as earlier but the script has been modified to change the description from “senior citizen at work” to “seniors”. This was to show that the script could be modified outside the application and yet it would be faster since the engine is already initialized (the way I wrote the evaluation). It takes .18 seconds to operate on 50 K tuples which I personally think is very good.

I hope you now have a clear picture on what we are trying to achieve. To further enhance your understanding, let us look at the python script. Note that the application was designed in such a way that it passes the list of persons using the variable “Context” and expects the modified list of person in “Output”.

   1: import clr


   2: clr.AddReference('System')


   3: from System import *


   4: from System.Diagnostics import Debug


   5:  


   6: for person in Context:


   7:   age = person.Age


   8:   if age >= 60:


   9:      person.Address = "Seniors"  


  10:   elif age > 25 and age < 60:


  11:      person.Address = "Adult"


  12:   elif age <=25 and age > 18:


  13:      person.Address = "Growing"


  14:   else:


  15:      person.Address = "Minor"


  16: Output = Context




The first 4 lines brings in the .NET assemblies and I import System.Diagnostics.Debug class just to show how to import .NET classes into IronPython code. The python script is pretty much readable (and you would not believe how long it took me to figure out how to write that).



Anyway lets get started by creating a new WPF project. Add the references to IronPython and DLR assemblies. Also add reference to WPF Toolkit assembly. The references should look like shown below.



image



XAML for the application.





   1: <Window x:Class="DynamicGrid.Window1"


   2:     xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"


   3:     xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"


   4:     Title="Click On Apply Filter to apply the Python Filter" Height="600" Width="300" 


   5:         WindowStyle="ThreeDBorderWindow"


   6:         xmlns:my="http://schemas.microsoft.com/wpf/2008/toolkit">


   7:     <Grid>


   8:         <Grid.RowDefinitions>


   9:             <RowDefinition Height="*"/>


  10:             <RowDefinition Height="35"/>


  11:             <RowDefinition Height="70"/>


  12:         </Grid.RowDefinitions>


  13:         <my:DataGrid Name="myGrid" ItemsSource="{Binding DataSource}" CanUserAddRows="False" CanUserResizeColumns="False">


  14:         </my:DataGrid>


  15:         <StackPanel Orientation="Horizontal"  Grid.Row="1">


  16:             <Button Name="ApplyRubyFilter" Content="Apply Python Filter" Click="ApplyRubyFilter_Click" Margin="5"


  17:                />


  18:             <Button Name="OpenScript" Content="Edit Python Script/Filter" Click="Edit_Script_Filter" Margin="5"/>


  19:         </StackPanel>


  20:         <ScrollViewer Grid.Row="2">


  21:             <TextBlock Name="Status" Text=""/>


  22:         </ScrollViewer>


  23:     </Grid>


  24: </Window>




Line 6 imports WPF toolkit which can then be used within the xaml file. Line 13/14 shows WPF datagrid being added to the first row of the container grid. Notice that it is bound to DataSource property. So it looks for this property in the datacontext of itself or the parent containers. For this, I added a dependency property with name “DataSource” in the code behind of the current window( which is Window1.xaml.cs. Shown below is the code for the property as well as the constructor.





   1: public IEnumerable DataSource


   2:        {


   3:            get { return (IEnumerable)GetValue(DataSourceProperty); }


   4:            set { SetValue(DataSourceProperty, value); }


   5:        }


   6:  


   7:        // Using a DependencyProperty as the backing store for DataSource.  This enables animation, styling, binding, etc...


   8:        public static readonly DependencyProperty DataSourceProperty =


   9:            DependencyProperty.Register("DataSource", typeof(IEnumerable), typeof(Window1), new UIPropertyMetadata(null));


  10:  


  11:        private IEnumerable<Person> _tempData;


  12:  


  13:        public Window1()


  14:        {


  15:            InitializeComponent();


  16:            this.DataContext = this;


  17:            _tempData = GenerateData();


  18:            myGrid.AutoGenerateColumns = true;


  19:            UpdateDataSource(_tempData);


  20:        }




For the dependency property, I usually use Visual Studio code snippet (Ctrl+K,X, Then NetFX30).



Now talking about the constructor, line 16 sets the DataContext of the window to itself. So the grid would finally find DataSource property in here. Note that you could set DataContext within XAML as shown below.





   1: <Window x:Class="DynamicGrid.Window1"


   2:     xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"


   3:     xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"


   4:     Title="Click On Apply Filter to apply the Python Filter" Height="600" Width="300" 


   5:         WindowStyle="ThreeDBorderWindow"


   6:         xmlns:my="http://schemas.microsoft.com/wpf/2008/toolkit"


   7:         Name="myWindow" DataContext="{Binding ElementName=myWindow}">




Line 7 in the above code snippet would be of interest to set DataContext to the current Window class in XAML.



Anyway back to what the constructor was doing. It invokes GenerateData() which generates a list of Person objects and assigns it to _tempData. This variable is what we would be using to pass to the Python script. Code for the GenerateDatat() is shown below.





   1: public IEnumerable<Person> GenerateData()


   2:         {


   3:             var list = new List<Person>();


   4:             Random rand = new Random();


   5:             for (int i = 0; i < 50000; i++)


   6:             {


   7:                 var person = new Person()


   8:                 {


   9:                     Name = "Name " + i,


  10:                     Age = rand.Next(100),


  11:                     Address = string.Format("Address {0}", i * 10)


  12:                 };


  13:                 list.Add(person);


  14:             }


  15:             return list.AsEnumerable();


  16:         }




Finally the constructor invokes UpdateDataSource() passing the data that the grid has to bind to. As you can expect the UpdateDataSource method just sets the value on the DataSource and performs a refresh on the datagrid items. Since DataSource is a dependency property, the grid automatically loads the datasource.





   1: private void UpdateDataSource(IEnumerable<Person> source)


   2:         {


   3:             DataSource = source;


   4:             myGrid.Items.Refresh(); //refresh the grid manually.


   5:         }




Now when the user clicks the “apply filter button”, the python script has to be executed. Code that does just that.





   1: private void ApplyRubyFilter_Click(object sender, RoutedEventArgs e)


   2:         {


   3:             var temp = ApplyIronPyFilter();


   4:             Debug.WriteLine(temp.GetType().Name);


   5:             if (temp != null)


   6:                 UpdateDataSource(temp.ToList());


   7:         }




For now, lets just say ApplyIronPyFilter does everything that it needs to execute the script on the _tempData and returns a new list of items. Ignore the line 4 which I added to verify what was being returned from the Python script. And if there is something in what is being returned, I update the datasource with that list.



Before we look into what ApplyIronPyFilter does, let us see the wrapper for the IronPython engine that I made to make things easy.





   1: using System;


   2: using System.Collections.Generic;


   3: using System.Diagnostics;


   4: using IronPython;


   5: using IronPython.Hosting;


   6: using Microsoft.Scripting.Hosting;


   7:  


   8: namespace DynamicGrid


   9: {


  10:     public class PythonEngine


  11:     {


  12:         private ScriptEngine engine;


  13:         private ScriptScope scope;


  14:  


  15:         private PythonEngine()


  16:         {


  17:             engine = Python.CreateEngine(new Dictionary<string, object>() { { "DivisionOptions", PythonDivisionOptions.New } });


  18:             scope = engine.CreateScope();


  19:         }


  20:  


  21:         private static PythonEngine _pe;


  22:  


  23:         public static PythonEngine Engine


  24:         {


  25:             get


  26:             {


  27:                 if (_pe == null)


  28:                     _pe = new PythonEngine();


  29:                 return _pe;


  30:             }


  31:         }


  32:  


  33:         public void SetVariable<T>(string name, T value)


  34:         {


  35:             scope.SetVariable(name, value);


  36:         }


  37:  


  38:         public T Evaluate<T>(T Context, string scriptFile)


  39:         {


  40:             this.SetVariable<T>("Context", Context);


  41:             try


  42:             {


  43:                 var source = engine.CreateScriptSourceFromFile(scriptFile);


  44:                 source.Execute(scope);


  45:             }


  46:             catch (Exception ex)


  47:             {


  48:                 Debug.WriteLine(ex);


  49:             }


  50:             //execute the script


  51:             //return value from "Output"


  52:             T value = default(T);


  53:             if (scope.TryGetVariable<T>("Output", out value))


  54:                 return (T)value;


  55:             return default(T);


  56:         }


  57:     }


  58: }




I am not going into details for each and every line of code that I am writing (may be sometime later).  But I would like to talk a little about Evaluate method which I think is important. The first parameter it takes would be the input for the text and as you can see in Line 40, it is being set as a Python variable with name “Context”. Similarly line 53 attempts to read a variable “Output” using TryGetVariable method on the “scope” object which was created in the constructor of the PythonEngine class. Lines 41-49 shows on how to load the script from a file and execute it. I would recommend you to look at the whole 58 lines of code for the PythonEngine since it is simple and yet important. At least you are required to look at the constructor and the SetVariable,Evaluate methods.



Finally, we come to the ApplyIronPyFilter method which as you can expect would use the PythonEngine#Evaluate() method passing the context as _tempData and path to the python script file that would be executed. The method then captures the return value from the Evaluate method and returns it back. Look at the source code.





   1: private IEnumerable<Person> ApplyIronPyFilter()


   2:         {


   3:             var sw = Stopwatch.StartNew();


   4:             try


   5:             {


   6:                 var engine = PythonEngine.Engine;


   7:                 return engine.Evaluate<IEnumerable<Person>>(_tempData, @"test.py") ;


   8:             }


   9:             finally


  10:             {


  11:                 sw.Stop();


  12:                 Status.Text += ("Time taken for Python Script : " + sw.ElapsedMilliseconds + " milliseconds \n");


  13:             }


  14:         }




Also shown is code on how to time the execution which is set to the Text property of the “Status” which is a textblock (look at the XAML). Looks like we are done with the most important pieces of source code that makes this application extendible.



I personally think it would be really cool if we have a IronGroovy kind of runtime. Having worked on Groovy for fairly complex applications (during my Masters), I am a huge fan of Groovy.



I totally had fun doing research on making this application work. I hope you find all the information you need to get started embedding DLR into your applications. Please let me know if you have any specific questions.



I got the book “Data Driven Services with Silverlight 2 by John Papa” which looks like book. Anyway I expected it to be dedicated to WCF driven data applications in Silverlight but that should be fine. I think the book would be very useful to understand more about data-driven silverlight applications, just that the source of data differs. I am actually looking for a good WCF book (should be comparable in good-ness with Programming WPF by Chris Sells). If you know of any good one, drop a comment.