Understanding the Relationship between Dynamic and Static Languages
Something that most developers fail to consider is that, at some point, all languages generate the same thing — machine code. Without machine code, the software doesn’t execute. Your computer cares nothing at all about the idiosyncrasies of human language and it doesn’t care about communicating with you at all. Computers are quite selfish when you think about it. The circuitry that makes up your computer relies on software to change the position of switches — trillions of them in some cases. So computers use machine code and only machine code; languages are for humans.
When it comes to dynamic and static languages, it’s the way that humans view the languages that make them useful. A dynamic language offers the developer freedom of choice, call it the creative solution. A static language offers a reliable and stable paradigm — call it the comfort solution, the one that everyone’s used. How you feel about the languages partly affects your use of them. In the end, both dynamic and static language output ends up as machine code. Dynamic and static languages end up being tools that help you create applications faster and with fewer errors. If you really wanted to do so, you could write any application today using assembler (a low-level language just above machine code, see http://www.bing.com/reference/semhtml/Assembly_language for more information), but assembler is hardly the correct tool any longer — humans need a better tool to put applications together. The point is that you should use the tool that works best for a particular development process and not think that the tool is doing anything for your computer.
Anytime you use multiple languages, you must consider issues that have nothing to do with the dynamic or static nature of that language. For example, you must consider the data types that the languages support and provide a method for marshaling data from one language to the other. In fact, marshaling data is an important element in many areas of coding. If you want to communicate with the Win32 API from a .NET-managed language such as C# or Visual Basic.NET, you must marshal the data between the two environments. It’s important not to confuse communication and infrastructure requirements with differences between dynamic and static languages. Many resources you find do confuse these issues, which makes it hard for anyone to truly understand how dynamic and static languages differ.
Before you can use IronPython from other languages, it’s important to consider the way in which IronPython performs tasks. When an IronPython session starts, nothing exists — the environment begins with an empty slate. You’ve discovered throughout this book that IronPython calls upon certain script files as it starts to configure the environment automatically. These configuration tasks aren’t part of the startup; they are part of the configuration — something that occurs after the startup. The dynamic nature of IronPython means that all activity begins and ends with adding, changing, and removing environment features. There aren’t any compiled bits that you can examine statically. Everything in IronPython is dynamic.
When a static language such as C# or Visual Basic.NET attempts to access IronPython, it must accommodate the constant change. If you got nothing else out of Chapter 14 but this one fact, then the chapter was worth reading. In order to do this, C# and Visual Basic.NET rely upon events because they can’t actually accommodate change as part of the language. An event signals a change — an IronPython application has modified a class to contain a new method or property. It isn’t just the idea that the output or value has changed, but the method or property itself is new. In some cases, C# or Visual Basic.NET will also need to deal with the situation where a method or property simply goes away as well. The underlying mechanism of events, delegates, and caches is inspired and all but invisible, but to be successful at using the languages together, you must know they’re present.
The differences between dynamic and static languages go further than simply not knowing what code will execute next in a dynamic language. There’s also the matter of data typing. A static language assigns a type to the data it manages, which means that the compiler can make assumptions about the data and optimize access to it. A dynamic language also assigns types to the data it manages, but only does so at run time and even then the data type can change. Now, consider how this changeability complicates the matter of marshaling data from one language to the other. Because the data no longer has a stable type, the marshaling code can’t assume anything about it and must constantly check type to ensure the data it marshals appears in the right form in the target language.
The difference between dynamic and static languages, at least from a programming perspective, comes down to flexible coding and data typing. Everything else you may have heard either relates to differences between any two languages (such as the need to marshal data) or the political drama of which tool works best. This book won’t endeavor to tell you what tool to use. Certainly, I don’t tell anyone that a hammer works best for driving screws or that screwdrivers make wonderful ice picks (not that I believe either of these statements myself). The tool you use for a particular task is the one you can use best or the one called for by a particular job requirement. The point of this chapter and the rest of the book is to demonstrate that dynamic and static languages can work together successfully and in more than one way. The tool you use is up to you.
Creating an Externally Accessible IronPython Module
The first requirement for building an application that allows external access is to create the IronPython script you want to use. Ideally, this script will contain code that is fully debugged. You also want to test the code before you try to use it within C# or Visual Basic.NET. The following sections provide you with the techniques you use to create an IronPython script that you access from C# or Visual Basic .NET.
Considering Requirements for Externally Accessible Modules
The mistake that many developers will make is to think they must do something special in IronPython to make the code accessible. What you really need to do is create an IronPython script using the same techniques as always, and then test it directly. After you test the script using IronPython code, work with the target static language to gain the required access. This pretesting process is important to ensure that you aren’t fighting with a bad script in addition to potential problems marshaling data or interacting with methods that change.
Creating the IronPython Script
The IronPython script used for this example is quite simple in approach. All that the example call really does is add two numbers together. You could perform the task with far less code, but the point of this class is to demonstrate access techniques, so it’s purposely simple. Listing 15-1 shows the external module code and the code used to test it. As previously mentioned, testing your IronPython script is essential if you want the application to work properly.
Listin g 15-1: A test IronPython class for use in the examples
[code]
# The class you want to access externally.
class DoCalculations():
# A method within the class that adds two numbers.
def DoAdd(self, First, Second):
# Provide a result.
return First + Second
# A test suite in IronPython.
def __test__():
# Create the object.
MyCalc = DoCalculations()
# Perform the test.
print MyCalc.DoAdd(5, 10)
# Pause after the test session.
raw_input(‘nPress any key to continue…’)
# Execute the test.
# Comment this call out when you finish testing the code.
__test__()
[/code]
The class used for this example is DoCalculations(). It contains a single method, DoAdd(), that returns the sum of two numbers, First and Second. Overall, the class is simple.
The TestClass.py file also contains a __test__() function. This function creates an instance of DoCalculations(), MyCalc. It then prints the result of calling the DoAdd() method with values of 5 and 10. The example waits until you press Enter to exit.
In __main__(), you see a call to __test__(). You can execute the example at the command line, as shown in Figure 15-1. Make sure you use the –D command line switch to place the interpreter in debug mode. You could also open IPY.EXE interactively, load the file, and execute it inside the interpreter. When you know that the code works properly, be sure to comment out the call to __test__() in __main__().
Accessing the Module from C#
Now that you have an external module to use, you’ll probably want to access from an application. This section considers the requirements for accessing IronPython from C#. Don’t worry; the section “Accessing the Module from Visual Basic.NET” later in this chapter discusses access from Visual Basic. NET as well. If you follow these steps, you’ll find that access is relatively straightforward, even if it does get a bit convoluted at times. Microsoft promises the future versions of C# will make dynamic language access even easier.
Adding the Required C# References
Any application you create requires access to the dynamic language assemblies. The IronPython assemblies appear in the Program FilesIronPython 2.6 folder on your machine. Right-click References and choose Add Reference from the content menu to display the Add Reference dialog box. Select the Browse tab. In most cases, you only need the three DLLs shown in Figure 15-2 to access any IronPython script. (You may also need to add the IronPython .Modules.DLL file to the list in some cases.)
Select the assemblies you require by Ctrlclicking them in the Add Reference dialog box. Click OK when you’re finished. You’ll see the assemblies added to the References folder in Solution Explorer.
Adding the Required References to the Host Language
You can perform a multitude of tasks with IronPython. In fact, later chapters in the book show how to perform tasks such as testing your static application code. IronPython really is quite flexible. However, most people will start by executing external scripts and only need a few of the namespaces in the IronPython assemblies to do it. The following using statements provide everything needed to execute and manage most IronPython scripts.
[code]
using System;
using IronPython.Hosting;
using IronPython.Runtime;
using Microsoft.Scripting.Hosting;
[/code]
Understanding the Use of ScriptEngine
You have many options for working with IronPython scripts. This first example takes an approach that works fine for Visual Studio 2008 developers, as well as those using Visual Studio 2010. It doesn’t require anything fancy and it works reliably for most scripts. Ease and flexibility concerns aside, this isn’t the shortest technique for working with IronPython scripts. This is the Method1 approach to working with IronPython scripts — the technique that nearly everyone can use and it appears in Listing 15-2.
Listin g 15-2: Using the script engine to access the script
[code]
static void Main(string[] args)
{
// Create an engine to access IronPython.
ScriptEngine Eng = Python.CreateEngine();
// Describe where to load the script.
ScriptSource Source = Eng.CreateScriptSourceFromFile(“TestClass.py”);
// Obtain the default scope for executing the script.
ScriptScope Scope = Eng.CreateScope();
// Create an object for performing tasks with the script.
ObjectOperations Ops = Eng.CreateOperations();
// Create the class object.
Source.Execute(Scope);
// Obtain the class object.
Object CalcClass = Scope.GetVariable(“DoCalculations”);
// Create an instance of the class.
Object CalcObj = Ops.Invoke(CalcClass);
// Get the method you want to use from the class instance.
Object AddMe = Ops.GetMember(CalcObj, “DoAdd”);
// Perform the add.
Int32 Result = (Int32)Ops.Invoke(AddMe, 5, 10);
// Display the result.
Console.WriteLine(“5 + 10 = {0}“, Result);
// Pause after running the test.
Console.WriteLine(“rnPress any key when ready…”);
Console.ReadKey();
}
[/code]
Now that you have access to Eng, you can use it to perform various tasks. For example, you must tell Eng what scope to use when executing code, so the example creates a ScriptScope object, Scope. In order to perform tasks, you must also have an ObjectOperations object, Ops. The example uses the defaults provided for each of these objects. However, in a production application, you might decide to change some properties to make the application execute faster or with better security.
At this point, you can execute the script. The act of executing the script using Source.Execute() loads the script into memory and compiles it in a form that the static application can use. The Source.Execute() method associates Scope with the execution environment. At this point, the parameters for executing the script are set in stone — you can’t change them.
The script is in memory, but you can’t access any of its features just yet. The script contains a DoCalculations class that you access by calling Scope.GetVariable() to create CalcObj. The code gains access to the class by creating an instance of it, CalcObj, using Ops.Invoke(). At this point, CalcObj contains an instance of DoCalculations() in the IronPython module, but you can’t use it directly. Remember that you must marshal data between C# and IronPython. In addition, C# has to have a way to deal with the potential changes in the IronPython script.
This seems like a lot of work just to gain access to DoAdd, but you can finally use AddMe to perform the addition. A call to Ops.Invoke() with AddMe and the arguments you want to use performs all of the required marshaling for you. You must coerce the output to an Int32 (something that C# understands). Finally, the application outputs the result, as shown in Figure 15-3.
Using the dynamic Keyword
One of the new ways in which you can access IronPython in C# 4.0 is to use the dynamic keyword. This keyword makes it possible for you to cut out a lot of the code shown in Listing 15-2 to perform tasks with IronPython. It’s still not perfect, but you’ll do a lot less work. Listing 15-3 shows a short example that accesses the __test__() function found in Listing 15-1.
Listin g 15-3: Accessing IronPython using the dynamic keyword
[code]
static void Main(string[] args)
{
// Obtain the runtime.
var IPY = Python.CreateRuntime();
// Create a dynamic object containing the script.
dynamic TestPy = IPY.UseFile(“TestClass.py”);
// Execute the __test__() method.
TestPy.__test__();
}
[/code]
The next step is to load the script. The dynamic type, TestPy, contains all the features of the TestClass.py script after you load it using IPY.UseFile(). Figure 15-4 shows how TestPy appears after the script loads. Notice that the Locals window correctly identifies all the IronPython types in the file. (Visual Basic.NET developers will have to wait for an update).
In this case, the example calls the __test__() function. This function outputs the same information shown in Figure 15-1.
Working with the App.CONFIG File
In some cases, you might want to configure your application using an App.CONFIG file. Using the App.CONFIG file tends to ensure that your application works better between development machines. In addition, using the App.CONFIG file can make it easier to work with DLR using older versions of Visual Studio. Most important of all, using the App.CONFIG file ensures that anyone working with the application uses the correct version of the DLLs so that any DLL differences aren’t a problem.
Your project won’t contain an App.CONFIG file at the outset. To add this file, right-click the project entry in Solution Explorer and choose Add ➪ New Item from the context menu. You see the Add New Item dialog box shown in Figure 15-5. Highlight the Application Configuration File entry as shown and click Add. Visual Studio automatically opens the file for you.
The App.CONFIG file contains entries that describe the Microsoft scripting configuration. In most cases, you begin by defining a <section> element, which describes a <microsoft.scripting> element. The <microsoft.scripting> element contains a list of languages you want to use in a <languages> element, as shown in Listing 15-4.
Listin g 15-4: Defining the App.CONFIG file content
[code]
<?xml version=”1.0” encoding=”utf-8” ?>
<configuration>
<configSections>
<section name=”microsoft.scripting”
type=”Microsoft.Scripting.Hosting.Configuration.Section,
Microsoft.Scripting, Version=1.0.0.0, Culture=neutral,
PublicKeyToken=31bf3856ad364e35”
requirePermission=”false” />
</configSections>
<microsoft.scripting>
<languages>
<language names=”IronPython,Python,py”
extensions=”.py”
displayName=”IronPython 2.0 Beta”
type=”IronPython.Runtime.PythonContext,IronPython,
Version=2.6.10920.0, Culture=neutral,
PublicKeyToken=31bf3856ad364e35” />
</languages>
</microsoft.scripting>
</configuration>
[/code]
The <section> element includes attributes for name, type, and requirePermission. The type attribute should appear on one line, even though it appears on multiple lines in the book. This attribute describes the Microsoft.Scripting.DLL attributes. Especially important is the Version and PublicKeyToken entries.
The <microsoft.scripting> element contains a <languages> element at a minimum. Within the <languages> element you find individual <language> elements that are descriptions of the languages you want to use in your application.
For this example, you create a <language> element for IronPython that starts with a names attribute. It’s important to define all the names you plan to use to access the language — the example defines three of them. The extensions attribute describes the file extensions associated with the language, which is .py in this case. The displayName attribute simply tells how to display the language. Finally, the type attribute contains a description of the IronPython.DLL file. As with the type element for Microsoft.Scripting.DLL. Again, you need to exercise special care with the Version and PublicKeyToken entries.
Now that you have the App.CONFIG file created, it’s time to look at the application code. Listing 15-5 contains the source for this example.
Listin g 15-5: Using the App.CONFIG file in an application
[code]
static void Main(string[] args)
{
// Read the configuration information from App.CONFIG.
ScriptRuntimeSetup srs = ScriptRuntimeSetup.ReadConfiguration();
// Create a ScriptRuntime object from the configuration
// information.
ScriptRuntime runtime = new ScriptRuntime(srs);
// Create an engine to access IronPython.
ScriptEngine Eng = runtime.GetEngine(“Python”);
// Describe where to load the script.
ScriptSource Source = Eng.CreateScriptSourceFromFile(“TestClass.py”);
// Obtain the default scope for executing the script.
ScriptScope Scope = Eng.CreateScope();
// Create an object for performing tasks with the script.
ObjectOperations Ops = Eng.CreateOperations();
// Create the class object.
Source.Execute(Scope);
// Obtain the class object.
Object CalcClass = Scope.GetVariable(“DoCalculations”);
// Create an instance of the class.
Object CalcObj = Ops.Invoke(CalcClass);
// Get the method you want to use from the class instance.
Object AddMe = Ops.GetMember(CalcObj, “DoAdd”);
// Perform the add.
Int32 Result = (Int32)Ops.Invoke(AddMe, 5, 10);
// Display the result.
Console.WriteLine(“5 + 10 = {0}“, Result);
// Pause after running the test.
Console.WriteLine(“rnPress any key when ready…”);
Console.ReadKey();
}
[/code]
The biggest difference between this example and the one shown in Listing 15-2 is that you don’t create the script engine immediately. Rather, the code begins by reading the configuration from the App.CONFIG file using ScriptRuntimeSetup.ReadConfiguration(). This information appears in srs and is used to create a ScriptRuntime object, runtime.
At this point, the code finally creates the ScriptEngine, Eng, as in the previous example. However, instead of using Python.CreateEngine(), this example relies on the runtime.GetEngine() method. For this example, the result is the same, except that you’ve had better control over how the ScriptEngine is created, which is the entire point of the example — exercising control over the IronPython environment. The rest of the example works the same as the example shown in Listing 15-2. The output is the same, as shown in Figure 15-3.
Accessing the Module from Visual Basic.NET
You might get the idea from the lack of Visual Basic.NET examples online that Microsoft has somehow forgotten Visual Basic.NET when it comes to DLR. Surprise! Just because the examples are nowhere to be seen (send me an e‑mail at JMueller@mwt.net if you find a stash of Visual Basic.NET examples somewhere) doesn’t mean that you can’t work with IronPython from Visual Basic. In fact, the requirements for working with Visual Basic.NET are much the same as those for working with C#, as shown in the following sections.
Adding the Required Visual Basic.NET References
Visual Basic requires the same DLL references as C# does to work with IronPython. Figure 15-2 shows the assemblies you should add to your application to make it work properly. In this case, you right-click the project entry and choose Add Reference from the context menu to display an Add Reference dialog box similar to the one shown in Figure 15-2. Select the Browse tab and add the IronPython assemblies shown in Figure 15-2 by Ctrl-clicking on each of the assembly entries. Click OK. Visual Basic will add the references, but you won’t see them in Solution Explorer unless you click Show All Files at the top of the Solution Explorer window.
As with C#, you need to add some Imports statements to your code to access the various IronPython assemblies with ease. Most applications will require the following Imports statements at a minimum.
[code]
Imports System
Imports IronPython.Hosting
Imports IronPython.Runtime
Imports Microsoft.Scripting.Hosting
[/code]
Creating the Visual Basic.NET Code
As with all the other examples, you shouldn’t let the IronPython example dictate what you do in your own applications. You can obtain full access to any IronPython script from Visual Basic.NET and fully use every feature it provides.
Accessing IronPython scripts from Visual Basic.NET is much the same as accessing them from C# using the ScriptEngine object. Listing 15-6 shows the code you need to access the IronPython script used for all the examples
Listin g 15-6: Accessing IronPython from Visual Basic.NET
[code]
Sub Main()
‘ Create an engine to access IronPython.
Dim Eng As ScriptEngine = Python.CreateEngine()
‘ Describe where to load the script.
Dim Source As ScriptSource = Eng.CreateScriptSourceFromFile(“TestClass.py”)
‘ Obtain the default scope for executing the script.
Dim Scope As ScriptScope = Eng.CreateScope()
‘ Create an object for performing tasks with the script.
Dim Ops As ObjectOperations = Eng.CreateOperations()
‘ Create the class object.
Source.Execute(Scope)
‘ Obtain the class object.
Dim CalcClass As Object = Scope.GetVariable(“DoCalculations”)
‘ Create an instance of the class.
Dim CalcObj As Object = Ops.Invoke(CalcClass)
‘ Get the method you want to use from the class instance.
Dim AddMe As Object = Ops.GetMember(CalcObj, “DoAdd”)
‘ Perform the add.
Dim Result As Int32 = Ops.Invoke(AddMe, 5, 10)
‘ Display the result.
Console.WriteLine(“5 + 10 = {0}“, Result)
‘ Pause after running the test.
Console.WriteLine(vbCrLf + “Press any key when ready…”)
Console.ReadKey()
End Sub
[/code]
As you can see from the listing, Visual Basic.NET code uses precisely the same process as C# does to access IronPython scripts. In fact, you should compare this listing to the content of Listing 15-2. The two examples are similar so that you can compare them. The output is also precisely the same. You’ll see the output shown in Figure 15-3 when you execute this example.
Developing Test Procedures for External Modules
Many developers are beginning to realize the benefits of extensive application testing. There are entire product categories devoted to the testing process now because testing is so important. Most, if not all, developer tools now include some idea of application testing with them. In short, you should have all the testing tools you need to test the static portion of your IronPython application.
Unfortunately, the testing tools might not work particularly well with the dynamic portion of the application. Creating a test that goes from the static portion of the application to the dynamic portion of the application is hard. (Consequently, you need to include a test harness with your dynamic code and perform thorough testing of the dynamic code before you use it with the static application. (When you think about a test harness, think about a horse, your application that has a harness added externally for testing purposes. You add the harness for testing and remove it for production work without modifying the application.) Listing 15-1 shows an example of how you might perform this task.
The test harness you create has to test everything, which is a daunting task to say the least. In addition, you need to expend extra effort to make the test harness error free — nothing would be worse than to chase an error through your code, only to find out that the error is in the test harness. At a minimum, your test harness should perform the following checks on your dynamic code:
- Outputs with good inputs
- Outputs with erroneous inputs
- Exception handling within methods
- Property value handling
- Exceptions that occur on public members that would normally be private
Of course, you want to check every method and property of every class within the dynamic code. To ensure you actually test everything, make sure you create a checklist to use to verify your test harness. Because IronPython isn’t compiled, you’ll find that you must manually perform some checks to ensure the code works precisely as planned, but use as much automation as possible.
Debugging the External Module
Debugging isn’t hard, but it also isn’t as straightforward as you might think when working with IronPython. The debugger won’t take you directly to an error. You can’t test variables using the debugger from within the static language. In short, you have to poke and prod the external script to discover what ails it. Fortunately, you do have three tools at your disposal for discovering errors.
- Exceptions
- print Statements
- An ErrorListener object
Let’s begin with the easiest of the three tools. The static language application won’t ignore outright errors in the script code. For example, you might have the following error in the script:
[code]
# Introduce an error.
print 1/0
[/code]
If your code has this error (and it really shouldn’t), you’ll see an exception dialog box like the one shown in Figure 15-6. Unfortunately, when you click View Detail, the content of the View Detail dialog box is nearly useless. The exception information won’t tell you where to find the error in your script. In fact, it may very well lead you on a wild goose chase that ends in frustration.
The name of the exception will provide clues as to where the error might exist, but you can’t confirm your suspicions without help. The only tool, besides vigorous script testing, is to include print statements such as these in your code.
[code]
# Display the values of First and Second.
print ‘Values in IronPython Script’
print ‘First = ‘, First
print ‘Second = ‘, Second
[/code]
When you run the script, you see the output shown in Figure 15-7. Most developers view print statements as a bit old school, but they do work if you use them correctly. Make sure you provide enough information to know where the script is failing to perform as expected. Even so, using print statements may feel a bit like wandering around in the dark, so you should place an emphasis on testing the script before you use it and after each change you make.
In some cases, you might make a small change to a script and it stops running completely — you might not see a script exception, just an indicator that something’s wrong because the application raises an unrelated exception. Syntax errors and other problems where the interpreter simply fails can cause the developer a lot of woe. For example, your application might have the following syntax error:
[code]
# Create a syntax error.
while True print ‘This is an error!’
[/code]
This code obviously won’t run. Because of the nature of the error, you might even pass it by while looking through your code. The answer to this problem is to create an ErrorListener class like the one shown in Listing 15-7.
Listin g 15-7: Create an ErrorListener to hear script semantic errors
[code]
class MyListener : ErrorListener
{
public override void ErrorReported(ScriptSource source,
string message,
SourceSpan span,
int errorCode,
Severity severity)
{
Console.WriteLine(“Script Error {0}: {1}“, errorCode, message);
Console.WriteLine(“Source: {0}“, source.GetCodeLine(span.Start.Line));
Console.WriteLine(“Severity: {0}“, severity.ToString());
}
}
[/code]
The ErrorListener contains just one method, ErrorReported(). This method can contain anything you need to diagnose errors. The example provides an adequate amount of information for most needs. However, you might decide to provide additional information based on the kind of script you’re using.
In order to use this approach, you must compile the script before you execute it. The compilation process must include the ErrorListener, as shown here.
[code]
// Compile the script.
Source.Compile(new MyListener());
[/code]
When you run the application now, you get some useful information about the syntax error, as shown in Figure 15-8.
