How to Write an Extension in Xtend for Xpand in Xtext

I ran into a problem in Xtext some weeks ago. I wanted to generate a serialization id for my generated Java classes. After some dumbfounded attempt to do so in Xpand, I realized that Xpand does not allow for an arbitrary Java algorithm to be interspersed in the (almost) declarative Xpand template language.

I asked the generous Xtext team for help, and it became clear that I would have to create an extension using Xtend. I’ve never done that before, but the Xtext team pointed me to the documentation. After having read the documentation forwards and backwards, I was still confused. I think the presumed knowledge did not quite fit my profile. To help the Xtext team and maybe others with the same background as I, I thought I'd share my experiences.

What I wanted to achieve

I want to use Xpand to generate some Java code that use Java serialization. One of the foundations for Java serialization is that you provide ‘long’-value. It doesn’t much matter what the number is, but it better change if you change the class. This is to ensure that previously serialized data no longer is compatible with the new Java code. Hence, when writing my code expansion template, I would have to generate a sensible number that would be different given two different input models.

The template looked something like this:

In Xpand you there is no way to inject arbitrary Java code (as say in JET or Velocity templates), however, you can achieve this in a clean way using Xtend.
Reading the Xtend documentation (see link to above), it was not clear to me exactly how to achieve this. In particular, I was not sure where to put the files and how to link everything.

What was clear was that it would be possible for me to extend Xpand such that I can now use a simple new function. That is, after I’m done, I should be able to write my template as this:

How to extend Xpand

Here is a list of facts that you may want to keep in mind as you read through the rest of this blog entry:

• Xtend allows you to extend Xpand in two different languages. You can use OCL (or a derivative of it). This is very simple and the preferred way. The other is to declare the extension in Xtend and implement it in Java. This is a bit trickier, but not too difficult after you’ve read this (I hope).
• The Xtend declaration or definitions are defined in files with extension ‘ext’. If you are using Xtext, you would want to place them in the generator project. You may put them in any subdirectory of the main source (default called ‘src’).
• If you implement part of your extensions in Java, you should also place the Java code in your generator project.

Step 1, define your Xtend file

The first step is to create a file that defines your extensions. You can place the Xtend file anywhere in the classpath defined when you run Xpand. The natural place to place the file(s) if you use Xtext would be in your ‘generator’ project src directory. In my case I placed them in ‘src/extensions’. The file must have the extension ‘ext’. In my case, I created a file called JavaSerialUID.ext.

Step 2, declare the content in the Xtend file

As mentioned earlier, there are two languages that you can use. This fact is clear in the official documentation, so I’ll not dwell with this much. I’ll just focus on creating a Java extension. My extension generates UID’s as mentioned earlier, and the content looks like this:

Notice that the content of the Xtend file for Java type extensions is really nothing but a simple mapping. It is basically stating that we want to extend the Xpand language with a new function called ‘generateSerialID’ that requires an sdol::Type (sdol is the name of the language I’ve created, and Type is an object type in that language). It is also mapping this new function in Xpand to an implemented function in Java. This code is placed in the package ‘com.inferdata.sdol.extensions’. The class name is ‘JavaUtil’. The method takes one argument, that of type ‘com.inferdata.sdol.Type’, that is we’re providing the fully qualified name of the type (e.g., if you wanted to pass a string, we would have to specify ‘java.lang.String’). Also note that we promise that the result of the invocation will be a string object.

An important little digression may be in order here. Notice that the declaration refers to my model object ‘Type’ in two different ways. It is using the Xtend language to refer to the Type in the declaration. That is, in Xtend my type is called ‘sdol::Type’. In Java, the same model object is being referred to as com.inferdata.sdol.Type. The Xtend reference is identical to the one we would use in Xpand, the Java reference is based on the generated EMF code.

Step 3. Create the Java implementation

Now we need to write the Java code we want executed when someone use our function.
My Java implementation for generating a serialization ID is not important and whatever algorithm you may need would obviously be different, but here is a simplified version of what I did (generated a hash code by xor’ing the names of all features):

The important thing to note here is that the class and method name maps EXACTLY to that declared in the Xtend file.

Using Xpand after the extension

After you have finished your extension, you can now use the new function from within Xpand. The Xpand editor will even help with auto-completion.
The first thing you need to do in your Xpand file is to import the newly created extension. You do this by declaring an «EXTENSION» element.

This declaration above has to be exact also. The ‘extensions’ is the directory where I placed my extension file. ‘JavaSerialUID’ is the name of the extension file (the file name is really ‘JavaSerialUID.ext’, but we drop the file extension).
Next we need to know one more thing… The extensions can be invoked two different ways. You may invoke the function in a pseudo object oriented fashion. That is we’ll write __OBJECT_REFERENCE__.expansionFunction(…), or we may use a more functional syntax, expansionFunction(__OBJECT_REFERENCE__, …). The variation is nothing but syntactic sugar, hence, you may pick whichever method you like.
In my code, Xpand code, the use of the Java extension looks something like this:
abstract public class Generated«this.name» {
private static final long serialVersionUID = «this.generateSerialID()»L;
…

One impressive feature of the Xpand editor that you may take advantage of when debugging your extension (and of course when using it), is that the editor will parse the declared extensions and help you auto-complete. E.g.:

In the screen capture above, I’ve just typed ‘«'.
In the example below I'm using the pseudo-object-oriented option using the this as the object receiver:

Conclusion

In this blog entry I’ve shown how one may create an extension to Xpand using Java. This is a last resort option when using Xpand and Xtend, but not an uncommon one. The trick to building an extension is:

• Create a Xtend file to expand the language
  • Must be in the class path where you use Xpand
  • A file with the extension ‘ext’
  • Place the file in any directory
• Declare a mapping between a new Xpand function and the Java function you want invoked
• Declaration of Xpand function in Xtend language
• Declaration of Java in Java language (use Java packages, Java types, etc)
• Use fully qualified names for model objects and Java implementations
• Implement the Java function
• Must be a static method
• Can take any number of parameters
• The first parameter may be used as this
• Must be placed in the classpath where you use Xpand
• Use the extension in your Xpand files
• Import the extension using «EXTENSION»
• Invoke the extension function using the pseudo-object-oriented or functional approach