codeslower.com Savor Your Code.

Syntax Coloring for Your Custom Mg Language with Intellipad

So, you've downloaded the Oslo SDK and you've created your own custom grammar using Mg, but there is one piece missing -- getting Intellipad to automatically provide error highlighting and syntax coloring for you. "Tree Preview Mode" will highlight text in its "DynamicParserMode" buffer, but there isn't a way to tell Intellipad to apply a specific Mg grammar to a specific file. This article will show you how create a custom "mode" for your specific language. You'll also learn a bit about adding Mg support to your Visual Studio projects.

The code for this project can be found at:

The project contains sample grammars and files from my previous post. The relevant portions to this article can be found in the ToDoPlugin directory. All the code is browseable at the links above. You can also download it as a zip file from GitHub -- just click the download link at the top of any page.

My example is based on sources found in the SDK. Under Bin\Intellipad\Sample Sources, you'll find the Microsoft.M.Grammar.IntellipadPlugin project. That sample implements "Tree Preview Mode" for interactively developing Mg grammars and is the source of much of my code.

Update

A new post with updates necessary for the January 2009 CTP is now available.

Overview

There are four steps we'll follow to create the custom mode for Intellipad:

  • Define a language using Mg.
  • Add meta-data to our language for syntax highlighting support.
  • Create a .NET assembly implementing our custom mode.
  • Add our custom mode to Intellipad.

Step 1 -- Define our language

For this step I'll be using todo4.mg. The syntax of the language can be inspected in the file -- I will not repeat it in full here. A sample file which uses the language is available as todo4.todo.

Step 2 -- Add meta-data

Meta-data can be added to an Mg language definition with attributes, much like C#. By attaching Classification attributes to various elements, we can tell Intellipad how to interpret those elements. For example, keywords can be specified like so:

@{Classification["Keyword"]}
token Task = "task";

Literals, comments, and other elements are also possible:

{Classification["Comment"]}
token Comment = "#" ^('\r' | '\n')*;

@{Classification["Literal"]}
token Digit = "0".."9";

@{Classification["String"]}
token SingleQuotedText = QuotedText('"');

Examples of these attributes can be found in the Mg and M specifications shipped with the SDK (in Samples\MGrammar\Languages). An article on the Intellipad team blog also details how to customize syntax coloring -- the relevant information is about halfway through.

Step 3 -- Create a .NET Assembly

This step is the most complicated. There are 3 major tasks to accomplish:

  • Create and modify a Visual Studio project for Mg.
  • Provide hooks between the parser for our language and Intellipad.
  • Create Python scripts for Intellipad to use for hooking up the custom mode.

The project and code found under the ToDoPlugin directory implements these steps. I recommend following along as you read.

Step 3.1 -- Create and Modify a Visual Studio Project

Start by creating a VS "Class Library" project. Add your Mg file and the following references to the project:

   PresentationCore
   PresentationFramework
   PresentationFramework.Aero
   System
   System.Core
   WindowsBase

Next, right-click the project in the Solution Explorer, select "Unload Project", then right-click again and select "Edit <project filename>". You need to add a few custom tasks in order to have Mg build a parser for your language and include it the final assembly. Under the first PropertyGroup tag, add these lines:

<MBinaries32Path Condition="'$(CsaExportPath)'!=''">$(CsaExportPath)\Compilers</MBinaries32Path>
<MBinaries32Path Condition="'$(MBinaries32Path)'==''">$(ProgramFiles)\Microsoft Oslo SDK 1.0\Bin</MBinaries32Path>
<MGrammarTargetsPath Condition="'$(MGrammarTargetsPath)'==''">$(ProgramFiles)\MsBuild\Microsoft\M\Grammar\v1.0</MGrammarTargetsPath>
<MgTarget>Mgx</MgTarget>

Under the ItemGroup tag containing many Reference tags, add these lines:

<Reference Include="$(MBinaries32Path)\Intellipad\ComponentModel.dll" />
<Reference Include="$(MBinaries32Path)\Intellipad\Microsoft.Intellipad.Framework.dll" />
<Reference Include="$(MBinaries32Path)\Intellipad\Microsoft.Intellipad.Core.dll" />
<Reference Include="$(MBinaries32Path)\Intellipad\Microsoft.VisualStudio.Platform.Editor.dll" />
<Reference Include="$(MBinaries32Path)\Microsoft.M.Grammar.dll" />
<Reference Include="$(MBinaries32Path)\System.Dataflow.dll" />
<Reference Include="$(MBinaries32Path)\Xaml.dll" />

Finally, find the node containing the name of your Mg file (e.g., "todo4.mg") and change its tag to MgCompile:

<ItemGroup>
  <MgCompile Include="todo4.mg" />
</ItemGroup>

Save your changes and reload the project. These changes can be seen in ToDoPlugin.csproj.

Step 3.2 -- Provide Hooks

Now we write the code that will implement the custom mode and, most importantly, retrieve the parser created by Mg and provide it to Intellipad for syntax analysis. You can follow along in ToDoMode.cs.

A custom mode is defined by creating a class which inherits from Microsoft.Intellipad.Mode. Attributes attached to the class specify the name of the mode and file extensions associated with it:

[Export("{Microsoft.Intellipad}Mode")]
[Export("{Microsoft.Intellipad}ToDoMode")]
[ExportProperty("{Microsoft.Intellipad}Name", "To-do Mode")]
[ExportProperty("{Microsoft.Intellipad}Extension", ".todo")]
public class ToDoMode : Mode
{
    ...
}

Intellipad calls CreateComponentDomain in order to instantiate our mode, so we override it as below:

protected override ComponentDomain CreateComponentDomain()
{
    // boilerplate
    var domain = StandardMode.CreateChildDomain();
    domain.AddComponent(new LanguageServiceItemProvider());
    domain.Bind();
    return domain;
}

Notice the LanguageServiceItemProvider object created. We need to define that for our grammar. An inner class does the trick. We add attributes and boilerplate to the class:

[Export("{Microsoft.Intellipad}LanguageServiceItemProvider")]
[ComponentOptions(ComponentDiscoveryMode = ComponentDiscoveryMode.Never)]
class LanguageServiceItemProvider : ILanguageServiceItemProvider
{
    // boilerplate
    [Import]
    public ISquiggleProviderFactory SquiggleProviderFactory { get; set; }

    public ILanguageServiceItem CreateItem(BufferView bufferView)
    {
        return new ToDoLanguageServiceItem(bufferView, SquiggleProviderFactory);
    }
}

CreateItem returns another custom object, ToDoLanguageServiceItem. It's defined as another inner class:

class ToDoLanguageServiceItem : ILanguageServiceItem
{
    public ToDoLanguageServiceItem(BufferView b, 
             ISquiggleProviderFactory squiggleProviderFactory)
    {
        ...
    }
}

This class is where the real meat is. Our constructor creates some bookkeeping objects which are not detailed here. Then it loads the parser from our assembly:

using (Stream stream = Assembly.GetExecutingAssembly().
                         GetManifestResourceStream("ToDo.mgx"))
{
    parser = Microsoft.M.Grammar.MGrammarCompiler.
              LoadParserFromMgx(stream, "ToDo.Tasks4");
}

Hooking the parser up to Intellipad is accomplished through a ParserClassifier object. We create this object with our parser and a reference to our textbuffer:

this.classifier = new ParserClassifier(parser, bufferView.Buffer.TextBuffer);

classifier is used in the GetClassificationItems method, which is a member of the ILanguageServiceItem interface that we are implementing:

public IEnumerable<ClassificationItem> GetClassificationItems(SnapshotSpan span)
{
    return classifier.GetClassificationItems(span);
}

Errors are detected through the private method Reparse, which is called every 100 milliseconds. The text in the window is parsed if it has changed since the last call:

ToDoErrorReporter reporter = new ToDoErrorReporter();
parser.ParseObject(uri.ToString(), 
    new TextSnapshotToTextReader(this.textBuffer.CurrentSnapshot),
    reporter);

The ToDoErrorReporter class provides a means of storing parsing errors. It inherits from System.Dataflow.ErrorReporter and is not very interesting. After parsing, reporter will contain any errors. We pass that object to ProcessSquiggles, which adds appropriate highlights for each error. ProcessSquiggles also takes care of cleaning up leftover squiggles if no errors are present.

The MgCompile task we added to the project in step 3.1 runs mgx.exe, the Mg compiler that ships with the SDK. It takes the grammar specified, builds a custom parser for it, and adds it as a resource to the assembly. In our case, the Mg file is called todo4.mg, but contains a module named ToDo. The resource is named after the module, so ours is called "ToDo.mgx". To load a parser, a language name must be provided. The language name is a combination of the module and language names given in the file. In our case, that is "ToDo.Tasks4". You can see how those names were used when we loaded our parser above.

Step 3.3 -- Create Python Scripts

A Python script, ToDoCommands.py, provides the glue between Intellipad and our custom mode:

@Metadata.ImportSingleValue('{Microsoft.Intellipad}Core')
def Initialize(value):
    global Core
    Core = value
    Common.Initialize(Core)

@Metadata.CommandExecuted('{Microsoft.Intellipad}BufferView', 
                          '{Microsoft.Intellipad}SetToDoMode', 
                          'Ctrl+Shift+G')
def SetToDoMode(target, sender, args):
    sender.Mode = Core.ComponentDomain.
                   GetBoundValue[System.Object](
                    '{Microsoft.Intellipad}ToDoMode')

The second method, SetToDoMode, is all that must be customized.

Step 4 -- Add our Custom Mode to Intellipad

Adding our mode to Intellipad is as easy as copying the assembly and python script to Intellipad\Components\ToDo. In our case the files needed are:

  • ToDo.dll
  • ToDoCommands.py
  • Microsoft.M.Grammar.dll

After copying the files, start Intellipad. Open a file with the "todo" extension (one is included in the project sample). The mode changes to "ToDoMode" and syntax coloring is apparent. Keywords such as task are colored correctly and errors show up as red squiggles. Hovering your mouse over an error even shows what the syntax problem is.

The last file above is a bit odd. Without it, Intellipad crashes the second time it's opened after each build. You'll notice the project has a post build event to copy the necessary files to the default instal location. They are commented out initially, so you'll need to update them if you'd like to avoid manually copying these files.

These articles may also be useful:

Conclusion

This sample has barely scratched the surface of what a custom mode can do. An obvious enhancment is to provide a mode that takes a Mg language, some means of specifying file extensions, and hooks up all the necessary plumbing to provide syntax highlighting and error support. I submit that idea as challenge to any ambitious readers!

As always, feedback, comments and patches are welcome.

Category: None

Please login to comment.