This is a technical blog post meant for developers

This blog post introduces XQuery template tags, describes how we are upgrading our configuration to use them and where they are headed. This post hopes to inspire you to refactor XPaths in your editor to start benefiting from XQuery Template Tags as early as you can.

XQuery

XQuery (and XPath) is a big part of how Fonto Editor is configured. Up until recently, these XPaths were JavaScript strings that are inlined in JavaScript files. Just about any editor contains a lot of strings that start with self::, all of these are XPath selectors. For more complex selectors it was common to just string-concatenate expressions together to generate new ones:

const selector = 'self::' + nodeName + ' and [@attr = "' + additionalFilter + '"]';

These selectors are hard to read and even harder to debug. Even more so when those strings are coming from external systems! When the parts that build up a selector contain the wrong quotes, mismatched parentheses or anything unexpected, the editor can misbehave unless they are sanitized! No one wants a bobby tables situation!

// This may crash the editor when the XPath is evaluated<br>const nodeName = 'this is a piece of nonsense (:';
// And this can be very tricky to debug: the ampersand is illegal like this in XQuery<br>const additionalFilter = 'looks &amp; feels good';
const selector = 'self::' + nodeName + ' and [@attr = "' + additionalFilter + '"]';

The 7.17.0 release of Fonto introduces the xq function, which can create XQExpression instances, a typed version of XPaths. We like to call this feature XQuery Template Tags. These Template Tags are more robust against common pitfalls of joining strings together by working more like a query builder: passed strings are replaced with variables to remove the need for escaping strings and subexpressions are injected in a safe way.

Last week we upgraded our dita example configuration to use XQuery Template Tags, in this blog post we want to explain how we did that. We then lift the curtains a bit to show off what our plans are for template tags in the long run and why we are so enthusiastic about them.

Upgrading the example configuration

While not all expressions are convertible yet, it already makes sense to convert whatever can be converted so that the caveats described above can be dodged! We upgraded the example configuration and we want to share the lessons we learned doing that.

Of course in every file we touched, we added the following import to make the xq function available:

import xq from 'fontoxml-selectors/src/xq';

The conversions we did fell in a few patterns:

Plain and simple XPaths

These are by far the most common. They look like self::p or self::italic or self::p and ancestor::div. They are not dynamic, they do not include variables and they are just simple strings.

BEFORE	AFTER
configureAsBlock(sxModule, ‘self::p’)	configureAsBlock(sxModule, xq`self::p`)
configureAsInlineFormatting(sxModule, ‘self::italic’)	configureAsBlock(sxModule, xq`self::italic`)
useXPath(‘fonto:selection-common-ancestor()’)	useXPath(xq`fonto:selection-common-ancestor()`)
evaluateXPathToBoolean(‘self::p and ancestor::div’, node, blueprint)	evaluateXPathToBoolean(xq`self::p and ancestor::div`, node, blueprint)

Dynamic XPaths that use strings

We encountered a few XPaths that were concatenating a string into an XPath. In the example config they were used to reuse configuration for the DITA <hazardstatement/> element, which has different visualizations for the type attribute:

BEFORE	AFTER
useXPath(‘@value = “‘ + value + ‘”‘, node)	useXPath(xq`@value = ${value}`, node)
useXPath(`@value = “${value}”‘, node)	useXPath(xq`@value = ${value}`, node)
Object.keys(HAZARD_VISUALIZATION_BY_TYPE) .forEach(function(hazardType) { const hazardVisualization = HAZARD_VISUALIZATION_BY_TYPE[hazardType]; configureProperties( sxModule, xq`self::hazardstatement[@type=”${hazardType}”]`, { markupLabel: hazardVisualization.label, backgroundColor: hazardVisualization.backgroundColor } ); });	Object.keys(HAZARD_VISUALIZATION_BY_TYPE) .forEach(function(hazardType) { const hazardVisualization = HAZARD_VISUALIZATION_BY_TYPE[hazardType]; configureProperties( sxModule, xq`self::hazardstatement[@type=${hazardType}]`, { markupLabel: hazardVisualization.label, backgroundColor: hazardVisualization.backgroundColor } ); });

Full code generation: dynamic node names

There were a few places where the example configuration was generating XPath expressions. One of them was in a custom mutation that tried to find nodes with a certain unknown nodeName. We discovered that by using the name() function those queries are easier to read, and again, more robust:

BEFORE	AFTER
const columnNodes = evaluateXPathToNodes( ‘parent::*/parent::properties/*/*[self::’ + contextNodeName + ‘ or self::’ + otherNodeName + ‘]’, contextNode, blueprint );	const columnNodes = evaluateXPathToNodes( xq`parent::*/parent::properties/*/*[name() = ${ contextNodeName } or name() = ${ otherNodeName }]`, contextNode, blueprint );

Another place where we generated code was in a transform that created a JSONML representation of a table. Again, the node names are dynamic, but this time even the amount of node names was unknown! We had two choices:

• We could string join the names with spaces to separate them and tokenize the names afterwards. We did not want to do this because it is again not really robust against unexpected input: we would have unexpected behaviour if the name erroneously contained spaces themselves!
• We can loop over the list of node names in JavaScript. We chose this because it made the code very readable.

BEFORE	AFTER
stepData.columns.forEach(function(column) { let selector = ‘child::*/’ + column.currentNodeName; selector += column.otherNodeNames ? ‘ or child::*/’ + column.otherNodeNames.join(‘ or child::*/’) : ”; if (!evaluateXPathToBoolean(selector, tableNode, readOnlyBlueprint)) { return; } … }	stepData.columns.forEach(function(column) { if (!evaluateXPathToBoolean(xq`child::*/child::*[name() = ${ column.currentNodeName }]`, tableNode, readOnlyBlueprint)) { return; } if (column.otherNodeNames && column.otherNodeNames.some(nodeName => evaluateXPathToBoolean(xq`child::*/child::*[name() = ${ nodeName }]`, tableNode, readOnlyBlueprint))) { return; } … }

Do pay attention to performance here! The name() function does not perform the same as a name test regarding buckets. Check our earlier blogpost for more information on this! If the code is performance-heavy, consider solving this in another way!

Sub expressions

While we did not find any expressions that consist of other expressions, we did have a few in the platform code. They are used to ‘fold’ a reusable expression that is dynamic into another one. This is made a lot easier with XQuery Template Tags: just interpolate them! Take this one for example, which comes from the configureAsConref family:

BEFORE	AFTER
const matchElementWithConref = ‘@conref and (‘ + selector + ‘)’;	const matchElementWithConref = xq`@conref and ${selector}`;

Status of our refactorings on the platform side

Not every API that accepts XPath expressions accepts XQExpressions yet. The most notable ones are the APIs related to tables, like configureAsCalsTableElements, configureAsXhtmlTableElements or configureAsSimpleTableElements. The configureAsMathMlContainer and its configureAsInlineMathMlContainer are not supported yet. The configureAsLabelQueryWidget widget is also not yet able to handle XQuery Template Tags. These APIs are planned to support XQuery Template Tags in the 7.18 release.

The whole operations pipeline is unable to use XQuery Template Tags because they are defined in JSON. We are working to make XQuery Template Tags available in the operation pipeline but we do not expect this to change in the 7.18 release already.

Future plans

At this moment there is already a benefit to using XQuery Template Tags in your editor: complex XPaths are easier to build. We have a few plans to continue improving those benefits and to make configuring Fonto even easier!

Get full feature parity with all APIs

Obviously the first step is to make every API imaginable work with XQuery Template Tags. This is ongoing work, but we expect to have XQuery Template Tags available in all JavaScript, TypeScript, and React APIs at the 7.18.0 release.

IDE support for syntax highlighting

Because the XQuery Template Tags always look like a template string that start with xq, we can write an IDE plugin that recognizes them. We are planning to publish it for VSCode either during the development of the 7.18 version of Fonto, or directly after.

Example of XQuery Template Tags being syntax highlighted

Locally bound namespace prefixes

Namespace prefixes are one of the last global configuration variables that affect just about the whole editor. When you have an editor that configures elements in more than one namespace you are forced to repeat a lot of namespace prefixes. Because we can now give XPaths context, we can make the prefix to namespace URI configuration local to the XPath. We are planning to release this in the 7.18 release.

Similarly, we will allow binding module imports to an XQExpression. This will make it easier to use functions that are defined in an XQuery module without having to repeat the import module prefix=’http://my-namespace-uri’; in every XPath string. 

Syntax checking while building the editor

It is annoying to run an editor, open a modal you are developing, walk through some steps in a wizard just to get a big error about a typo somewhere in an XPath. Because we can now recognize XPaths in JavaScript and TypeScript source code, we will be able to warn you when there is such a typo in an XPath, just like you get a warning or an error when you have syntax errors in your JavaScript code.

More

We have many more plans for XQuery Template tags in the future. If you have any ideas, please don’t hesitate to share them! We are always open to suggestions. Reach out on Twitter to @fontoxml or me (@dr_rataplan) directly, send a mail to the team@fontoxml.com address or file a support issue!

Conclusion

This blogpost introduced XQuery Template Tags, their benefits and how we use them in editors that we are maintaining. Furthermore we presented our ideas for the future. We hope this blogpost made you enthusiastic on the future of configuring Fonto!