2011-02-05

Novelang-Desktop

This is the result of a conversation with another Novelang user, coming with ideas for a better integration with the desktop environment.

First, as strange as it seems, Novelang is a desktop application. While it embeds a Web server it’s not meant to serve content to remote clients. The support of a Web client is just the fastest approach to give quick preview of files edited in a desktop-integrated text editor. (For serving Novelang documents on the World Wide Web, it’s much faster to generate them as static files first.)

By now the desktop integration relies on a java -jar Novelang-...jar typed in a command-line console. That’s a bit harsh.

The yet-to-develop Novelang-desktop project would be fancier. Novelang-desktop is a graphical application where user sets content roots and associated TCP ports. With this, editing or viewing Novelang documents from multiple content directories doesn’t require to start several JVMs anymore.

Here is a skeletal view of the main window of Novelang-desktop:

 ___________________________________________
|  __________________________               |
| | C:\projects\docs  | 8081 |  [ Add...  ] |
| | I:\shared\foo     | 8082 |  [ Edit... ] |
| |___________________|______|  [ Remove  ] |
|                                           |
| [?]                      [ Configure... ] |
|___________________________________________|

Double-clicking on a table line opens a Web browser with the directory listing of the content root.

The directory listing comes with great improvements. It shows every files, and each Novelang source file has several associated links:

  • Open in current Web browser (one link per rendition MIME type, like PDF, HTML, FO, XML).
  • Open in default editor.
  • Batch generation.

For the last two there is a trick: the link loops back to a special service of Novelang-desktop, which has all the powers to execute arbitrary code and scan local filesystem.

  • For editor opening it executes a pre-configured command with the file path as an argument.
  • For batch generation first it opens a dialog requesting target directory.

Novelang-desktop’s configuration goes in user’s home directory by default.

2011-02-04

Novelang-0.55.0 released!

Just released Novelang-0.55.0!

Summary of changes:

  • New --temporary-dir option. For better error messages, Novelang now buffers the whole document before sending it to the HTTP client. When the document is too big Novelang buffers it into a temporary file under this directory. If an error occurs then its not too late to send an HTTP redirection.
  • Fixed SVG embedding for PDF. Now the image appears as true vector image inside the PDF. Correct reference to the SVG resource implied adding a $content-directory parameter passed to XSL stylesheets.
  • Fixed loss of request parameters when issuing error page.
  • Less verbose logging of Logback configuration at startup.
  • Fixed various cases of bad problem reporting, where location in origin file was missing.
  • Plenty of other small fixes.

Download it from here.

Enjoy!

2011-01-23

Secrets for embedding SVG images in PDF documents

Nhovestone reports don’t look good when zooming, because the charts are bitmap images. While trying to render them as vector images, a few interesting things did appear.

(Most of them relate to FOP-1.0 and Batik-1.7 which are the latest versions at this time.)

There was a bug (fixed in HEAD) preventing from embedding SVG images in PDF documents. Novelang resolves the image path (in n:vector-image/n:resource-location) as relative to the content directory. But most of times the stylesheet doesn’t reside right under the content root, so FOP (the PDF renderer) couldn’t find the SVG file. Now XSL stylesheets receive a $content-directory parameter to solve this.

There are several manners to embed an SVG file into a PDF. In order to keep the vector graphics (means: avoiding an ugly conversion to a bitmap) FOP needs a rather twisted declaration of the form:

<fo:instream-foreign-object>
  <svg:svg width="123px" height="456px" >
    <svg:image width="123px" height="456px" xlink:href="file:/my/image.svg" />
  </svg:svg>

The documentation explains that width and the height must appear two times, with exactly the same values.

It’s important to note that an SVG image must always have width and height. Moreover, they must be in pixel unit. Pixel unit is bad, as it introduces a dependency to document’s resolution. But units other than pixel don’t work.

Knowing this, the graphs started to look really good. Unfortunately, the grid becomes bigger when zooming in. In theory, the solution is to set line with to a zoom-independant value with vector-effect="non-scaling-stroke" property (thanks to StackOverflow for that hint). Unfortunately, this is SVG 1.2. Batik doesn’t support this yet and some archived mails show this is somewhat tricky to implement.

Conclusion: while SVG embedding in PDF is far from perfect, it’s much better than rasterized images that seemed to be FOP’s approach. In order to get best vector images we'll have to wait for non-scaling stroke feature in Batik, and maybe some FOP upgrade in order to propagate this feature to generated PDF.

2011-01-18

Aggregated javadoc with Maven

This generates a javadoc covering every subproject. From Novelang root project:

mvn -Dnovelang.javadoc.skip=false javadoc:aggregate

Generated HTML goes in target/site/apidocs.

2011-01-17

Nhovestone report for Novelang-0.54.0

The Nhovestone report shows that Novelang-0.54.0 is as at least as fast as previous versions. Good news!

2011-01-07

Novelang-0.54.0 released!

Just released Novelang-0.54.0!

Summary of changes:

  • Because of some cleanup, n:relative-identifier becomes illegal in stylesheets (wasn’t generated since a few versions). This might break a few existing stylesheets.
  • Better logging of accessed resources.
  • Documentation enhancements.

Download it from here.

Enjoy!

2011-01-03

Novelang-0.53.6 released!

Just released Novelang-0.53.6!

Summary of changes:

  • Fixed resource loading bug appeared in 0.53.5, that caused to ignore overriden resources (like stylesheets).

Download it from here.

Enjoy!

2011-01-01

Novelang-0.53.5 released!

Just released Novelang-0.53.5!

Summary of changes:

  • Various logging enhancements.
  • Various enhancements to the documentation.
  • Disabled full parsing and validation for SVG files.

Download it from here.

Enjoy!

2010-12-29

Novelang-0.53.4 released!

Just released Novelang-0.53.4!

Summary of changes:

  • Fixed logging configuration with --log-dir option.

Download it from here.

Enjoy!

2010-12-27

Novelang-0.53.3 released!

Just released Novelang-0.53.3!

Summary of changes:

  • Fixed barcode generation.
  • Minor logging enhancements.

Download it from here.

Enjoy!

Browser compatibility for HTML documentation

Since version 0.53.0 Novelang HTML documentation displays correctly on following browsers:

  • Safari 5.0 (Mac OS X 10.6.5, Windows XP 32bits SP3).
  • Firefox 3.6 (Mac OS X 10.6.5, Windows XP 32bits SP3).
  • Google Chrome 8.0 (Windows XP 32bits SP3).
  • Internet Explorer 8 (Windows XP 32bits SP3).

The website also passed W3C validation for HTML 4.0 Transitional.

Some links about CSS layout

Three Column Stretch : strech isn’t that good, makes too long lines for main text.

Piefecta – A superb 3-col tableless layout – long right col looks good on Safari and zooms well. Tells it deals with various browser bugs. Probably the finest piece of engineering but too many fixes are making it unreadable. We don’t care about supporting IE6. License: unknown.

CSS Fixed Layout #3.1 (Fixed-Fixed-Fixed) zooms correctly. Much simpler. License: unknown.

Elastic-fluid hybrid got it right. It scales up and down in with, staying in fixed limits. Read author’s comments .

Also check this tutorial about elastic layout.

The Holy Grail 3 column Liquid Layout has great explainations. (Only 1 error on W3C validator but the element reported to be missing appears as it should – validator bug?) License: free to use, linkback appreciated. After a close look, it turned out that forcing min-width and max-width for the column (worst case: nesting another div) and using proper text alignment does the job.

2010-12-26

Novelang-0.53.2 released!

Just released Novelang-0.53.2!

Summary of changes:

  • Indicating error location when something goes bad during XSL transformation.
  • Minor fixes on HTML documentation.

Download it from here.

Enjoy!

2010-12-23

Novelang-0.53.1 released!

Just released Novelang-0.53.1!

Summary of changes:

  • Minor cosmetic changes for HTML generation.

Download it from here.

Enjoy!

Novelang-0.53.0 released!

Just released Novelang-0.53.0!

Summary of changes:

  • Experimental support for Multipage. See the result in Novelang documentation.
  • Small logging enhancements.
  • More restrictive rules when applying XSL stylesheets. Generation now breaks on warnings. This might break existing incorrect stylesheets.
  • Changed default representation of Fragment Identifiers, both Implicit and Explicit. Removed leading double reverse solidus \\ when rendering (still required in document sources).

Download it from here.

Enjoy!

2010-11-19

Rule-based number spelling

Novelang comes with a Numbering class which formats an integer value in words. This adds a bit of magic when the stylesheet writes "Chapter fourty-two" from a stupid counter.

Currently the Numbering class only supports French and English, and values from 0 to 50 (all values are hardcoded). The ICU project offers the RuleBasedNumberFormat which supports rule-based formatting. This makes easy to support much greater ranges.

2010-11-17

XSL mockup for multipage rendering

Here is how an XSL would render a multipage document.

First, let’s consider the whole document defining the opus:

== One

Some text of level one.

== Two

Some text of level two.

=== Two-one

Some text of level two-one.

The XML form of the document above is:

<?xml version="1.0" encoding="UTF-8" ?>
<opus>
  <level>
    <title>One</title>
    <paragraph>Some text of level one.</paragraph>
  </level>
  <level>
    <title>Two</title>
    <paragraph>Some text of level two.</paragraph>
    <level>
      <title>Two-one</title>
      <paragraph>Some text of level two-one.</paragraph>
    </level>
  </level>
</opus>

Let’s take for granted that Novelang supports XSL metadata. Our multipage-enabled stylesheet would define an embedded stylesheet that transforms a whole opus into a simple map of page names and page paths. A path is whatever the stylesheet may reprocess, but an XPath expression is quite good. For the document above, here is how our map could look like, if we want to support 2 levels:

page1 -> /opus/level[1]
page2 -> /opus/level[2]
page3 -> /opus/level[2]/level[1]

Please note that, at this point, the decisision to support a given depth, or exclude some tagged levels, entirely belongs to the page-extracting stylesheet.

By merging the page map with the opus, we get the XML input for the rendering of one page. Novelang knows which page it is either because it is iterating over all known pages of the map (batch mode), or because the page name is a part of the request issued to the HTTP dæmon.

<op>us>
  <meta>
    <page>
      <name>page2</name>
      <path>/opus/level[2]</path>
    </page>
  </meta>

  <level>
    <title>One</title>
    <paragraph>Some text of level one.</paragraph>
  </level>
  <level>
    <title>Two</title>
    <paragraph>Some text of level two.</paragraph>
    <level>
      <title>Two-one</title>
      <paragraph>Some text of level two-one.</paragraph>
    </level>
  </level>
</opus>

(Note: the n: namespace prefix doesn’t appear here for brevity.)

The stylesheet gets this whole document as input for every page. All what changes is the name, path pair in the meta/page element. The stylesheet needs to know which page it is rendering, and the whole document tree as well, in order to create a navigation bar or any kind of header or footer corresponding to a specially-titled or tagged level of the document.

This involves some XSL trickery: evaluating an XPath expression at runtime. While it’s not part of XPath 1.0 specification, it is a part of semi-official EXSLT communitiy initiative. The dyn:evaluate http://www.exslt.org/dyn/functions/evaluate does that for us. It works well with Xalan-2.7.1 which is the XSLT engine bundled with Novelang (it works a slightly better than JDK’s one).

In the stylesheet below, we save useful expressions into variables.

The root template prints those variables, then a pseudo-navigation bar made of nested lists.

The nested loop for iterating over level elements is rather ugly but it makes sense as we don’t want infinite deph of titles in a navigation bar.

The title-with-locator template just adds bold on the title in the navigation bar that corresponds to current page.

All other templates mimic Novelang’s standard rendering.

<xsl:stylesheet
    version="1.0"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:dyn="http://exslt.org/dynamic"
    extension-element-prefixes="dyn"
>
  <!-- Be sure to use Xalan-2.7.1 (not JDK's default). -->

  <!--
    Here, expect a meta section, embedding a stylesheet 
    that extracts the pages we'll find in the meta section 
    of input document.
  -->

  <xsl:output method="html" />

  <xsl:variable name="page-name" select="/opus/meta/page/name" />
  <xsl:variable name="page-path" select="/opus/meta/page/path" />
  <xsl:variable name="page-nodeset" 
      select="dyn:evaluate( $page-path )" />
  <xsl:variable name="page-id" 
      select="generate-id( $page-nodeset )" />

  <xsl:template match="meta/page" >
    $page-name=<xsl:value-of select="$page-name" />
    $page-path=<xsl:value-of select="$page-path" />
    $page-id=<xsl:value-of select="$page-id" />
  </xsl:template>

  <xsl:template match="/opus" >
    <html>
      <xsl:apply-templates select="meta" />

      <!-- Navigation bar -->
      <ul>
        <xsl:for-each select="level">
          <li>
            <xsl:call-template name="title-with-locator"/>
          </li>
          <xsl:if test="level">
            <ul>
              <xsl:for-each select="level">
                <li>
                  <xsl:call-template name="title-with-locator"/>
                </li>
              </xsl:for-each>
            </ul>
          </xsl:if>
        </xsl:for-each>
      </ul>

      <!-- Document body, same templates as usual -->
      <xsl:apply-templates select="$page-nodeset" />

    </html>

  </xsl:template>


  <xsl:template match="paragraph" >
    <p>
      <xsl:value-of select="." />
    </p>
  </xsl:template>

  <xsl:template match="title" />

  <xsl:template match="level" >
    <h2><xsl:value-of select="title" /></h2>
    <xsl:apply-templates/>
  </xsl:template>

  <xsl:template match="level/level" >
    <h3><xsl:value-of select="title" /></h3>
    <xsl:apply-templates/>
  </xsl:template>

  <xsl:template name="title-with-locator" >
    <xsl:text>
    </xsl:text>
    <xsl:choose>
      <xsl:when test="generate-id( . ) = $page-id" >
        <b><xsl:call-template name="title-alone" /></b>
      </xsl:when>
      <xsl:otherwise>
        <xsl:call-template name="title-alone" />
      </xsl:otherwise>
    </xsl:choose>
  </xsl:template>

  <xsl:template name="title-alone" >
    Title: <xsl:value-of select="title" />
  </xsl:template>

</xsl:stylesheet>

Finally, this is how the rendering looks like:

2010-11-13

Novelang-0.52.0 released!

Just released Novelang-0.52.0!

Summary of changes:

  • Added n:block-inside-asterisk-pairs. Default stylesheet render it as bold.

Download it from here.

Enjoy!

Grammar pattern: twin delimiters

This post describes a tricky point of Novelang’s grammar design: how to handle twin delimiters like // in a non-ambiguous manner for an ANTLR grammar. It’s a useful refresh before adding long-awaited ** (asterisk pair) delimiter.

The problem

For paired delimiters like ( and ) or [ and ] it’s easy to know when to “open” or “close” a block, and support nested blocks. In contrast, a twin delimiter is an opening one if not preceded by a closing one inside the same block, regardless of what happens in subblocks. This is a complicated way to say we support this kind of nesting:

// block-1 ( block-2 //block-3// ) //

+ block-inside-solidus-pairs
    block-1
  + block-inside-parenthesis
      block-2
    + block-inside-solidus-pairs
        block-3

We also support this:

block-1 // block-2 // block-3 // block-4 //

  block-1
+ block-inside-solidus-pairs
    block-2
  block-3
+ block-inside-solidus-pairs
    block-4

(We have only one level of nesting here. 2 levels of nesting is counter-intuitive and would have required very complex lookahead.)

The pattern

The pattern is to define special grammatical elements when inside a block defined by a twin delimiter, to propagate this element cannot appear again, unless inside some other subblock.

Taking “XXX” for the name of some twin delimiter, here is a simplified version of the grammar for spreadblocks. The term “spreadblock” stands for a block that may spread on several lines (containing single line breaks).

paragraph
  : ... mixedDelimitedSpreadblock
  ;

mixedDelimitedSpreadblock
  : word ( punctuationSign | delimitedSpreadblock ) ...

delimitedSpreadblock
  : xxxSpreadblock
  : parenthesizedSpreadblock
  | squareBracketsSpreadblock
  | doubleQuotedSpreadblock
  | hyphenPairSpreadblock
  ;

parenthesizedSpreadblock  
  : '(' spreadblockBody ')' // Same for other paired delimiters.
  ;

spreadblockBody
  : ... mixedDelimitedSpreadblock
  ;

xxxSpreadblock
  : XXX spreadblockBodyNoXxx XXX
  ;

spreadblockBodyNoXxx
  : ... mixedDelimitedSpreadblockNoXxx ...
  ;

mixedDelimitedSpreadblockNoXxx
  : ... delimitedSpreadblockNoXxx ...
  ;

delimitedSpreadblockNoXxx
  : parenthesizedSpreadblock
  | squareBracketsSpreadblock
  | doubleQuotedSpreadblock
  | hyphenPairSpreadblock
  ;

This is more or less the same for tightblocks. “Tightblocks” stand for blocks containing no line breaks, like cells and embedded lists.

acell  // Same for embedded list items.
  : ... mixedDelimitedTightblock ..
  ;

mixedDelimitedTightblock
  : word ( punctuationSign | delimitedTightblock | ... ) ...
  : word ( punctuationSign | delimitedSpreadblock | ... ) ...
  ;

delimitedTightblock
  : xxxTightblock
  | parenthesizedTightblock
  | squareBracketsTightblock
  | doubleQuotedTightblock
  | hyphenPairTightblock
  ;

xxxTightblock
  : XXX tightblockBodyNoXxx XXX
  ;

tightblockBodyNoXxx
  : ... mixedDelimitedTightblockNoXxx ...
  ;

mixedDelimitedTightblockNoXxx
  : word ( punctuationSign | delimitedTightblockNoXxx ) ...
  ;

delimitedTightblockNoXxx
  : parenthesizedTightblock 
  | squarebracketsTightblock
  | doubleQuotedTightblock
  | hyphenPairTightblock
  ; // That's all.

Thought it is over? There is another kind of block, the delimitedTightblockNoSeparator used inside the subblockAfterTilde which reflects each block inside ~x~y~z! But at this point you probably got the idea.

Yes this makes the grammar quite verbose, but factoring it would reduce ANTLR’s ability to check for inconsistencies. Anyways, the slightest addition brings the need of writing test cases for every logical path inside each ANTLR grammar rule.

2010-11-07

Novelang-0.51.1 released!

Just released Novelang-0.51.1!

Summary of changes:

  • Upgraded from FOP-0.95 to FOP-1.0. FOP is the library for generating PDF documents.
  • Various other library upgrades that shouldn’t affect normal users.

Download it from here.

Enjoy!