A year out of Sun

I was going to do this on the day of the anniversary of my leaving Sun but, you know, I was too busy at the time.

In the past year I have:

  • Helped companies and organisations in the USA, England, and France with their XSLT, XSL, and XML, including:
    • Making transforming to HTML go faster for an online retailer
    • Reviewing XSLT stylesheets and suggesting improvements for a major library
    • Writing XSLT for XML-XML transformations for a journal publisher
    • Specifying and implementing XSLT transformations for an archive service
    • Augmenting a XSLT-based automated schema documentation system that produces both HTML and PDF
    • Providing expert help to get a Perl XML::LibXSLT project off the ground
  • Presented on XSLT profiling and unit testing at XTech 2007 in Paris in May
  • Been selected to present training sessions on transitioning to XSLT 2.0 and on testing XSLT at XML 2007 in Boston in December
  • Rejoined the W3C XSL FO subgroup as an invited expert
  • Made four xmlroff releases, with another happening any day now
  • Learned more about VAT and PRSI than I ever wanted to know (okay, maybe that’s not such a high point)
  • Participated in the Workshop of the W3C Japanese Layout Taskforce in Tokyo in September
  • Helped kids by completing two projects with the International Telementor Program

This is also the point at which I retire the “RIF” blog category as it has become irrelevant.

BOM in UTF-8: good, bad, or ugly?

The usefulness or otherwise of U+FEFF (ZERO WIDTH NON-BREAKING SPACE and BYTE ORDER MARK) in UTF-8 has been subject to reinterpretation over the years. It wasn’t mentioned in the original XML 1.0 Recommendation but was added later, rather like how its use was added to the Unicode Standard.

In the Unicode Standard 2.0, there was no mention of U+FEFF with UTF-8, either in the section on the BOM or in the appendix defining UTF-8.

In the Unicode Standard 3.0, section 13.6, “Specials”, includes:

Although there are never any questions of byte-order with UTF-8 text, this sequence can serve as signature for UTF-8 encoded text where the character set is unmarked.

In the Unicode Standard 5.0, section 3.10, “Unicode Encoding Schemes”, includes:

While there is obviously no need for a byte order signature when using UTF-8, there are occasions when processes convert UTF-16 or UTF-32 data containing a byte order mark into UTF-8. When represented in UTF-8, the byte order mark turns into the byte sequence <EF BB BF>. Its usage at the beginning of a UTF-8 data stream is neither required nor recommended by the Unicode Standard, but its presence does not affect conformance to the UTF-8 encoding scheme. Identification of the <EF BB BF> byte sequence at the beginning of a data stream can, however, be taken as a near-certain indication that the data stream is using the UTF-8 encoding scheme.

So in the Unicode Standard it’s gone from irrelevant to useful to “Oh, if you must”.

(BTW, in other reinterpretations, “Unicode Encoding Scheme” results from splitting the meaning of “UTF”, and the use of U+FEFF to indicate non-breaking is deprecated these days.)

The Unicode FAQ both lists its use as a signature and says to avoid its use where “byte oriented protocols expect ASCII characters at the beginning of a file“. However, I don’t think that XML necessarily counts as one such byte oriented protocol.