xparser - Lua XML Parser

Introduction
Implementation
Constructor
Parser methods
Event handlers
Sample code
xml2table.lua

Introduction

xparser reads XML in chunks and calls Lua functions as it encounters elements, text, and other XML content. Use it to process a stream without first building a table for the whole document. For a table representation, see xml2table.lua. See also the JSON API.

The parser accepts UTF-8, including an optional leading UTF-8 byte order mark. It is not a validating XML parser: it skips DOCTYPE declarations and passes namespace-qualified names to callbacks without resolving their namespaces.

Implementation

Create a parser with xparser.create(), feed it nonempty strings with parser:parse(), and call parser:destroy() when finished. The parser is a userdata object. It also supports automatic cleanup through Lua's garbage collector and to-be-closed variables.

Successful parsing returns true, complete. The boolean complete becomes true when the root element has closed. A successful call with complete == false needs more input. There is no separate end-of-input call or end-of-document event; comments and processing instructions can follow the root element. At the end of your input, check completion yourself.

Malformed XML and errors raised inside callbacks are reported as return values. Incorrect API use, such as a wrong argument type or calling a destroyed parser, throws a Lua error. Each function below lists its own throw conditions. As with other Lua APIs, memory exhaustion in Lua allocations can also raise a Lua error; the listed programmer-error checks are separate from that runtime condition.

Constructor

xparser.create([handlers [, context [, textmode]]])

Create an independent parser. If supplied, the INIT handler runs during this call.

Parameters

table handlers - Optional table of functions indexed by the case-sensitive event names. Omit it to parse without callbacks. An explicit nil is not accepted. Individual handlers may be absent or nil.
any context - Optional value passed as the first argument to each handler; default: nil. INIT may replace it. The parser retains the value while the parser object remains reachable.
string textmode - Optional: "PRESERVE" (default), "TRIM", or "SKIPBLANK". PRESERVE keeps text whitespace. TRIM removes leading and trailing whitespace from each text segment and omits blank segments. SKIPBLANK omits blank text segments while preserving whitespace in other segments. These options do not trim CDATA. Pass nil for context if you only need to select textmode.

Return values

userdata or nil parser - New parser on success; nil if native parser allocation or INIT fails.
any error - On native parser allocation failure, a string message. On INIT failure, the second value returned by INIT, if any; its type is chosen by the callback. A caught INIT exception produces a string message.
any, ... results - On INIT failure, any callback results after the second value are also returned. Success returns only parser.

Throws

Throws if handlers is supplied without being a table, a recognized handler is neither a function nor nil, or textmode is not one of the supported strings. INIT exceptions are caught, logged through the BAS error handler, and returned as nil plus a string message.

Parser methods

In each method below, parser is the userdata returned by xparser.create(). The colon supplies this receiver as the first argument. Do not call parse, reset, or destroy on a parser from one of that same parser's callbacks. Read-only methods remain available inside callbacks.

parser:parse(data)

Process the next input chunk. Handlers run synchronously on the Lua stack that calls this method.

Parameters

string data - Required nonempty chunk of UTF-8 XML. A chunk may end in the middle of a tag or UTF-8 character.

Return values

boolean, boolean ok, complete - Normal result: true followed by whether the root element has closed. No parser-state string is returned.
nil, string ok, error - Failure result: nil and a message for malformed XML, empty input, or a parser error state that requires reset. A caught callback exception also uses this result shape and is logged.
any, ... results - If an XML event handler returns at least one non-nil value, parsing stops and all its return values are forwarded unchanged. A false value also stops parsing. Returning no values, or only nil values, lets parsing continue.

After a parsing error or callback interruption, reset the parser before using it for another document. Input after the interruption has not been consumed. Custom callback results can resemble the normal result shapes; choose distinguishable results if your caller needs to tell them apart.

Throws

Throws for an invalid or destroyed parser, a data argument that cannot be read as a Lua string, or a call from the same parser's callback. Malformed XML does not throw. Exceptions raised in handlers are caught and returned as described above.

parser:reset([full])

Reset the parser for another document, preserving the installed handlers, context, and textmode. If installed, RESET runs before the internal state is cleared.

Parameters

boolean full - Optional; default: false. True also releases internal buffers; false retains allocated buffers for reuse. Lua truthiness is used: nil and false select false, and other values select true. This value is not passed to RESET.

Return values

any, ... results - All RESET results if at least one is non-nil. No values if RESET is absent, returns nothing, or returns only nil values.
nil, string ok, error - If RESET raises an exception, it is caught and logged, and nil plus a message is returned. The internal parser state is still reset.

Throws

Throws for an invalid or destroyed parser, or a call from the same parser's callback. Exceptions in RESET are caught.

parser:destroy()

Release the native parser and call TERM if installed. Repeated calls are harmless. The same cleanup is used by the __gc and __close metamethods.

Parameters

None beyond the parser receiver.

Return values

None. TERM return values are discarded.

Throws

Throws for an invalid receiver or a call from the same parser's callback. An already destroyed parser does not cause an error. Exceptions in TERM are caught and logged; cleanup continues and destroy does not return the error.

parser:count()

Parameters

None beyond the parser receiver.

Return values

integer count - Input byte count since construction or reset. Initially 0. A recognized leading UTF-8 byte order mark is excluded. This is a byte count, not a Unicode character count.

Throws

Throws for an invalid or destroyed parser.

parser:line()

Parameters

None beyond the parser receiver.

Return values

integer line - Current input line counter, initially 1. CR, LF, and CRLF line endings advance it by one line.

Throws

Throws for an invalid or destroyed parser.

parser:col()

Parameters

None beyond the parser receiver.

Return values

integer column - Internal input column counter. Initially 1; processing an ordinary character increments it. LF sets it to 0, while CR sets it to 1; an LF following CR is skipped. Use it as a parser diagnostic rather than assuming a conventional source-editor column number.

Throws

Throws for an invalid or destroyed parser.

parser:depth()

Parameters

None beyond the parser receiver.

Return values

integer depth - Number of currently open nonempty elements, initially 0. The opening element is included during START_ELEMENT; the closing element has already been removed during END_ELEMENT. EMPTY_ELEMENT does not increase depth.

Throws

Throws for an invalid or destroyed parser.

parser:has_doc()

Parameters

None beyond the parser receiver.

Return values

boolean complete - Whether the parser has reached the end of the root element; initially false. This does not replace checking the result of parse for errors.

Throws

Throws for an invalid or destroyed parser.

tostring(parser)

Parameters

userdata parser - The parser object, which may already have been destroyed.

Return values

string description - Diagnostic text containing the userdata and native parser addresses. It is not serialized XML.

Throws

The parser's __tostring implementation throws if directly called with an invalid receiver. Converting a valid or destroyed parser does not report an operational error.

Parser state

The userdata has no documented writable fields. Use the methods above to read its state and supply application state through context. The binding does not expose init, get_state, get_pos, get_depth, get_context, or get_events methods. Use count, line, col, and depth for position and nesting information; create another parser to install different handlers or a different textmode.

Event handlers

All handlers are optional functions stored under the exact event names below. Each receives context as its first argument, even when context is nil. The parser object is not passed as an argument. Keep a reference in your context or closure if a handler needs to call a read-only parser method.

Callbacks run inside the initiating create, parse, reset, or destroy call and cannot yield. The binding catches and logs Lua errors raised by handlers, including attempts to yield or to call parse, reset, or destroy on the same parser. The initiating method determines how the caught error is returned, as documented above.

Event names and values

In the parameter tables, attributes is a new table containing string keys mapped to string values, plus integer keys 1 through n listing the attribute names in input order. qname is the element name as written, including any namespace prefix. All text and name arguments are Lua strings.

Lifecycle handlers

INIT(context)

Called during construction, before parsing.

Parameters

any context - Context supplied to create; default: nil. No additional create arguments are forwarded.

Return values

any context - A truthy first result replaces the supplied context. Returning no values or only nil values leaves the supplied context unchanged.
nil or boolean, any, any ... ok, error, results - A false or nil first result with at least one non-nil result rejects initialization. create replaces the first result with nil and forwards any remaining results. Return false and a string message to report a failure.

Throws

An exception raised by your handler is caught and logged; create returns nil and a string message.

RESET(context)

Called by reset before parser state is cleared. The full argument is not forwarded.

Parameters

any context - Current application context; nil if none was supplied or installed by INIT.

Return values

any, ... results - All results are forwarded by reset if at least one is non-nil. No results or only nil results produce no return values. A returned context does not replace the parser's context.

Throws

An exception raised by your handler is caught and logged; reset returns nil and a string message and still clears the parser state.

TERM(context)

Called during destruction, including automatic cleanup. Use it to release resources held by your context.

Parameters

any context - Current application context; nil if none was supplied or installed by INIT.

Return values

any, ... results - Ignored by destroy and automatic cleanup. Return no values unless your own code needs them when calling the handler directly.

Throws

An exception raised by your handler is caught and logged. Destruction continues; no error value is returned.

XML handlers

START(context)

Called before consuming the first character after construction or reset. It receives no attributes argument.

Parameters

any context - Current application context; nil if none was supplied or installed by INIT.

Return values

any, ... results - Returning no values or only nil values continues parsing. If any result is non-nil, including false, parse stops and forwards all results unchanged.

Throws

An exception raised by your handler is caught and logged; parse returns nil and a string message.

XML(context, attributes)

Called for an XML declaration.

Parameters

any context - Current application context; nil if none was supplied or installed by INIT.
table attributes - Attribute values and ordered names, as described above.

Return values

any, ... results - Returning no values or only nil values continues parsing. If any result is non-nil, including false, parse stops and forwards all results unchanged.

Throws

An exception raised by your handler is caught and logged; parse returns nil and a string message.

START_ELEMENT(context, qname, attributes)

Called after a nonempty opening tag.

Parameters

any context - Current application context; nil if none was supplied or installed by INIT.
string qname - Element name, including any namespace prefix.
table attributes - Attribute values and ordered names, as described above.

Return values

any, ... results - Returning no values or only nil values continues parsing. If any result is non-nil, including false, parse stops and forwards all results unchanged.

Throws

An exception raised by your handler is caught and logged; parse returns nil and a string message.

END_ELEMENT(context, qname)

Called after a closing tag.

Parameters

any context - Current application context; nil if none was supplied or installed by INIT.
string qname - Element name, including any namespace prefix.

Return values

any, ... results - Returning no values or only nil values continues parsing. If any result is non-nil, including false, parse stops and forwards all results unchanged.

Throws

An exception raised by your handler is caught and logged; parse returns nil and a string message.

EMPTY_ELEMENT(context, qname, attributes)

Called for a self-closing tag such as <item/>. It replaces separate START_ELEMENT and END_ELEMENT calls for that tag.

Parameters

any context - Current application context; nil if none was supplied or installed by INIT.
string qname - Element name, including any namespace prefix.
table attributes - Attribute values and ordered names, as described above.

Return values

any, ... results - Returning no values or only nil values continues parsing. If any result is non-nil, including false, parse stops and forwards all results unchanged.

Throws

An exception raised by your handler is caught and logged; parse returns nil and a string message.

TEXT(context, text)

Called for text between tags. A single element can produce several text events when other content separates its text.

Parameters

any context - Current application context; nil if none was supplied or installed by INIT.
string text - Text content, with entity references decoded and textmode applied.

Return values

any, ... results - Returning no values or only nil values continues parsing. If any result is non-nil, including false, parse stops and forwards all results unchanged.

Throws

An exception raised by your handler is caught and logged; parse returns nil and a string message.

CDATA(context, text)

Called for a CDATA section.

Parameters

any context - Current application context; nil if none was supplied or installed by INIT.
string text - CDATA content without the <![CDATA[ and ]]> delimiters.

Return values

any, ... results - Returning no values or only nil values continues parsing. If any result is non-nil, including false, parse stops and forwards all results unchanged.

Throws

An exception raised by your handler is caught and logged; parse returns nil and a string message.

COMMENT(context, text)

Called for an XML comment.

Parameters

any context - Current application context; nil if none was supplied or installed by INIT.
string text - Comment content without the  delimiters.

Return values

any, ... results - Returning no values or only nil values continues parsing. If any result is non-nil, including false, parse stops and forwards all results unchanged.

Throws

An exception raised by your handler is caught and logged; parse returns nil and a string message.

PI(context, name, body)

Called for a processing instruction other than the XML declaration.

Parameters

any context - Current application context; nil if none was supplied or installed by INIT.
string name - Processing instruction target.
string body - Processing instruction data.

Return values

any, ... results - Returning no values or only nil values continues parsing. If any result is non-nil, including false, parse stops and forwards all results unchanged.

Throws

An exception raised by your handler is caught and logged; parse returns nil and a string message.

Sample event handlers

Read a document in chunks

-- Keep application state in context; callback arguments do not include the parser.
local context = {names = {}}
local parser, err = xparser.create({
   START_ELEMENT = function(ctx, name)
      ctx.names[#ctx.names + 1] = name
   end,
   EMPTY_ELEMENT = function(ctx, name)
      ctx.names[#ctx.names + 1] = name
   end
}, context)
assert(parser, err) -- This example treats initialization failure as fatal.

local complete = false
for _, chunk in ipairs({"<root>", "<item/></root>"}) do
   local ok, result = parser:parse(chunk)
   if not ok then
      parser:destroy()
      error(result) -- The caller chooses to throw; parse returned the error.
   end
   complete = result
end
parser:destroy()
assert(complete, "Incomplete XML input")
print(table.concat(context.names, ", ")) -- root, item

Convert XML to a table

The xml2table Lua module provides a handler table for xparser. It builds a document table and returns "DONE", document when the root closes. It is separate from the C binding and must be available in the installed Lua resources.

Parameters

Use the module table as the handlers argument to xparser.create. The module's INIT handler creates a context table when context is nil; an explicitly supplied context must be a table.

Return values

string, table status, document - "DONE" and the document table when the root closes. Parsing stops at that event, so trailing input in the chunk is not checked.
boolean, boolean ok, complete - The underlying parser returns true, false when a chunk is consumed and more input is needed.
string or nil, string status, error - "ERROR" and a message for a namespace error reported by the module; nil and a message for an underlying parse failure or caught callback exception.

Throws

The parse method has the same programmer-error checks as parser:parse. Module callback exceptions are caught by the binding. Loading the module with require can throw if the module is unavailable.

The completed document and element tables have the following fields:

table or nil document.xml - XML declaration attributes, if present.
table document[index], element[index] - Child nodes in input order, using integer indices starting at 1. Document nodes may include comments and processing instructions before the root.
table or nil elements - Present when the document or element has child elements. Integer keys list child elements; name keys reference the first child with that name. Expanded names may also be keys.
string type - "element" for elements; "TEXT", "CDATA", "COMMENT", or "PI" for other nodes.
string tag_name, local_name, expanded_name - Element name as written, local name, and the module's expanded-name lookup key.
string or nil namespace_prefix, namespace_ref - Explicit namespace prefix and its resolved reference; absent for an unprefixed element.
table attributes - Element attributes, with string values and integer entries listing attribute names.
table namespaces - Namespace mapping retained for the element.
integer sequence - Number of preceding sibling nodes at insertion, starting at 0.
string or nil text - For a completed element, its direct TEXT segments joined by a space. CDATA is stored separately as child nodes.
string value - Content of a TEXT, CDATA, COMMENT, or PI node.
string target - PI node target.

-- The module supplies callbacks; no global xml2table variable is installed.
local handlers = require("xml2table")
local parser, err = xparser.create(handlers)
assert(parser, err)
local status, document = parser:parse("<root><item>value</item></root>")
parser:destroy()
assert(status == "DONE", document)
print(document.elements.root.elements.item.text) -- value

Lua libraries required

The xparser binding is registered by BAS. The optional xml2table module uses the standard string and table libraries.