Using the generateDS API -- Guidance, hints, clues, and suggestions

Contents

1 Introduction
2 The API and its methods
3 Additional hints and suggestions

1 Introduction

More information:

There is additional information on this topic in the generateDS online documentation: http://www.davekuhlman.org/generateDS.html#how-to-use-the-generated-source-code
The documentation on generateDS is here: http://www.davekuhlman.org/generateDS.html
The generateDS repository is here: https://sourceforge.net/projects/generateds/

You can find the generateDS distribution here:

generateDS reads an XML schema and generates Python source code from it. Importantly, it generates a Python class for each xs:complexType defined in the schema. Those Python classes and the methods in them are, once you have looked at a few, reasonably predictable and can be considered to form an API (application programming interface) for the programmatic use of that generated code.

2.1 What to look for

The constructor (ctor) -- __init__
The export methods -- export, exportAttributes, exportChildren
The build methods -- build, buildAttributes, buildChildren
The get and set methods. And, if the element/child was defined with maxOccurs="unbounded", add and insert methods.
If the module was generated (by generateDS) with --member-specs="dict" or --member-specs="list", then look for member_data_items_. It will be a class variable in each class that was generated from a complexType. It will be either a dictionary or a list. It describes the attributes and children of the xs:complexType from which this class was generated. You may be find it helpful if you need to programmatically determine the type of a child or attribute. See class MemberSpec_ in the generated code for information about the items in this dictionary or list.

2.2 The constructor

The constructor (or "ctor") initializes each attribute and each child in a generated class. So, you can look at the ctor to see (1) the signature for the constructor, (2) a list of the attributes and children, and (3) how each item is intialized.

Here is a sample script that uses the ctor to construct an instance of one of the generated classes and then display/export it:

[ins] In [19]: import mexconfiglib as lib
[ins] In [20]: configuration = lib.configuration()
[ins] In [21]: configuration.export(sys.stdout, 0)
<configuration xmlns:tns="http://mcuxpresso.nxp.com/XSD/mex_configuration_1.8" xmlns:None="http://www.w3.org/2001/XMLSchema" />
[ins] In [22]: tools = lib.toolsType()
[ins] In [23]: configuration.set_tools(tools)
[ins] In [24]: configuration.export(sys.stdout, 0)
<configuration xmlns:tns="http://mcuxpresso.nxp.com/XSD/mex_configuration_1.8" xmlns:None="http://www.w3.org/2001/XMLSchema" >
        <tools/>
</configuration>

Notes:

In the above example, an instance of generated class configuration contains an instance of generated class toolsType.

The parameters to the constructor include the attributes and children of the class (xs:complexType). These parameters have default values (mostly None). So, we can also do:

[ins] In [28]: configuration = lib.configuration(
          ...:     tools=lib.toolsType(), version="1.23")
[ins] In [29]: configuration.export(sys.stdout, 0)
<configuration xmlns:tns="http://mcuxpresso.nxp.com/XSD/mex_configuration_1.8" xmlns:None="http://www.w3.org/2001/XMLSchema"  version="1.23">
    <tools/>
</configuration>

2.3 The get and set methods etc

These are very simple access methods for the attributes and children of a class generated from a xs:complexType. Their names are of the form "get_xxxx" and "set_xxxx", where "xxxx" is the name of the attribute or child to which they give access.

If the child was declared in the schema with maxOccurs="N" with N > 1 or with maxOccurs="unbounded", then there will also be methods named "add_xxxx", "insert_xxxx", and "replace_xxxx" (where "xxxx" is the name of the child.

2.4 Validator methods

If an attribute or child is declared as being an instance of a xs:simpleType and that xs:simpleType is defined in the schema, then there will be validator methods for that attribute or child.

If you include "validate" in the list of export options when you run generateDS.py, for example:

./generateDS.py -o mylib.py --export="write validate" Schema/myschema.xsd

Then, you will also find a validate_ method in the generated class that performs validation on attributes and children that support it.

2.5 The `export` methods

You can use this method to write out the XML content that corresponds to the model that you have constructed. Examples:

configuration.export(sys.stdout, 0)

# Or

with open("content.xml", "w") as outfile:
    configuration.export(outfile, 0, namespaceprefix_="tns:")

2.6 The `build` methods

Even if you do not intend to these methods, you are likely to want to look an the code. It contains examples that show how to create each of the attributes and children of the complexType that it was generated from. Here is an example:

def buildAttributes(self, node, attrs, already_processed):
    value = find_attr_value_('class', node)
    if value is not None and 'class' not in already_processed:
        already_processed.add('class')
        self.class_ = value
def buildChildren(self, child_, node, nodeName_, fromsubclass_=False, gds_collector_=None):
    if nodeName_ == 'dependency':
        obj_ = dependencyType.factory(parent_object_=self)
        obj_.build(child_, gds_collector_=gds_collector_)
        self.dependency.append(obj_)
        obj_.original_tagname_ = 'dependency'

Notes:

Looking in the XML schema from which this was generated I see that class is an attribute declared to be of type xs:string.
The child dependency is declared as an anonymous xs:complexType. Since that anonymous xs:complexType is nested inside a child element named "dependency", generateDS gives the class generated from that anonymous xs:complexType the name "dependencyType". Note that if there are multiple anonymous types nested inside different elements with the same name, then there may be a number appended to the class name in order to make it unique, for example "descriptionType" and "descriptionType4". See below: Generated class names and anonymous xs:complexTypes.

3 Additional hints and suggestions

3.1 IPython and exploring the API

IPython provides convenient ways to explore a generated module and the API of the generated classes in it. Appending "?" or "??" to a name is especially helpful. IPython provides an easy way to edit source code with the %edit magic operator. At the IPython interactive prompt, type "%edit?" for help.

For examples, try a few of the following:

In [11]: # import the generated module
In [12]: import mygeneratedmodule as lib
In [13]: # show the signature of the ctor of class `dependencies`
In [14]: lib.dependencies?
In [15]: # show the source code for class `dependencies`
In [16]: lib.dependencies??
In [17]: # show the source code for the constructor only
In [18]: lib.dependencies.__init__??
In [19]: # edit the ``buildChildren`` method.  -x says don't execute on exit
In [21]: %edit -x lib.dependencies.buildChildren

Also consider using the Jupyter notebook, especially if you have installed the Anaconda distribution of Python. See: https://docs.anaconda.com/ae-notebooks/4.2.2/user-guide/basic-tasks/apps/jupyter/

3.2 Generated class names and anonymous xs:complexTypes

When an xs:complexType is defined at top level and has a name (via the name attribute), the class generated by generateDS will be the same as the name of the xs:complexType.

When the xs:complexType is anonymous (that is it has no name attribute and it is nested inside an xs:element declaration), then the class name is constructed from the name of the enclosing xs:element declaration. Usually, it has the characters "Type" appended. Examples: "heightType", "sizeType".

When there are multiple anonymous xs:complexType definitions inside xs:element declarations with the same name, then a number is appended to form a unique name. Examples: "sizeType", "sizeType4", "sizeType7". Compare the names of the attributes and children in the generated class with those declared in the xs:complexType to determine which of these similarly named classes is the one you want. It might help to generate your module with command line option --member-specs="dict" or --member-specs="list", then look in the relevant classes for member_data_items_.

For a quick look at the names of the generated classes, take a look at the __all__ global variable. Example:

>>> import plants01lib as lib
>>> print(lib.__all__)

3.3 Using the generated parse methods

In some use cases you may want to (1) read an existing XML file, (2) parse it, (3) modify it, and (4) then write it out.

But, you might want to ensure that your XML instance document validates against the XML schema. Note: This may not be true if you XML document contains a fragment of the XML tree. One way to valid the document is to use xmlling. And example:

$ xmllint --schema myxmlschema.xsd myxmldoc.xml

Note that lxml, which is used by generateDS, also can do validation. So, you could validation your document against a schema from within your Python code. See https://lxml.de/validation.html. Here is an example:

from lxml import etree

def test():
    schema = etree.XMLSchema(file='plants.xsd')
    parser = etree.XMLParser(schema=schema)
    etree.parse('plants01.xml', parser=parser)

And, you can find an example of using a module generated by generateDS to parse, modify, and write out an XML file here: plants.zip.

Here are the files needed in this example:

plants.xsd -- The XML schema.
plants01.xml -- The XML instance document that we use for our initial input.
plants_update01.py -- The Python script that does the work.

Here are instructions on how to use it:

Generate the module with something like the following:

$ python ./generateDS.py -o plants01lib.py --member-specs=dict '--export=write etree validate' plants.xsd

Run the script with any of the following:

# get help
$ python plants_update01.py --help
# write to stdout
$ python plants_update01.py plants01.xml
$ python plants_update01.py -v plants01.xml
# write to a file.
$ python plants_update01.py -o updated.xml plants01.xml
$ python plants_update01.py -f -o updated.xml plants01.xml

Notes on the code/script (plants_update01.py):

The parse function in this example was copied from the module generated by generateDS. Then the rootTag and rootClass were modified to correspond to our input XML document. The references to the generateDS module imported as gdslib were added to refer to items in the module from which the parse function had been copied.
In function parse_and_update we (1) take the root object; (2) find the fruits and vegetables objects; (3) create a new fruit and a new vegetable; and (4) add the new fruit to fruits and the new vegetable to vegetables. Then we "export" it.
Notice that the names of the classes from which we create our new fruit and vegetable are named fruitType and vegetableType, even though the xs:complexType from which they were generated were nested inside element declarations named fruit and vegetable. This happens because generateDS needs to generate a unique name for the class. And, in some cases, where there are duplicates, a number will be appended to the name.