Class DataTree

Validate compliance of top-node. To be provided by derived classes for specific syntaxes.

Args:

value:

Top node.

kargs:

Specific syntax related dynamic parameters to be defined by derived classes.

Returns:

None

Results:

YapyDataTreeError:

Raises exception when not valid.

Decorators:

Args:

data:

Configuration data in accordance to the selected data language syntax. The default is the Python syntax including the appropriate data types. This may impose additional constraints by derived classes e.g. in case of persistent data such as JSON and XML - see other classes within this module.

The default Python DL implementation supports in-memory access only, while persistence will be available for example by pickling.

The initial data defines the permitted type of the first item within the subpath of the spanned data tree. The default value acts here as a placeholder for an empty structure, which could be defined by following extension operations arbitrarily.

The basic constraint introduced here is that intermediate nodes require support of subscription. This is due to the addressing concepts implemented by DataTree. Thus even though a set could technically be an intermediate node, it could not be indexed, thus could not be addressed by the standard member functions. Resulting 'set' and 'frozenset' are supported by DataTree as end-nodes only, same as atomic data types.

Anyhow, the <top-node> is by default permitted to be an end-node. Thus the context defines the applicability dynamically.

The consistency of the data tree including the valid intermediate nodes is verified by access, so basically within the responsibility of the caller.

Returns:

None / initialized object

Raises:

YapyDataDataTreeError

pass-through

Decorators:

Readout the value of a node, or an attribute. The name binding of the path is provided as a tuple of path items.

Args:

subpath:

The list of keys constituting a branch of a data tree. The subpath is treated as a branch of one of the nodes of a provided searchpath - which is by default the top node. The supported values are:

subpath := <list-of-node-ids>
<list-of-node-ids> := <node-id> [',' <list-of-node-ids>]
node-id := (
      str            # string:  dict
    | int            # integer: lists, tuple, dict
)

The value of the node within data:

nodeid := (
      <single-nodeid>
    | <list-of-nodeids>
    | <tuple-of-nodeids>
)
single-nodeid := <nodeid>
list-of-nodeids := '[' <nodeidlists> ']'
tuple-of-nodeids := '(' <nodeidlists> ')'
nodeidlists := <nodeid> [',' <nodeidlists>]
nodeid := (
      ItemKey
    | ListIndex
)
ItemKey := "valid dict-key"
ListIndex := "valid list-index"

The derived syntax classes may impose specific constraints. Thus it is recommended to use integers and strings only for maximum compatibility, and the ease of using mixed syntaxes:

ItemKey :=    str  # string:  dict
ListIndex :=  int  # integer: lists, tuple, dict

kargs:

searchpath:

Optional search path for the match of the provided address subpath. The provided subpath is applied to each node of the searchpath in accordance to the direction option. This provides the search and enumeration of side branches:

searchpath := <path-item-list>

path-item-list := <path-item> [, <path-item-list>]
path-item := (
      str  # item name
    | int  # item index
)

default := <top-node>

The search path entries has to be actually present by default. These could be either created by loading a complete tree structure, or by using the Capabilities.create() member. See also parameter 'strict'.

direction:

The search direction of the subpath within the searchpath. In case of multiple superpositioned attributes the first traversed match.

The provided values are:

direction := (
      up   | 0  | False # search from right-to-left
    | down | 1  | True  # search from left-to-right
)

default:= up

match:

Sets the match criteria for the search operation. Interferes with direction:

match := (
      M_FIRST | 'first'   # use first matching node
    | M_LAST  | 'last'    # use last matching node
    | M_ALL   | 'all'     # use all - iterate all matches
)

default := M_FIRST

partial:

Enables the return of partial sub paths in case the requested path is not completely present.

partial := (
      True   # when not completely present, the longest
             # existing part is returned, the completeness
             # is provided by the result attribute <partial>
    | False  # when not completely present an exception
             # is raised
)

strict:

Controls the required consistency. This comprises:

1. the presence of the search path entries

2. the presence of the requested subpath within the set of search paths

pysyn:

Activates full Python syntax. This in particular enables all container types of intermediate nodes for arbitrary paths. Includes tuple, set, frozenset, etc.

pysyn := (
      True   # allows all python types as container nodes
    | False  # allows list and dict only as container nodes
)

default := False

Returns:

In case of a match returns the tuple:

return := (<attr-value-path>, <attr-value>, <partial>)

attr-value-path := (
      "the list of keys of the top-down path"
    | "empty list when no item exists"        # see <complete>
)
attr-value := "value of the targeted node/attribute"
partial := (
      False   # the complete requested path
    | True    # the actually present part of the path
)

Else raises YapyDataDataTreeOidError.

Raises:

YapyDataDataTreeOidError

pass-through

Decorators:

Creates a subpath to a given node, default is from top. Reuses existing nodes, starts the creation at the first point of branch-out from the exiting tree.

In general no padding of pre-required entries is done. This e.g. requires in case of a list the start with the index 0, while in case of the dict arbitrary keys could be assigned.

Args:

subpath:

The list of keys constituting a branch of a data tree. The subpath is treated as a branch of one of the nodes of a provided searchpath - which is by default the top node. The supported values are:

subpath := <list-of-node-ids>
<list-of-node-ids> := <node-id> [',' <list-of-node-ids>]
node-id := (
      <present-intermediate-node>
    | <new-node>
)
present-intermediate-node := (
      str             # string:  dict
    | int             # integer: lists, tuple, dict
    | True | False    # logic:   dict
    | None            # null:    dict
)
new-node := (
      str             # string:  dict
    | int             # integer: list
    | True | False    # logic:   dict
    | None            # null:    dict
)

Some Python data types are immutable, which could be subscripted read-only, e.g. strings. While others such as sets are iterable, but not subscriptable at all. Refer to the manual for a detailed list.

kargs:

hook:

Optional node as parent of the insertion point for the new sub path. The node must exist and be part of the targeted data structure. No additional checks are done:

hook := <memory-node>
memory-node := "node address"

default := <top-node>

The hook node could either be a border node of the existing tree, or any arbitrary node with a partial of complete part of the requested subpath. Existing nodes are reused.

strict:

If True requires all present nodes of the subpath to of the appropriate type, missing are created. When False present nodes of inappropriate type are simply replaced.

strict := (
      True   # nodes must be present
    | False  # missing are created
)
default := False

Nodes of type None are always treated as non-present placeholder, thus replaced in any case.

value:

Value of the created final node:

value := <value-of-node>

value-of-node := <valid-json-node-type>
valid-json-node-type := (
                      int | float
                    | str                  # unicode
                    | dict | list
                    | None | True | False  # equivalent: null|true|false
                    )

default := None

Returns:

In case of success the in-memory nodes of the sub path:

return := (<attr-value-path>)

attr-value-path-tuple := (
      <in-memory nodes>
    | <non-subscriptable-node>
)
in-memory nodes := (
    "the list of in-memory nodes with support of subscription"
)
non-subscriptable-node := "any valid type"

else raises YapyDataDataTreeOidError.

The last node contains in case of an atomic type the value of the node, while the intermediate nodes represent the indexed containers.

Raises:

YapyDataDataTreeOidError

pass-through

Decorators:

Gets the value of the path within the member 'data':

self.data[key]
self.data[key0][key1][key2]...

Args:

key:

The value of the node within data:

key := (
      <single-key>
    | <list-of-keys>
    | <tuple-of-keys>
)
single-key := <key>
list-of-keys := '[' <keylists> ']'
tuple-of-keys := '(' <keylist> ')'
keylist := <key> [',' <keylist>]
key := (
      ItemKey
    | ListIndex
)
ItemKey := "valid dict-key"
ListIndex := "valid list-index"

Returns:

The value of the addressed node/value.

Raises:

pass-through

Decorators:

The core class *DataTree *does not support serialization.

For serialization use either a derived syntax class, or 
serialize it e..g. by pickling and use the in-memory data.
For example by pickling:: 

    def import_data(self, fpname, key=None, node=None, **kargs):
        if not os.path.isfile(fpname):
            if not os.path.isfile(fpname + '.pkl'):
                raise YapyDataTreeError(
                    "Missing file: " + str(fpname))
            else:
                fpname = fpname + '.pkl'
         
        if key and node == None:
            raise YapyDataTreeError(
                "Given key(%s) requires a valid node." % (str(key)))
         
        datafile = os.path.abspath(fpname)
        with open(datafile, 'rb') as data_file:
            pval = pickle.load(data_file)
         
        if key:
            node[key] = pval
        else:
            self.data = pval
         
        return pval

    See manuals for security issues of pickling.

Decorators:

Superposes a JSON structure onto an existing. This is a fixed mode and strategy special case of the generic method superpose(). Implemented by recursion. The reduced parameter set provides better performance on large trees while the graph parameters still could be efficiently set by default values.

The superpositioning is supported by multiple strategies defined by the parameter mode. The provided algorithm of the strategy is join, where the input data is processed on the exisiting tree by modification and creation as required.

• branches are resolved from top to the leafs

• missing sub-branches are created

• missing leafs are created

• existing leafs are replaced

This implements a last-wins strategy, thus in particular supports incremental load of configurations by raising priority.

Args:

data:

Top node of the data tree to be superposed.

keyidx:

Key or index to be used at the insertion node. If not given the insertion node is:

into dict: update by the new node
into list: append the new node

default := None

hook:

The insertion node for the new data:

when  not given: use top.

default := None

Returns:

Returns the merged data tree, raises an exception in case of failure.

Raises:

YapyDataDataTreeError

pass-through

Decorators:

match_map

Value:

{M_FIRST: 1, M_LAST: 2, M_ALL: 3, 'first': 1, 'last': 2, 'all': 3,}