Module - Metadata Handler (Metadata Handler)

This module takes care of reading & writing meta data and meta data profiles. Further it also defines the XML schemata and security rules bound to the data packages.

Meta Data

Meta data packages are based on XML and implement a type safe binding to supporting programming languages using an XML schema. Meta data is build on a simple key and value pair binding, where the key (name) is always unique and may reference a single to multiple values of the same type.

Note: It is not possible to mix multiple types. One meta data element, identified by a unique name may only have one to many values of the same type.

The XML schema defines 5 major types to allow the storage of any data:

  • String, one-to-many values
  • Boolean, one-to-many values
  • Numeric, one-to-many values, accuracy is 64bit singed floating point (xs:double)
  • Date, one-to-many values, accuracy is at milliseconds, timezone is included
  • Binary, one-to-many bytes

Schema and Examples:

  • Metadata Schema:
  • Plain example
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    <metadata xmlns="http://grid.trendmicro.com/metadata/1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://grid.trendmicro.com/metadata/1.0 metadata.xsd">
        <meta name="String" value="value"/>
        <meta name="Strings">
            <v>valueA</v>
            <v>valueB</v>
        </meta>
        <meta booleanValues="true" name="Boolean"/>
        <meta booleanValues="true false false true" name="Booleans"/>
        <meta name="Numeric" numericValues="1.423"/>
        <meta name="Numerics" numericValues="1.0 2.0 3.0"/>
        <meta dateValues="2014-02-10T01:04:33.614-08:00" name="Date"/>
        <meta dateValues="2014-02-10T01:04:33.614-08:00 2014-02-10T01:04:33.614-08:00" name="Dates"/>
        <meta binaryValue="AQID" name="Binary"/>
    </metadata>
  • Realworld example
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    <metadata xmlns="http://grid.trendmicro.com/metadata/1.0">
        <meta name="originalFileName" value="WinWord.exe"/>
        <meta name="fileVendor" value="Microsoft Corp."/>
        <meta name="fileVersion" value="12.0.6504.5000"/>
        <meta name="buildNumber" numericValues="3410.0"/>
        <meta dateValues="2014-02-10T01:04:34.739-08:00" name="buildDate"/>
        <meta binaryValue="FnDeXZrZEUwCekPyhvE40lukwUhNCT/gjACyllTi5MA=" name="sha256"/>
        <meta binaryValue="JJjS0oGiAGjGngbG0oh5w+Pdzdjq2x/6iumYl9oZdhqeTUNv6g8Rnj/xmUHUun1O0iHIk2RfSJ8aLMt6gDM4/Q==" name="sha512"/>
        <meta name="scanVendors">
            <v>trendmicro</v>
            <v>symantec</v>
            <v>sophos</v>
        </meta>
        <meta booleanValues="true true true" name="scanPassed"/>
    </metadata>

Meta Data Profiles

Meta data profiles are also described in a machine readable format using XML allowing to validate the meta data packages in software. Furthermore it's possible to enforce access restrictions in relation to the used interface types.

Usage inside the ACL

From version 1.1 the ACL uses a built-in metadata profiles document to validate metadata and enforce access restrictions in relation to the invoked interfaces. (This implementation is part of the feature "Public query for Metadata")

The built-in default is automatically dumped to "config/default-metadata-profiles.xml" when the ACL starts and can be adjusted to suite the needs of the GRID installation.

In order to adjust the applied profiles, rename this file to "config/metadata-profiles.xml" and modify the values as needed. On the next service restart, the ACL will use the settings of this document instead of the built-in defaults.

Concepts

  • All profiles are stored in a single XML document and are identified by a document wide unique name.
  • Profiles may depend on one-to-many parents and inherit all definitions from them.
  • Every profile contains one or more entries that describe a meta element of a metadata package. The binding is made on the meta element's name only.
  • Every entry describes:
    • To what meta element it is bound.
    • What the purpose of the element is (a human readable description).
    • What data types may be used by the element.
    • Who requires and who produces the element.
    • What user roles are required to access an element.
    • What protection level is applied (PRIVATE = site local, PROTECTED = TM network, PUBLIC = internet)

Structure

  • Schema:
  • HTML Conversion Stylesheet:
  • Simple example
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    <?xml-stylesheet type="text/xsl" href="metadata-profile-to-html.xsl"?>
    <ns2:metadata-profiles xmlns:ns2="http://grid.trendmicro.com/metadata-profile/1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://grid.trendmicro.com/metadata-profile/1.0 profiles-document.xsd">
        <profile name="basic.profile-level0">
            <description xsi:nil="true"/>
            <entry accessLevel="PUBLIC" metaName="originalFileName" readonly="false" validType="STRING">
                <description>Stores the original filename as it was set inside the file content properties.</description>
            </entry>
            <entry accessLevel="PUBLIC" metaName="fileVendor" readonly="false" validType="STRING">
                <description>Stores the vendor string as it was set inside the file content properties.</description>
            </entry>
            <entry accessLevel="PUBLIC" metaName="fileVersion" readonly="false" validType="STRING">
                <description>Stores the file version as it was set inside the file content properties.</description>
            </entry>
            <entry accessLevel="PUBLIC" metaName="buildNumber" readonly="false" validType="NUMERIC">
                <description>Stores the build number as it was set inside the file content properties.</description>
            </entry>
            <entry accessLevel="PUBLIC" metaName="buildDate" readonly="false" validType="DATE">
                <description>Stores the build date as it was set inside the file content properties.</description>
            </entry>
            <entry accessLevel="PUBLIC" metaName="sha256" readonly="false" validType="BINARY">
                <description>Stored the sha256 hash of the file content.</description>
            </entry>
            <entry accessLevel="PUBLIC" metaName="sha512" readonly="false" validType="BINARY">
                <description>Stored the sha512 hash of the file content.</description>
            </entry>
        </profile>
        <profile name="processing.profile-level0">
            <description xsi:nil="true"/>
            <parentProfile>basic.profile-level0</parentProfile>
            <entry accessLevel="PROTECTED" metaName="scanVendors" readonly="false" validType="STRING">
                <requiredBy>app-intelligence</requiredBy>
                <requiredBy>tm-internal-services</requiredBy>
                <providedBy>pafi</providedBy>
                <requiredRole>internal-process</requiredRole>
                <requiredRole>internal-appint-process</requiredRole>
                <description>Lists the scan vendors that were used to check the file content.</description>
            </entry>
            <entry accessLevel="PROTECTED" metaName="scanPassed" readonly="false" validType="BOOLEAN">
                <requiredBy>app-intelligence</requiredBy>
                <providedBy>pafi</providedBy>
                <description>Maps the scan results for this file content against the listed 'scanVendors'.</description>
                <comment>Comment 01</comment>
                <comment>Comment 02</comment>
                <comment>Comment 03</comment>
            </entry>
        </profile>
    </ns2:metadata-profiles>