ldproxy is a software tool to share spatial data according to the W3C/OGC Spatial Data on the Web Best Practices and the draft OGC Web Feature Service standard. ldproxy 1.3.0 supports two data sources: OGC Web Feature Services 2.0 and PostgreSQL/PostGIS databases (PostgreSQL versions 9.5, 9.6, 10.5; PostGIS version 2.4). The ldproxy configuration for a dataset is internally stored in a set of JSON documents.

This ShapeChange target generates ldproxy configurations for data according to one or more application schemas. Currently only PostgreSQL/PostGIS databases are supported at the backend.

Description

Overview

The configuration depends on two aspects:

how the feature types of the application schema(s) are mapped to the database;
how you want the data to appear to users of the API published by ldproxy.

The configuration can be influenced using conversion rules and parameters that are described below.

The following configuration directories and JSON files are created. The JSON files are underlined. The directories and files in red are not created by ShapeChange and have to be created or edited by the administrator as described below.

config-store
- entities
  - services
    - xyz [the serviceId, see below]
    - #overrides#
      - xyz [the same serviceId]
  - codelists
    - codelist1 [the name of the code list classifier]
    - codelist2
    - …
- settings
  - ldproxy-target-geojson
    - #overrides#
      - GeoJsonConfig
  - ldproxy-target-gml
    - #overrides#
      - GmlConfig
  - xtraplatform-server
    - CoreServerConfig
  - xtraplatform-auth-external
    - #overrides#
      - ExternalAuthConfig

The service override configuration file

The additional service override (xyz) is a copy of the file with the same name created by ShapeChange. Unchanged JSON members can be removed. The service override at least has to contain the database connection info (JSON path “$.featureProvider.connectionInfo.*”) and the coordinate reference system information (JSON path “$.featureProvider.nativeCrs”). For example:

{
  "id": "xyz",
  "featureProvider": {
    "connectionInfo": {
      "host": "172.17.0.1",
      "database": "postgres",
      "user": "postgres",
      "password": "eHl6Cg=="
    },
    "nativeCrs": {
      "code": 25832
    },
    "providerType": "PGIS"
  }
}

The value of “$.featureProvider.connectionInfo.host” has to be the hostname that is reachable from the ldproxy docker container (see below). In a local test environment where the database is on the same machine, this would typically be “172.17.0.1” (except on macOS, where it is “host.docker.internal”).

The value of “$.featureProvider.connectionInfo.password” is the base64 encoded password.

The value of “$.featureProvider.nativeCrs.code” is the integer EPSG code of the coordinate reference system of the geometries in the database. 25832 from the example above is ETRS89 / UTM zone 32N.

In addition, the spatial extents for the feature types in the dataset should be set. The coordinate reference system is WGS 84 longitude/latitude. For feature types with temporal properties the temporal extent should be set, too.

The default values set by ShapeChange are the whole Earth as the spatial extent (-180° to 180° longitude, -90° to 90° latitude) and unlimited temporal extent (indicated by null values). To change this, set the values of “$.featureTypes.{featureTypeId}.extent.spatial.*” and “$.featureTypes.{featureTypeId}.extent.temporal.*”

{
  "id": "xyz",
  "featureProvider": { ...
  },
  "featureTypes": {
    "afeaturetype": {
      "id": "afeaturetype",
      "label": "A Feature Type",
      "description": "A description of the feature type",
      "extent": {
        "spatial": {
          "xmin": 6.1173598760,
          "ymin": 48.9662745077,
          "xmax": 8.5084754437,
          "ymax": 50.9404435711
        },
        "temporal": {
          "start": "2010-01-01T00:00:00Z",
          "end": null
        }
      } 
    },
    "anotherfeaturetype": {
      "id": "anotherfeaturetype",
      "label": "Another Feature Type",
      "description": "A description of the feature type",
      "extent": {
        "spatial": {
          "xmin": 6.1173598760,
          "ymin": 48.9662745077,
          "xmax": 8.5084754437,
          "ymax": 50.9404435711
        },
       "temporal": {
         "start": "2010-01-01T00:00:00Z",
         "end": null
       }
     } 
   },
   ...
  }
}

The security configuration file (optional)

If the ldproxy deployment includes services with access control (see the parameter “secured”), the configuration file ExternalAuthConfig has to be provided, too.

See the ldproxy documentation for details.

Example:

{
  "jwtSigningKey" : "eW91ci0yNTYtYml0LXNlY3JldA==",
  "externalDynamicAuthorizationEndpoint": "http://127.0.0.1:10010/authorize",
  "postProcessingEndpoint": "http://127.0.0.1:10010/postprocess"
}

The configuration has to contain either a “$.userInfoEndpoint” or a “$.jwtSigningKey” member for token validation.

If dynamic authorization using JSON XACML is used (JSON path”$.externalDynamicAuthorizationEndpoint”), ldproxy will provide the following attributes in the POST request to the endpoint:

“urn:oasis:names:tc:xacml:1.0:subject:subject-id” is the username, if it can be derived from the token.
“urn:oasis:names:tc:xacml:1.0:resource:resource-id” is the access path of the API for this request.
“urn:oasis:names:tc:xacml:1.0:action:action-id” is always “POST”.
“payload” is the base64-encoded JSON payload.

The JSON Web Token is provided in the Authorization header, too.

If the response value is “Deny”, ldproxy will reject the request. If the value is “Permit”, the request will be processed.

In secured services, ldproxy also supports a preprocessing of the payload of a POST/PUT request to insert or update a feature. If the request has been permitted, the post-processing endpoint (JSON path”$.postProcessingEndpoint”) is invoked. The post-processing endpoint has to return an updated feature that will then be inserted/updated in the dataset.

Request:

{
  "id": "abc",
  ...
}

Response:

{
  "id": "abc",
  ...
  "status": 0,
  "createdBy": "johndoe",
  "created": "2018-06-22T11:28:34Z",
  "changedBy": "johndoe",
  "changed": "2018-06-22T11:28:34Z"
}

The ldproxy deployment

To deploy an instance of ldproxy with the configuration, a docker container with a mounted volume has to be created, for example with the docker command line tool:

docker run --name ldproxy -d -p 7080:7080 -v /home/user/ldproxy_data:/ldproxy/data iide/ldproxy:v1.3.0

In this example “/home/user/ldproxy_data” has to be the parent directory of the “config-store” directory.

The published services are then available at http://localhost:7080/rest/services.

To change the external URL of a service, see the ldproxy documentation. Public services will typically hide the ldproxy manager. In this case, the value of the external URL (JSON path “$.externalUrl” in CoreServerConfig) from “/rest/services” to the new value.

Deployment issues

If you encounter issues with the deployment, inspect the log files to determine the cause of the issue. See the ldproxy documentation.

Limitations

Currently the conversion rules and parameters allow limited control. For example, flattening complex data structures in GeoJSON cannot be activated, the GeoJsonConfig is currently fixed. This may be changed in the future.

Configuration

Class

The class for the target implementation is de.interactive_instruments.ShapeChange.Target.Ldproxy.Config

Conversion Rules

rule-ldp-prop-all-datatype-relations-as-n-to-m-relations

This rule states that all properties with a value that is a data type are represented in the database using an intermediate table. The name of the table is determined by parameter templateNtoMTable.

Parameter(s):

templateNtoMTable

rule-ldp-prop-all-featuretype-relations-as-n-to-m-relations

This rule states that all properties with a value that is a feature type are represented in the database using an intermediate table. The name of the table is determined by parameter templateNtoMTable.

Parameter(s):

templateNtoMTable

rule-ldp-prop-all-codelist-values-as-strings

This rule states that all properties with a code list value are represented in the database using a string field.

Currently, no other mapping is supported.

Parameter(s): none

rule-ldp-prop-multiple-single-values-as-1-to-n-relations

This rule states that all properties with a value that is a simple value and a maximum multiplicity greater than 1 are represented in the database using a separate table. The name of the table is determined by parameter template1toNTable.

Parameter(s):

template1toNTable

rule-ldp-prop-separate-geometry-table

This rule states that all geometries are stored in a single geometry table (and not directly in the feature tables).

Parameter(s): none

rule-ldp-all-names-in-lowercase

This rule derives all table and field names as lower case from the model.

Currently, no other mapping is supported.

Parameter(s): none

rule-ldp-all-names-max-length

This rule limits table and field names to n characters where n is set using parameter maxLength.

Parameter(s):

maxLength

rule-ldp-cls-non-abstract-feature-types-as-collection

This rule maps all non–abstract feature types to a collection.

Parameter(s): none

rule-ldp-cls-table-per-feature-type

This rule states that all feature type classifiers are mapped to a separate table. I.e., for classes with superclasses, each of them will have a separate table. Each feature will have a row in each of those tables, all with the same identifier.

Note that this rule is a requirement, if the schema includes feature associations that involve non–abstract feature types.

Parameter(s): none

rule-ldp-cls-id-field

Create an id field for each feature or data type table

Parameter(s): none

rule-ldp-cls-generate-codelist

Try to process also the code list values and add a code list file to the configuration.

Parameter(s): none

rule-ldp-cls-oneo-metadata

Add OSIRIS-Neo metadata fields in feature tables. This assumes the following fields in each feature table:

erstelltvon character varying(255)
erstelltam timestamp without time zone
geaendertvon character varying(255)
geaendertam timestamp without time zone

Parameter(s): none

Parameters

outputDirectory

Required / Optional: optional

Type: String

Default Value: the current directory

Explanation: The directory where the configuration files will be created.

Applies to Rule(s): none – default behaviour

serviceId

Required / Optional: optional

Type: String (no whitespace etc.)

Default Value: name of the XML namespace prefix of a selected schema; or “fixme”, if none is specified

Explanation: The id of this service. The id is used as the last step in the base URI of the service.

Applies to Rule(s): none – default behaviour

serviceLabel

Required / Optional: optional

Type: String

Default Value: Alias or name of a selected schema; or “Some Dataset“, if none is specified

Explanation: Human readable label of this service.

Applies to Rule(s): none – default behaviour

serviceDescription

Required / Optional: optional

Type: String

Default Value: Documentation or the definition of a selected schema; or an empty string, if none is specified

Explanation: Description of this service.

Applies to Rule(s): none – default behaviour

serviceVersion

Required / Optional: optional

Type: String

Default Value: “1.0.0”

Explanation:Version of this service API.

Applies to Rule(s): none – default behaviour

secured

Required / Optional: optional

Type: Boolean (“true” or “false”)

Default Value: false (not secured)

Explanation: If “true”, the service is marked as secured.

Applies to Rule(s): none – default behaviour

primaryKeyField

Required / Optional: optional

Type:String

Default Value: “id”

Explanation: Name of the field in tables used for the primary key

Applies to Rule(s):

rule-ldp-cls-id-field

foreignKeySuffix

Required / Optional: optional

Type: String

Default Value: “_id”

Explanation: Suffix of the foreign key field in tables

Applies to Rule(s):

rule-ldp-cls-id-field

fieldNameMaxLength

Required / Optional: optional

Type: Integer

Default Value: 60

Explanation:Maximum length of field names

Applies to Rule(s):

rule-ldp-all-names-max-length

geometryTableName

Required / Optional: optional

Type: String

Default Value: “geom“, if there is a separate table for all geometries

Explanation: By default, it is assumed that geometry fields are part of the feature tables. However, if all geometries are in a separate table, specify the name of that table.

Applies to Rule(s):

rule-ldp-prop-separate-geometry-table

geometryFieldName

Required / Optional: optional

Type: String

Default Value: “geom“, if there is a separate table for all geometries

Explanation: By default, it is assumed that geometry fields are part of the feature tables. However, if all geometries are in a separate table, specify the field name of the geometry in that table.

Applies to Rule(s):

rule-ldp-prop-separate-geometry-table

templateNtoMTable

Required / Optional: optional

Type: String

Default Value: “{{class}}_2_{{property}}”

Explanation: By default, all intermediate tables capable of representing properties that could be n–to–m relations are called “{{class}}_2_{{property}}” where “class” is the name of the table representing objects that have the property and “property” would be the standard field name of the property according to the encoding rule.

That table will include foreign keys to the identifier of the “class” table and the table that represents the objects / data structures that are the values of the property.

Applies to Rule(s):

rule-ldp-prop-all-featuretype-relations-as-n-to-m-relations
rule-ldp-prop-all-datatype-relations-as-n-to-m-relations

template1toNTable

Required / Optional: optional

Type: String

Default Value: “{{class}}_{{property}}”

Explanation: By default, all dependent tables capable of representing properties that could be 1-to-n relations are called “{{class}}_{{property}}” where “class” is the name of the table representing objects that have the property and “property” would be the standard field name of the property according to the encoding rule.

That table will include a foreign key to the identifier of the “class” table.

Applies to Rule(s):

rule-ldp-prop-multiple-single-values-as-1-to-n-relations

rootFeatureTable

Required / Optional: optional

Type: String

Default Value: “root”

Explanation: By default, the collection to use as the target for a feature reference is determined by collection the feature is associated with. However, for associations to an abstract (root) feature type, this is unclear.

In that case, rule rule-ldp-cls-table-per-feature-type is required and the name of a table that includes a row for each feature in the dataset must be provided.

That table must include a field that has the collection name of the feature as its value. See parameter rootCollectionField.

For example, if the table name is “root” and the field name is “collection” then this parameter will be “root” and rootCollectionField will be “collection”. These are also the default values.

Applies to Rule(s):

rule-ldp-cls-table-per-feature-type

rootCollectionField

Required / Optional: optional

Type: String

Default Value: “collection”

In that case, rule rule-ldp-cls-table-per-feature-type is required and the name of a table that includes a row for each feature in the dataset must be provided. See parameter rootFeatureTable. That table must include a field that has the collection name of the feature as its value. For example, if the table name is “root” and the field name is “collection” then this parameter will be “collection” and parameter rootFeatureTable will be “root”. These are also the default values.

Applies to Rule(s):

rule-ldp-cls-table-per-feature-type

filterableFields

Required / Optional: optional

Type: String

Default Value: none

Explanation: By default, only the geometry fields are set as “filterable”. To mark other fields that are represented by a field (string, number or date) as filterable, specify them in a comma-separated list where each value uses the template “{{tablename}}.{{fieldname}}” where “tablename” is the name of the table that includes the filterable property (note that for nested data structures this is not the feature table) and “fieldname” is the name of the field. Example: “table1.field1,table2.field2”.

Applies to Rule(s): none – default behaviour

htmlLabels

Required / Optional: optional

Type: String

Default Value: primary key fields

Explanation: By default, the field that is the primary key is used as the label of a feature in HTML. To set the label for a feature type to another field, specify them in a comma-separated list where each value uses the template “{{tablename}}.{{fieldlabel}}” where “tablename” is the name of the feature table that includes the property and “fieldlabel” is the label of the field. Example: “table1.Field 1,table2.Field 2”. Use “*” as the table name to select a field for all feature types. Example: “*.Full Name”.

Applies to Rule(s): none – default behaviour

featureTypes

Required / Optional: optional

Type: String

Default Value: all feature types

Explanation: By default, the configuration is generated for all feature types. To reduce the configuration to a subset of feature types, specify them in a comma-separated list.

Applies to Rule(s): none – default behaviour

taggedValueForCodeListUrl

Required / Optional: optional

Type: String

Default Value: ‘codeList’

Explanation: By default, code lists are accessed using the standard ‘codeList’ tagged value. To support other UML profiles, another tagged value may be specified that is not normalized to ‘codeList’.

Applies to Rule(s):

rule-ldp-cls-generate-codeList

enablePropertiesReportable

Required / Optional: optional

Type: String

Default Value: all properties

Explanation: By default, all properties are included in the output. To support publishing only a subset, only the properties are enabled in the configuration with a tagged value ‘reportable’ and one of the values in this parameter. Specify the values in a comma-separated list, e.g. ‘true,internal’.

Applies to Rule(s): none – default behaviour

trigger_onDelete

Required / Optional: optional

Type: String

Default Value: no trigger on delete

Explanation: If a feature is deleted using the HTTP DELETE method, additional SQL statements can be executed in the database. Example: “DELETE FROM othertable WHERE objectid={{id}}” where “{{id}}” is the value of the “id” column in the table of the deleted feature. Specify the values in a semicolon-separated list.

Applies to Rule(s): none – default behaviour

MapEntries

MapEntry elements are used in ldproxy configurations to map types (UML classifiers) in the application schema to ldproxy. The name of the type is provided as usual in the XML attribute “type”. A map entry can be restricted to certain encoding rules in the XML attribute “rule”; “*” matches all encoding rules.

The XML attribute “targetType” specifies the type to use for the UML classifier in the applicable encoding rules. ldproxy supports:

STRING
DATE
NUMBER
BOOLEAN
GEOMETRY

The XML attribute “param” can be used to provide additional information in a semicolon separated list. Each value in the list is a key value pair separated by a colon:

Values with key “category” are used to identify spatial / temporal properties (the bbox / time parameter apply to this property). GEOMETRY values should have a value “SPATIAL” and DATE values should have a value “TEMPORAL”.
Values with key “format” are used to specify a format for JSON values according to the use of “format” for strings in JSON Schema.
Values with key “htmlformat” are used to specify a format for rendering a value in HTML, typically used for date/time values.
Values with key “jsongeometry” are used for GEOMETRY types to declare the GeoJSON geometry type. Valid values are:
- POINT
- MULTI_POINT
- LINE_STRING
- MULTI_LINE_STRING
- POLYGON
- MULTI_POLYGON
- GEOMETRY_COLLECTION
- GENERIC
Values with key “htmlgeometry” are used for GEOMETRY types to declare the schema.org geometry type. Valid values are:
- POINT
- LINE_STRING
- POLYGON
- GENERIC

Typical standard map entries are provided in https://shapechange.net/resources/config/StandardLdproxyMapEntries.xml.

Configuration Example

<Target class="de.interactive_instruments.ShapeChange.Target.Ldproxy.Config" 
 mode="enabled" inputs="INPUT">
 <targetParameter name="outputDirectory" value="results/api"/> 
 <targetParameter name="sortedOutput" value="true"/>
 <targetParameter name="serviceId" value="xyz"/>
 <targetParameter name="serviceLabel" value="Dataset XYZ"/>
 <targetParameter name="serviceDescription" value="This API ..."/>
 <targetParameter name="serviceVersion" value="0.0.1"/>
 <targetParameter name="secured" value="true"/>
 <targetParameter name="primaryKeyField" value="id"/>
 <targetParameter name="foreignKeySuffix" value="_id"/>
 <targetParameter name="fieldNameMaxLength" value="60"/>
 <targetParameter name="geometryTableName" value="geom"/>
 <targetParameter name="geometryFieldName" value="geom"/>
 <targetParameter name="templateNtoMTable" value="{{class}}_2_{{property}}"/>
 <targetParameter name="template1toNTable" value="{{class}}_{{property}}"/>
 <targetParameter name="rootFeatureTable" value="root"/>
 <targetParameter name="rootCollectionField" value="collection"/>
 <targetParameter name="featureTypes" value="building, parcel"/>
 <targetParameter name="filterableFields" value="root.identifier, root.timestamp, building.name, parcel.municipality"/>
 <targetParameter name="htmlLabels" value=".identifier"/>
 <targetParameter name="trigger_onDelete" value="DELETE FROM othertable WHERE objectid={{id}}"/>
 <targetParameter name="defaultEncodingRule" value="ldp"/>
 <rules>
  <EncodingRule name="ldp">
   <rule name="rule-ldp-cls-non-abstract-feature-types-as-collection"/>
   <rule name="rule-ldp-cls-table-per-feature-type"/>
   <rule name="rule-ldp-cls-id-field"/>
   <rule name="rule-ldp-cls-generate-codelist"/>
   <rule name="rule-ldp-all-names-in-lowercase"/>
   <rule name="rule-ldp-all-names-max-length"/>
   <rule name="rule-ldp-prop-all-datatype-relations-as-n-to-m-relations"/>
   <rule name="rule-ldp-prop-all-featuretype-relations-as-n-to-m-relations"/>
   <rule name="rule-ldp-prop-multiple-single-values-as-1-to-n-relations"/>
   <rule name="rule-ldp-prop-separate-geometry-table"/>
   <rule name="rule-ldp-prop-all-codelist-values-as-strings"/>
   <rule name="rule-ldp-cls-oneo-metadata"/>
  </EncodingRule>
 </rules>
 <xi:include href="https://shapechange.net/resources/config/StandardLdproxyMapEntries.xml"/>
</Target>

ldproxy Configuration

Introduction

Description

Overview

The service override configuration file

The security configuration file (optional)

The ldproxy deployment

Deployment issues

Limitations

Configuration

Class

Conversion Rules

rule-ldp-prop-all-datatype-relations-as-n-to-m-relations

rule-ldp-prop-all-featuretype-relations-as-n-to-m-relations

rule-ldp-prop-all-codelist-values-as-strings

rule-ldp-prop-multiple-single-values-as-1-to-n-relations

rule-ldp-prop-separate-geometry-table

rule-ldp-all-names-in-lowercase

rule-ldp-all-names-max-length

rule-ldp-cls-non-abstract-feature-types-as-collection

rule-ldp-cls-table-per-feature-type

rule-ldp-cls-id-field

rule-ldp-cls-generate-codelist

rule-ldp-cls-oneo-metadata

Parameters

outputDirectory

serviceId

serviceLabel

serviceDescription

serviceVersion

secured

primaryKeyField

foreignKeySuffix

fieldNameMaxLength

geometryTableName

geometryFieldName

templateNtoMTable

template1toNTable

rootFeatureTable

rootCollectionField

filterableFields

htmlLabels

featureTypes

taggedValueForCodeListUrl

enablePropertiesReportable

trigger_onDelete

MapEntries

Configuration Example