Prototyping an ABAP OData Entity Data Model Generator : Simple Transformation

As mentioned in my earlier post It's the best of times, I’ve been trying to auto-generate the entity service-data-model XML ( EDMX file ) from exiting SAP database tables and their relationships.

Background:

SAP NetWeaver Gateway Service Builder (transaction SEGW) is a design-time environment to support creation of OData Services. It provides several options to support both, Outside-in and inside-out service Development Approaches:

I'll try to summarize these two approaches and available options in Service Builder before moving further with the prototype.

Gateway Service Development Approaches

• Outside-in:

  In this approach, you will design the service data-model based on the requirement of the consumer UI and not contaminate the model design by allowing the existing backend data-model/APIs dictate the service design. A non-SAP or SAP UI developer can carry out service modelling in Eclipse Graphical plug-in (GWPA) [ or in Microsoft Visual Studio ] and then export the prepared entity-data-model XML/EDMX file. Further, this XML/EDMX file can be imported by an ABAP developer through Service Builder (SEGW) to generate gateway Meta/Data provider classes. Note that Data Provider class/methods will need to be implemented after generation.
  
  

• Inside-out:

  In this approach, existing DDIC structures and APIs can be reused to generate model and data provider classes – so there is not so much model-design going on here apart from identifying and adapting the existing model. There are several options available depending on the Gateway version. ABAP developer can accordingly choose one of the following options, depending on available/existing artifacts.
  
  
  • ABAP Developer can import individual DDIC structures and then manually define associations and navigations in Service Builder before generating the gateway meta/data provider classes. Data provider classes will need to be implemented after generation.
    
  • However, if APIs (BAPI/RFC) are already available for the related entities then new versions of Gateway ( from SP5 ) also provides the option to import RFC/BORs that can auto-generate meta/data provider classes. Data provider classes will already have the base implementation to call the related APIs.
    
  • There are other options available as well - depending on the Gateway version and availability of existing artifacts. Refer to : Development Approaches and SAP NetWeaver Gateway Focus Group.
    

If you are into it then you can always call names or add adjectives to the developers using one or other approach e.g. Purists/ Time-wasters/ Obsessionist/ UI-freaks/ UI-5vers‎/ Eclipsers Vs. Lazy/ Backend-Geeks/ Penny-savers / Workbenchers / SAP Developers ;) etc. But it’s good that SAP has taken a pragmatic approach and as per the documentation for the Development Approaches:

In short, the Service Builder caters for all levels of development experience and provides development approaches tailored to saving time and increasing efficiency without compromising on quality. Whichever development approach is taken, the appropriate set of ABAP classes is generated. This ensures that developers retain maximum flexibility when optimizing or extending OData services in Gateway.

Summary:

So with these two approaches and several options already on the table, I thought about bridging some of the gap between the two approaches by developing an ABAP Program that could generate the data-model edmx/xml file programmatically for a given main/header SAP database table. It will use the foreign key relationship to determine association/navigation to other tables.

Purpose:

Auto-generated entity data model XML can be useful for rapid prototyping or for a development approach that falls between outside-in and inside-out approach. For example, if you don't want to start the data-modelling in Eclipse from scratch then you can use this utility to generate EDMX file from existing SAP database table relationships, import it into Eclipse -> adapt in the graphical modelling tool -> and then export it for service generation through Service Builder.
You can also import the file directly in SEGW to generate the gateway meta/data provider classes as per standard procedure of importing data-model edmx/xml file.

Technical Details:

In order to develop such program, I needed to first understand the structure/schema of OData Service entity-meta-data file so that I can:

• Define a complex structure ( Transaction SE11 ) , equivalent to the hierarchical structure of entity-meta-data XML.
• Develop a Simple Transformation program to produce XML - by transforming data populated in a variable of above complex type.
• Once the XML transformation part of development is done then executable ABAP Program will be developed to fetch table-relationship and meta-data information and populate the variable of defined complex type - and then call the transformation program to produce xml.

However there was a seemingly simpler option too: Reuse the class/methods, structure and simple transformation program that is used in SAP standard to generate the same file – only difference is that SAP generates the file *after* the service definition is already completed. Why re-invent the wheel?
For Ref :

• Class/methods : 
• /IWCOR/CL_DS_EDM_IMPL_BASE->/IWCOR/IF_DS_EDM_SVC_METADATA~GET_METADATA
• /IWCOR/CL_DS_EDM_METADATA->WRITE_METADATA
• Simple Transformation Program: /IWCOR/ST_ODATA_METADATA

As it turns out, reusing the existing standard programs might be a bit complex and also at this stage, I am not really sure if I need all the bells and whistles in the xml file. I was tempted to reuse at least the simple transformation program but it would have been bit difficult to populate the ABAP data-model object.

Anyway, since I am looking forward to learn more about the wheels, I decided to re-invent a simple version of it, which might even work for the purpose - who knows? Rest assured, I am not going replace the wheels in my kid's stroller with this one though :)

So I went to the source to learn about OData Service Meta Data Document.

A Service Metadata Document describes the data model (i.e. structure and organization of all the resources) exposed as HTTP endpoints by the service. A Service Metadata Document describes its data in EDM terms using an XML language for describing models called the Conceptual Schema Definition Language (CSDL). CSDL is fully described in [CSDL]. When exposed by an OData service as a Service Metadata Document, the CSDL document is packed using the format described in [EDMX].

After skimming through articles and CSDL schema etc., I quickly resorted to a few example XML documents to understand the structure of EDMX document. Also realized that SAP has extended CSDL to add a few of its own attributes under its own namespace e.g. sap:label, sap:creatable, sap:updatable, sap:sortable, sap:filterable etc. These are to help the generation of Meta/Data service classes through SEGW.

After leaving out a few not so essentials elements e.g. "complex-structures", operations etc. and a few iterations to add SAP extended elements for text-description etc., I managed to create a complex DDIC structure(ZRAM_EDMX_CSDL_SCHEMA) that corresponds to entity data model hierarchy.

• Structure ZRAM_EDMX_CSDL_SCHEMA is defined as below, XMLNS field is not required but I'll remove that later.
  
  
  
  
  
  
  
  
• With Structure definition in place, I can use the graphical utility to generate a simple transformation (Z_RAM_EDMX_GENERATE_ST) as explained in one of my very old posts : Generate Simple Transformation for XML in ABAP - Part II
  
  
  
• 
  Generated Simple Transformation code is as below :
  
  

  <?sap.transform simple?>
  <tt:transform xmlns:tt="http://www.sap.com/transformation-templates" xmlns:ddic="http://www.sap.com/abapxml/types/dictionary" xmlns:def="http://www.sap.com/abapxml/types/defined">
    <tt:root name="ROOT" type="?"/>
    <tt:root name="SCHEMA" type="ddic:ZRAM_EDMX_CSDL_SCHEMA"/>
    <tt:template>
  <edmx:Edmx xmlns:edmx="http://schemas.microsoft.com/ado/2007/06/edmx" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns:sap="http://www.sap.com/Protocols/SAPData" Version="1.0">
  <edmx:DataServices m:DataServiceVersion="2.0">
          <Schema xmlns="http://schemas.microsoft.com/ado/2009/11/edm" Alias="Self">
            <tt:attribute name="Namespace" value-ref=".SCHEMA.NAMESPACE"/>
            <EntityContainer Name="default" m:IsDefaultEntityContainer="true">
              <tt:loop ref=".SCHEMA.ENTITYCONTAINER.ENTITYSET_T">
                <EntitySet>
                  <tt:attribute name="Name" value-ref="NAME"/>
                  <tt:attribute name="EntityType" value-ref="ENTITYTYPE"/>
                </EntitySet>
              </tt:loop>
              <tt:loop ref=".SCHEMA.ENTITYCONTAINER.ASSOCIATIONSET_T">
                <AssociationSet>
                  <tt:attribute name="Name" value-ref="NAME"/>
                  <tt:attribute name="Association" value-ref="ASSOCIATION"/>
                  <tt:loop ref="END">
                    <End>
                      <tt:attribute name="Role" value-ref="ROLE"/>
                      <tt:attribute name="EntitySet" value-ref="ENTITYSET"/>
                    </End>
                  </tt:loop>
                </AssociationSet>
              </tt:loop>
            </EntityContainer>
            <tt:loop ref=".SCHEMA.ENTITYTYPE_T">
              <EntityType>
                <tt:attribute name="Name" value-ref="$REF.NAME"/>
                <Key>
                  <tt:loop ref="KEY">
                    <PropertyRef>
                      <tt:attribute name="Name" value-ref="NAME"/>
                    </PropertyRef>
                  </tt:loop>
                </Key>
                <tt:loop ref="PROPERTY_T">
                  <Property>
                    <tt:attribute name="sap:filterable" value-ref="$REF.SAP_FILTERABLE"/>
                    <tt:attribute name="sap:sortable" value-ref="$REF.SAP_SORTABLE"/>
                    <tt:attribute name="sap:updatable" value-ref="$REF.SAP_UPDATABLE"/>
                    <tt:attribute name="sap:creatable" value-ref="$REF.SAP_CREATABLE"/>
                    <tt:attribute name="Maxlength" value-ref="$REF.MAXLENGTH"/>
                    <tt:attribute name="Name" value-ref="NAME"/>
                    <tt:attribute name="Type" value-ref="TYPE"/>
                    <tt:attribute name="Nullable" value-ref="NULLABLE"/>
                    <tt:attribute name="sap:label" value-ref="$REF.SAP_LABEL"/>
                  </Property>
                </tt:loop>
                <tt:loop ref="NAVIGATIONPROPERTY_T">
                  <NavigationProperty>
                    <tt:attribute name="Name" value-ref="NAME"/>
                    <tt:attribute name="Relationship" value-ref="RELATIONSHIP"/>
                    <tt:attribute name="FromRole" value-ref="FROMROLE"/>
                    <tt:attribute name="ToRole" value-ref="TOROLE"/>
                  </NavigationProperty>
                </tt:loop>
              </EntityType>
            </tt:loop>
            <tt:loop name="XALIAS2" ref=".SCHEMA.ASSOCIATION_T">
              <Association>
                <tt:attribute name="Name" value-ref="$REF.NAME"/>
                <tt:loop ref="ASSOCIATION">
                  <End>
                    <tt:attribute name="Type" value-ref="TYPE"/>
                    <tt:attribute name="Role" value-ref="ROLE"/>
                    <tt:attribute name="Multiplicity" value-ref="MULTIPLICITY"/>
                  </End>
                </tt:loop>
              </Association>
            </tt:loop>
          </Schema>
        </edmx:DataServices>
      </edmx:Edmx>
    </tt:template>
  </tt:transform>  
  

  
  

Next..

So we now have the simple transformation program in place and next step is to develop an ABAP executable program that will take header/main database table name as input, fetch details of table meta-data/foreign-key relationships, populate a variable of type structure ZRAM_EDMX_CSDL_SCHEMA using selected data and then finally use "call transformation" to convert data to XML. I'll write about this prototype program in my next post.