Skip to content

Commit

Permalink
Issue #73 - Initial ResourceInfo Model Generation
Browse files Browse the repository at this point in the history
  • Loading branch information
darnjo committed Apr 27, 2021
1 parent 19f7234 commit 0ac742e
Show file tree
Hide file tree
Showing 5 changed files with 328 additions and 51 deletions.
96 changes: 55 additions & 41 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,48 +58,51 @@ $ java -jar path/to/web-api-commander.jar
Doing so displays the following information:
```
usage: java -jar web-api-commander
--bearerToken <b> Bearer token to be used with the
request.
--clientId <d> Client Id to be used with the request.
--bearerToken <b> Bearer token to be used with the
request.
--clientId <d> Client Id to be used with the request.
--clientSecret <s>
--contentType <t> Results format: JSON (default),
JSON_NO_METADATA, JSON_FULL_METADATA,
XML.
--entityName <n> The name of the entity to fetch, e.g.
Property.
--generateDDAcceptanceTests Generates acceptance tests in the
current directory.
--generateMetadataReport Generates metadata report from given
<inputFile>.
--generateQueries Resolves queries in a given RESOScript
<inputFile> and displays them in
standard out.
--generateReferenceDDL Generates reference DDL to create a
RESO-compliant SQL database. Pass
--useKeyNumeric to generate the DB using
numeric keys.
--generateReferenceEDMX Generates reference metadata in EDMX
format.
--getMetadata Fetches metadata from <serviceRoot>
using <bearerToken> and saves results in
<outputFile>.
--help print help
--inputFile <i> Path to input file.
--outputFile <o> Path to output file.
--runRESOScript Runs commands in RESOScript file given
as <inputFile>.
--saveGetRequest Performs GET from <requestURI> using the
given <bearerToken> and saves output to
<outputFile>.
--serviceRoot <s> Service root URL on the host.
--uri <u> URI for raw request. Use 'single quotes'
to enclose.
--useEdmEnabledClient present if an EdmEnabledClient should be
used.
--useKeyNumeric present if numeric keys are to be used
for database DDL generation.
--validateMetadata Validates previously-fetched metadata in
the <inputFile> path.
--contentType <t> Results format: JSON (default),
JSON_NO_METADATA, JSON_FULL_METADATA,
XML.
--entityName <n> The name of the entity to fetch, e.g.
Property.
--generateDDAcceptanceTests Generates acceptance tests in the
current directory.
--generateMetadataReport Generates metadata report from given
<inputFile>.
--generateQueries Resolves queries in a given RESOScript
<inputFile> and displays them in
standard out.
--generateReferenceDDL Generates reference DDL to create a
RESO-compliant SQL database. Pass
--useKeyNumeric to generate the DB
using numeric keys.
--generateReferenceEDMX Generates reference metadata in EDMX
format.
--generateResourceInfoModels Generates Java Models for the Web API
Reference Server in the current
directory.
--getMetadata Fetches metadata from <serviceRoot>
using <bearerToken> and saves results
in <outputFile>.
--help print help
--inputFile <i> Path to input file.
--outputFile <o> Path to output file.
--runRESOScript Runs commands in RESOScript file given
as <inputFile>.
--saveGetRequest Performs GET from <requestURI> using
the given <bearerToken> and saves
output to <outputFile>.
--serviceRoot <s> Service root URL on the host.
--uri <u> URI for raw request. Use 'single
quotes' to enclose.
--useEdmEnabledClient present if an EdmEnabledClient should
be used.
--useKeyNumeric present if numeric keys are to be used
for database DDL generation.
--validateMetadata Validates previously-fetched metadata
in the <inputFile> path.
```
When using commands, if required arguments aren't provided, relevant feedback will be displayed in the terminal.
Expand Down Expand Up @@ -227,6 +230,17 @@ New Cucumber BDD acceptance tests will be generated and placed in a timestamped

To update the current tests, copy the newly generated ones into the [Data Dictionary BDD `.features` directory](src/main/java/org/reso/certification/features/data-dictionary/v1-7-0), run the `./gradlew build` task, and if everything works as expected, commit the newly generated tests.

## Generating RESO Web API Reference Server Data Models
The RESO Commander can be used to generate data models for the Web API Reference server from the currently approved [Data Dictionary Spreadsheet](src/main/resources/RESODataDictionary-1.7.xlsx).

The Commander project's copy of the sheet needs to be updated with a copy of the [DD Google Sheet](https://docs.google.com/spreadsheets/d/1SZ0b6T4_lz6ti6qB2Je7NSz_9iNOaV_v9dbfhPwWgXA/edit?usp=sharing) prior to generating reference metadata.

```
$ java -jar path/to/web-api-commander.jar --generateResourceInfoModels
```
New ResourceInfo Models for the Web API Reference Server will be generated and placed in a timestamped directory relative to your current path.


## Generating RESO Data Dictionary Reference Metadata
In addition to generating DD acceptance tests, the RESO Commander can generate reference metadata based on the current reference [Data Dictionary Spreadsheet](src/main/resources/RESODataDictionary-1.7.xlsx).

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ private static ArrayList<String> buildTags(ReferenceStandardField field) {
return tags;
}

private static String padLeft(String s, int n) {
public static String padLeft(String s, int n) {
String[] padding = new String[n];
Arrays.fill(padding, " ");
return String.join("", padding) + s;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ void generateOutput() {
.append("\n\n")
.append("CREATE TABLE IF NOT EXISTS ")
//exception for ouid so it doesn't become o_u_i_d
.append(CaseFormat.UPPER_CAMEL.to(CaseFormat.LOWER_UNDERSCORE, resourceName).replace("o_u_i_d", "ouid"))
.append(buildDbTableName(resourceName))
.append(" ( ")
.append(templateContent).append(",\n")
.append(PADDING).append(PADDING).append(buildPrimaryKeyMarkup(resourceName)).append("\n")
Expand All @@ -164,6 +164,12 @@ void generateOutput() {
LOG.info(this::buildInsertLookupsStatement);
}

public static String buildDbTableName(String resourceName) {
return CaseFormat.UPPER_CAMEL.to(CaseFormat.LOWER_UNDERSCORE, resourceName).replace("o_u_i_d", "ouid");
}



private static String buildCreateLookupStatement(boolean useKeyNumeric) {
return
"\n\n/**\n" +
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,249 @@
package org.reso.certification.codegen;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.poi.ss.usermodel.Sheet;
import org.reso.commander.common.Utils;
import org.reso.models.ReferenceStandardField;

import static org.reso.certification.codegen.DDLProcessor.buildDbTableName;
import static org.reso.certification.containers.WebAPITestContainer.EMPTY_STRING;

public class ResourceInfoProcessor extends WorksheetProcessor {
final static String
ANNOTATION_TERM_DISPLAY_NAME = "RESO.OData.Metadata.StandardName",
ANNOTATION_TERM_DESCRIPTION = "Core.Description",
ANNOTATION_TERM_URL = "RESO.DDWikiUrl";
private static final Logger LOG = LogManager.getLogger(ResourceInfoProcessor.class);
private static final String
FILE_EXTENSION = ".java";

public void processResourceSheet(Sheet sheet) {
super.processResourceSheet(sheet);
markup.append(ResourceInfoTemplates.buildClassInfo(sheet.getSheetName(), null));
}

@Override
void processNumber(ReferenceStandardField row) {
markup.append(ResourceInfoTemplates.buildNumberMarkup(row));
}

@Override
void processStringListSingle(ReferenceStandardField row) {
markup.append(ResourceInfoTemplates.buildStringListSingleMarkup(row));
}

@Override
void processString(ReferenceStandardField row) {
markup.append(ResourceInfoTemplates.buildStringMarkup(row));
}

@Override
void processBoolean(ReferenceStandardField row) {
markup.append(ResourceInfoTemplates.buildBooleanMarkup(row));
}

@Override
void processStringListMulti(ReferenceStandardField row) {
markup.append(ResourceInfoTemplates.buildStringListMultiMarkup(row));
}

@Override
void processDate(ReferenceStandardField row) {
markup.append(ResourceInfoTemplates.buildDateMarkup(row));
}

@Override
void processTimestamp(ReferenceStandardField row) {
markup.append(ResourceInfoTemplates.buildTimestampMarkup(row));
}

@Override
void processCollection(ReferenceStandardField row) {
LOG.debug("Collection Type is not supported!");
}

@Override
void generateOutput() {
LOG.info("Using reference worksheet: " + REFERENCE_WORKSHEET);
LOG.info("Generating ResourceInfo .java files for the following resources: " + resourceTemplates.keySet().toString());
resourceTemplates.forEach((resourceName, content) -> {
//put in local directory rather than relative to where the input file is
Utils.createFile(getDirectoryName(), resourceName + "Definition" + FILE_EXTENSION, content);
});
}

@Override
String getDirectoryName() {
return startTimestamp + "-ResourceInfoModels";
}

@Override
public void afterResourceSheetProcessed(Sheet sheet) {
assert sheet != null && sheet.getSheetName() != null;
String resourceName = sheet.getSheetName();

String templateContent =
markup.toString() + "\n" +
" return " + resourceName + "Definition.fieldList;\n" +
" }\n" +
"}";

resourceTemplates.put(resourceName, templateContent);
resetMarkupBuffer();
}

public static final class ResourceInfoTemplates {
/**
* Contains various templates used for test generation
* TODO: add a formatter rather than using inline spaces
*/
public static String buildClassInfo(String resourceName, String generatedTimestamp) {
if (resourceName == null) return null;
if (generatedTimestamp == null) generatedTimestamp = Utils.getIsoTimestamp();

final String definitionName = resourceName + "Definition";

return "package org.reso.service.data.definition;\n" + "\n" +
"import org.apache.olingo.commons.api.edm.EdmPrimitiveTypeKind;\n" +
"import org.reso.service.data.meta.FieldInfo;\n" +
"import org.reso.service.data.meta.ResourceInfo;\n" + "\n" +
"import java.util.ArrayList;\n" + "\n" +
"// This class was autogenerated on: " + generatedTimestamp + "\n" +
"public class " + definitionName + " extends ResourceInfo {\n" +
" private static ArrayList<FieldInfo> fieldList = null;\n" + "\n" +
" public " + definitionName + "() {" + "\n" +
" this.tableName = " + buildDbTableName(resourceName) + ";\n" +
" this.resourcesName = " + resourceName + ";\n" +
" this.resourceName = " + resourceName + ";\n" +
" }\n" + "\n" +
" public ArrayList<FieldInfo> getFieldList() {\n" +
" return " + definitionName + ".getStaticFieldList();\n" +
" }\n" + "\n" +
" public static ArrayList<FieldInfo> getStaticFieldList() {\n" +
" if (null != " + definitionName + ".fieldList) {\n" +
" return " + definitionName + ".fieldList;\n" +
" }\n" + "\n" +
" ArrayList<FieldInfo> list = new ArrayList<FieldInfo>();\n" +
" " + definitionName + ".fieldList = list;\n" +
" FieldInfo fieldInfo = null;\n";
}

public static String buildBooleanMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;

//TODO: refactor into one method that takes a type name and returns the appropriate content
return "\n" +
" fieldInfo = new FieldInfo(\"" + field.getStandardName() + "\", EdmPrimitiveTypeKind.Boolean.getFullQualifiedName());\n" +
" fieldInfo.addAnnotation(\"" + field.getDisplayName() + "\", \"" + ANNOTATION_TERM_DISPLAY_NAME + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getDefinition() + "\", \"" + ANNOTATION_TERM_DESCRIPTION + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getWikiPageUrl() + "\", \"" + ANNOTATION_TERM_URL + "\");\n" +
" list.add(fieldInfo);" +
"\n";
}

public static String buildDateMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;

return "\n" +
" fieldInfo = new FieldInfo(\"" + field.getStandardName() + "\", EdmPrimitiveTypeKind.Date.getFullQualifiedName());\n" +
" fieldInfo.addAnnotation(\"" + field.getDisplayName() + "\", \"" + ANNOTATION_TERM_DISPLAY_NAME + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getDefinition() + "\", \"" + ANNOTATION_TERM_DESCRIPTION + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getWikiPageUrl() + "\", \"" + ANNOTATION_TERM_URL + "\");\n" +
" list.add(fieldInfo);" +
"\n";
}

/**
* Provides special routing for Data Dictionary numeric types, which may be Integer or Decimal
*
* @param field the numeric field to build type markup for
* @return a string containing specific markup for the given field
*/
public static String buildNumberMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;

if (field.getSuggestedMaxPrecision() != null) return buildDecimalMarkup(field);
else return buildIntegerMarkup(field);
}

public static String buildDecimalMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;

return "\n" +
" fieldInfo = new FieldInfo(\"" + field.getStandardName() + "\", EdmPrimitiveTypeKind.Decimal.getFullQualifiedName());\n" +
" fieldInfo.addAnnotation(\"" + field.getDisplayName() + "\", \"" + ANNOTATION_TERM_DISPLAY_NAME + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getDefinition() + "\", \"" + ANNOTATION_TERM_DESCRIPTION + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getWikiPageUrl() + "\", \"" + ANNOTATION_TERM_URL + "\");\n" +
" list.add(fieldInfo);" +
"\n";

//TODO: Length is actually scale for Decimal fields by the DD! :/
//TODO: Add setScale property to Decimal types in FieldInfo

//TODO: Precision is actually Scale for Decimal fields by the DD! :/
//TODO: Add setPrecision property to Decimal types in FieldInfo
}

public static String buildIntegerMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;

return "\n" +
" fieldInfo = new FieldInfo(\"" + field.getStandardName() + "\", EdmPrimitiveTypeKind.Int64.getFullQualifiedName());\n" +
" fieldInfo.addAnnotation(\"" + field.getDisplayName() + "\", \"" + ANNOTATION_TERM_DISPLAY_NAME + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getDefinition() + "\", \"" + ANNOTATION_TERM_DESCRIPTION + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getWikiPageUrl() + "\", \"" + ANNOTATION_TERM_URL + "\");\n" +
" list.add(fieldInfo);" +
"\n";
}

private static String buildStandardEnumerationMarkup(String lookupName) {
//TODO: add code to build Lookups
return "\n /* TODO: buildStandardEnumerationMarkup */\n";
}

public static String buildStringListMultiMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;
//TODO: add multi lookup handler
return "\n /* TODO: buildStringListMultiMarkup */\n";
}

public static String buildStringListSingleMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;
//TODO: add single lookup handler
return "\n /* TODO: buildStringListSingleMarkup */\n";
}

public static String buildStringMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;

String content = "\n" +
" fieldInfo = new FieldInfo(\"" + field.getStandardName() + "\", EdmPrimitiveTypeKind.String.getFullQualifiedName());\n" +
" fieldInfo.addAnnotation(\"" + field.getDisplayName() + "\", \"" + ANNOTATION_TERM_DISPLAY_NAME + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getDefinition() + "\", \"" + ANNOTATION_TERM_DESCRIPTION + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getWikiPageUrl() + "\", \"" + ANNOTATION_TERM_URL + "\");\n";

if (field.getSuggestedMaxLength() != null) {
content +=
" fieldInfo.setMaxLength(" + field.getSuggestedMaxLength() + ");\n";
}

content +=
" list.add(fieldInfo);" + "\n";

return content;
}

public static String buildTimestampMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;

return "\n" +
" fieldInfo = new FieldInfo(\"" + field.getStandardName() + "\", EdmPrimitiveTypeKind.DateTime.getFullQualifiedName());\n" +
" fieldInfo.addAnnotation(\"" + field.getDisplayName() + "\", \"" + ANNOTATION_TERM_DISPLAY_NAME + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getDefinition() + "\", \"" + ANNOTATION_TERM_DESCRIPTION + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getWikiPageUrl() + "\", \"" + ANNOTATION_TERM_URL + "\");\n" +
" list.add(fieldInfo);" +
"\n";
}
}
}
Loading

0 comments on commit 0ac742e

Please sign in to comment.