Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix the BAI2 grammar from forked package #1

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Fix the BAI2 grammar from forked package #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 24 additions & 15 deletions src/BTR3.ohm
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
BTR3 {
BTRSfile = FileHeader Bank* FileTrailer
BTRSfile = FileHeader Bank* FileTrailer

Bank = BankHeader Account* BankTrailer

Account = AccountHeader TransactionDetail* AccountTrailer


FileHeader = "01" delim senderIdentification delim receiverIdentification delim fileCreationDate delim fileCreationTime delim fileIdentificationNumber delim physicalRecordLength delim blockSize delim versionNumber eor
senderIdentification = alnum +
receiverIdentification = alnum +
senderIdentification = alphaNumeric+
receiverIdentification = alphaNumeric+
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These alterations are hardening the definition, these values cannot include , or /

fileCreationDate = date
fileCreationTime = time
fileIdentificationNumber = alnum +
physicalRecordLength = digit *
blockSize = digit *
fileIdentificationNumber = alphaNumeric+
physicalRecordLength = digit*
blockSize = digit*
versionNumber = bai1 | bai2 | bai3

FileTrailer = "99" delim fileControlTotal delim numberofBanks delim numberofRecords eor
Expand All @@ -26,8 +26,8 @@ BTR3 {


BankHeader = "02" delim ultimateReceiverIdentification delim bankIdentification delim groupStatus delim asofDate delim asofTime delim currencyCodeBank delim asofDateModifier eor
ultimateReceiverIdentification = alnum *
bankIdentification = alnum +
ultimateReceiverIdentification = alphaNumeric*
bankIdentification = alphaNumeric+
groupStatus = "1"
asofDate = date
asofTime = time?
Expand All @@ -47,7 +47,7 @@ BTR3 {
AccountHeader = "03" delim customerAccountNumber delim currencyCodeAccount delim statusOrSummaryCodeFormatOptRepeat eor
statusOrSummaryCodeFormatOptRepeat = statusOrSummaryCodeFormat (delim statusOrSummaryCodeFormat)*
statusOrSummaryCodeFormat = statusCodeFormat | summaryCodeFormat
customerAccountNumber = alnum *
customerAccountNumber = alphaNumeric*
currencyCodeAccount (Currency Code is mandatory in BTR3 but not in BAI2)
= letter*

Expand All @@ -74,7 +74,7 @@ BTR3 {

amountOpt = optSign digit*
itemCount = digit*
fundsType = alnum?
fundsType = ("0".."2" | "S" | "V" | "D" | "Z") ?
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, hardening the definition, there is a specific set of values for this [pg. 29]


AccountTrailer = "49" delim accountControlTotal delim numberofRecords eor
accountControlTotal = optSignedN
Expand Down Expand Up @@ -102,8 +102,10 @@ BTR3 {
bankReferenceNumber = alphaNumeric*
customerReferenceNumber = alphaNumeric*
detailText (Text relaxed for CBA BAI2 format does not contain , / Optionally followed by /)
= (~ eor any) *
= & "/" // null text field
| text * // normal text field
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, we need to do a lookahead for a / (in the file, this would look like ,/ because detailText is preceded by delim. In this case, the text is null, and trying to parse it as a text would cause the expression to be unreasonably complex


beginningOfNonContinuationRecord = ""
optPos = "+"?
optSign = ("-" | "+")?

Expand All @@ -120,10 +122,17 @@ BTR3 {
hh = ("0".."1" digit) | ("2" "0" .. "3")
mm = "0".."5" digit

delim = ","
newLine = ("\r\n" | "\n" | "\r")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should accept all kinds of newlines

delim =
| "," -- comma_delimiter
| "/" newLine "88," -- continuation_record_delimiter

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is big. I have added support for 88 - Continuation Record [pg. 20]. It is simplest to treat \n88, as a delimiter rather than a new record.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a record is "split" between fields, it will look like

...field1,field2/
88,field3,field4...

which we want to parse as field1,field2,field3,field4

The slash replaces the comma that ordinarily delimits that field. The example below correctly illustrates the use of delimiters.

Screen Shot 2021-03-02 at 4 11 46 PM

eor = "/"

alphaNumeric (BTR3 Alphanumeric - string not containing comma or slash)
= ~(delim | eor) any
alphaNumeric (BTR3 Alphanumeric - string not containing comma or slash)
= ~("," | "/") any

continuationWithinTextField = newLine "88,"

}
text = (continuationWithinTextField | ~newLine any)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, a text field does not have a delimiter at the end of a logical record. we want to parse

...text
88,andmoretext

as textandmoretext

}
187 changes: 183 additions & 4 deletions test/_btr3-file-examples.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -443,7 +443,7 @@
"detailFundsType": "",
"bankReferenceNumber": "AGN",
"customerReferenceNumber": "935892",
"detailText": "POINT OF SALE"
"detailText": "POINT OF SALE/"
}
},
{
Expand All @@ -453,7 +453,7 @@
"detailFundsType": "",
"bankReferenceNumber": "AGN",
"customerReferenceNumber": "54",
"detailText": "1"
"detailText": "1/"
}
},
{
Expand All @@ -463,7 +463,7 @@
"detailFundsType": "",
"bankReferenceNumber": "MIS",
"customerReferenceNumber": "",
"detailText": "9791112535 AMEX GR"
"detailText": "9791112535 AMEX GR/"
}
},
{
Expand Down Expand Up @@ -581,7 +581,7 @@
"detailFundsType": "",
"bankReferenceNumber": "AGN",
"customerReferenceNumber": "935892",
"detailText": "POINT OF SALE"
"detailText": "POINT OF SALE/"
}
}
],
Expand All @@ -604,5 +604,184 @@
}
}
}
},
"SVBSampleBAI2file": {
"description": "Sample BAI2 file from SVB",
"startnode": "BTRSfile",
"examplelines": [
"01,121140399,121140399,180130,0452,1,,,2/",
"02,,121140399,1,180129,,USD,2/",
"03,XXXXXXXXXX,USD,010,000,,,015,000,,/",
"88,040,000,,,045,000,,/",
"88,072,000,,,074,000,,/",
"88,100,171431234,2,,400,171431234,1,/",
"16,201,7348559,,6,0/",
"88,CASH SWEEP REDEMPTION",
"16,495,171437001,,20180XXXX41234,20180129L1B00D0C001234/",
"88,SENDER BNK:=SIL VLY BK SCLA;",
"88, SENDER ID:=XXXX40399;",
"88, SENDER REF:=20180XXXX42000;",
"88, ORG ID:=XXXXXXXXXX;",
"88, ORG:=XYZ COMPANY;",
"88, ORG ADDRESS:=12345 FOLSOM ST., SAN FRANCISCO, CA 94107;",
"88, BNF ID:=10000XXXXX783;",
"88, BNF NAME:=SUNTRUST BANK;",
"88, BNF ADDRESS:=XXXXXXX XXX SERVICES OPERATING ATTN: DKHJ WHHHH RE:AAAA ",
"88,XYZ COMPANY;",
"88, REC FI:=SUXXXXXX ATL;",
"88, REC ID:=099000909;",
"16,142,164088442,,2,0/",
"88,ZXXXXX INC Sweep SABCDEFB8880123 sxx_txxt_spv",
"49,685741234,22/",
"98,685741234,1,24/",
"99,685741234,1,26/"
],
"expected": {
"BTRSfile": {
"FileHeader": {
"senderIdentification": "121140399",
"receiverIdentification": "121140399",
"fileCreationDate": "2018-01-30",
"fileCreationTime": "04:52",
"fileIdentificationNumber": "1",
"physicalRecordLength": "",
"blockSize": "",
"versionNumber": "2"
},
"Bank": {
"BankHeader": {
"ultimateReceiverIdentification": "",
"bankIdentification": "121140399",
"groupStatus": "1",
"asofDate": "2018-01-29",
"asofTime": "",
"currencyCodeBank": "USD",
"asofDateModifier": "2"
},
"Accounts": [
{
"Account": {
"AccountHeader": {
"customerAccountNumber": "XXXXXXXXXX",
"currencyCodeAccount": "USD",
"AccountStatusesSummaries": [
{
"AccountStatus": {
"TypeCode": "010",
"Amount": "000",
"itemCount": "",
"fundsType": ""
}
},
{
"AccountStatus": {
"TypeCode": "015",
"Amount": "000",
"itemCount": "",
"fundsType": ""
}
},
{
"AccountStatus": {
"TypeCode": "040",
"Amount": "000",
"itemCount": "",
"fundsType": ""
}
},
{
"AccountStatus": {
"TypeCode": "045",
"Amount": "000",
"itemCount": "",
"fundsType": ""
}
},
{
"AccountStatus": {
"TypeCode": "072",
"Amount": "000",
"itemCount": "",
"fundsType": ""
}
},
{
"AccountStatus": {
"TypeCode": "074",
"Amount": "000",
"itemCount": "",
"fundsType": ""
}
},
{
"AccountSummary": {
"TypeCode": "100",
"Amount": "171431234",
"itemCount": "2",
"fundsType": ""
}
},
{
"AccountSummary": {
"TypeCode": "400",
"Amount": "171431234",
"itemCount": "1",
"fundsType": ""
}
}
]
},
"TransactionDetails": [
{
"TransactionDetail": {
"detailTypeCode": "201",
"detailAmount": "7348559",
"detailFundsType": "",
"bankReferenceNumber": "6",
"customerReferenceNumber": "0",
"detailText": "CASHSWEEPREDEMPTION"
}
},
{
"TransactionDetail": {
"detailTypeCode": "495",
"detailAmount": "171437001",
"detailFundsType": "",
"bankReferenceNumber": "20180XXXX41234",
"customerReferenceNumber": "20180129L1B00D0C001234",
"detailText": "SENDERBNK:=SILVLYBKSCLA;88,SENDERID:=XXXX40399;88,SENDERREF:=20180XXXX42000;88,ORGID:=XXXXXXXXXX;88,ORG:=XYZCOMPANY;88,ORGADDRESS:=12345FOLSOMST.,SANFRANCISCO,CA94107;88,BNFID:=10000XXXXX783;88,BNFNAME:=SUNTRUSTBANK;88,BNFADDRESS:=XXXXXXXXXXSERVICESOPERATINGATTN:DKHJWHHHHRE:AAAA88,XYZCOMPANY;88,RECFI:=SUXXXXXXATL;88,RECID:=099000909;"
}
},
{
"TransactionDetail": {
"detailTypeCode": "142",
"detailAmount": "164088442",
"detailFundsType": "",
"bankReferenceNumber": "2",
"customerReferenceNumber": "0",
"detailText": "ZXXXXXINCSweepSABCDEFB8880123sxx_txxt_spv"
}
}
],
"AccountTrailer": {
"accountControlTotal": "685741234",
"numberofRecords": 22
}
}
}
],
"BankTrailer": {
"groupControlTotal": "685741234",
"numberofAccounts": 1,
"numberofRecords": 24
}
},
"FileTrailer": {
"fileControlTotal": "685741234",
"numberofBanks": 1,
"numberofRecords": 26
}
}
}
}
}
2 changes: 1 addition & 1 deletion test/_btr3-transactiondetail-examples.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@
"detailFundsType": "",
"bankReferenceNumber": "AGN",
"customerReferenceNumber": "935892",
"detailText": "POINT OF SALE"
"detailText": "POINT OF SALE/"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I make a few changes to the test cases, because I believe the grammar was implemented incorrectly.

The specification states that

Therefore, the end of a record ending in text cannot be delimited using these characters. Instead, the logical record continues until another record that is not a continuation (88) record begins
Screen Shot 2021-03-02 at 3 57 32 PM

}
}
},
Expand Down