Skip to content

Commit

Permalink
New v63005/gtm8877 subtest (Tests GTM-8877 in V63005 and YottaDB/YDB#318
Browse files Browse the repository at this point in the history 
) (#339)

Tests the following release note: 

The GT.M restriction mechanism recognizes the following lines:

ZSYSTEM_FILTER[:M labelref]
PIPE_FILTER[:M labelref]

The labelref must include a routine name. If a process is restricted by a ZSYSTEM or PIPE_OPEN line in the restrictions file that restriction takes precedence over the corresponding filter restriction. Otherwise when a process is subject to these restrictions, GT.M inserts an invocation of the labelref prior to the restricted command, passing a string containing the argument to the ZSYSTEM command or the command deviceparameter of the PIPE OPEN. The path to the filter routine must be included in $zroutines. FIS recommends that the filter routine is placed in a location with restricted access such as $gtm_dist. If the filter invocation return is -1,GT.M produces a RESTRICTEDOP error, otherwise it executes the command using the returned string via output parameters as a, possibly identical, replacement for the original string. Since GT.M uses the call-ins mechanism to execute the filters, a filter invocation inside a TP transaction in call-ins produces a CITPNESTED error. Note that because ZSYSTEM and OPEN are not Isolated actions FIS recommends against their use within a TP transaction. Filters also increment the nested level of call-ins. A recursive filter invocation produces a NOFILTERNEST error. GT.M reports all filter errors to the operator log accompanied by a COMMFILTERERR.

An example restrict file for this 

ZSYSTEM_FILTER:^filterzsy
PIPE_FILTER:^filterzsy

The actual filter routine:

```
filterzsy(inarg,outarg);

if ""=inarg set outarg="-1;must provide a command" quit

for i=1:1 set arg=$piece(inarg,";",i) quit:""=arg  do  quit:$data(outarg)

. for  quit:$zchar(9,32)'[$extract(arg)  set arg=$extract(arg,2,9999)

. set cmd=$piece(arg," ")

. for restrict="sudo","cd" if cmd=restrict set outarg="-1;command "_restrict_" not permitted" quit

. quit:$data(outarg)

. if "echo"=cmd set $piece(arg," ")="echo #",$piece(inarg,";",i)=arg    ;example of modification

set:'$data(outarg) outarg=inarg

quit +outarg
```

Filter execution starts with $STACK=1 ($ZLEVEL=2).

Following are the GT.M commands, Intrinsic Special Variables, and functions whose behavior changes in the context of a filter invocation.

ZGOTO 0 (zero) returns to the processing of the restricted command as does ZGOTO 1 (one) with no entryref, while ZGOTO 1:entryref replaces the originally invoked filter and continues filter execution.

$ZTRAP/$ETRAP NEW'd at level 1.

$ZLEVEL initializes to one (1) in GTM$CI, and increments for every new stack level.

$STACK initializes to zero (0) in GTM$CI frame, and increments for every new stack level.

$ESTACK NEW'd at level one (1) in GTM$CI frame.

$ECODE/$STACK() initialized to the empty string at level one (1) in GTM$CI frame.

After the filter completes, GT.M restores the above to their values at the invocation of the filter. (GTM-8877) 

Also tests YottaDB/YottaDB318:

Changes to ZSYSTEM_FILTER/PIPE_FILTER in restrict.txt (which resides in $ydb_dist/$gtm_dist) correctly take effect for future ZSYSTEM/PIPE OPEN commands. In GT.M V6.3-005, where this feature was introduced, if restrict.txt is changed multiple times within the same second for ZSYSTEM_FILTER/PIPE_FILTER, the changes did not take effect. A workaround was to "touch" the restrict.txt file afterwards to make its timestamp newer (relative to an auto-generated file filter_commands.tab).
  • Loading branch information
@VinayYottaDB
VinayYottaDB authored Jul 27, 2018
1 parent 19f6846 commit 31dbb38
Show file tree
Hide file tree
Showing 5 changed files with 262 additions and 1 deletion.
82 changes: 82 additions & 0 deletions v63005/inref/gtm8877.m
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; ;
; Copyright (c) 2018 YottaDB LLC. and/or its subsidiaries. ;
; All rights reserved. ;
; ;
; This source code contains the intellectual property ;
; of its copyright holder(s), and is made available ;
; under a license. If you do not know the terms of ;
; the license, please stop and do not read further. ;
; ;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;
zsystemfn
set $etrap="do incrtrap^incrtrap"
zsystem "echo 'original input'"
quit

pipeopenfn
set $etrap="do incrtrap^incrtrap"
set p="pipe"
open p:(command="echo 'original input'")::"PIPE"
use p
read x
use $p
write x,!
close p
quit

tpzsystemfn
set $etrap="do incrtrap^incrtrap"
tstart
do zsystemfn^gtm8877
tcommit
quit

tppipeopenfn
set $etrap="do incrtrap^incrtrap"
tstart
do pipeopenfn^gtm8877
tcommit
quit

tprestartfn
set restart=0
set $etrap="do incrtrap^incrtrap"
tstart ():(serial:transaction="BA")
if $trestart>0 do
. write:restart "restart occured outside of filter function",!
. write:'restart "restart occured within filter function",!
. write "$trestart = ",$trestart,!
set restart=0
if $trestart<=2 do
. do pipeopenfn
set restart=1
tcommit
quit

filterfn(inarg,outarg)
set outarg=-1
quit +outarg

filterfn2(inarg,outarg)
set outarg="echo 'filtered output'"
quit +outarg

reczsystfilter(inarg,outarg)
set outarg="echo 'recursive zsystem'"
zsystem outarg
quit +outarg

recpipefilter(inarg,outarg)
set outarg="echo 'recursive pipe'"
set p="pipe"
open p:(command=outarg)::"PIPE"
quit +outarg

tpfilter(inarg,outarg)
set outarg="echo 'Attempting to trigger a restart'"
set ^Y=$increment(^i)
zsystem "$ydb_dist/mumps -run ^%XCMD 'set ^Y=$increment(^i)'"
set ^Y=$increment(^i)
quit +outarg
3 changes: 2 additions & 1 deletion v63005/instream.csh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,14 @@
#-------------------------------------------------------------------------------------
# List of subtests of the form "subtestname [author] description"
#-------------------------------------------------------------------------------------
# gtm8877 [vinay] Tests the funcionality of ZSYSTEM_FILTER and PIPE_FILTER
#-------------------------------------------------------------------------------------

echo "v63005 test starts..."

# List the subtests separated by spaces under the appropriate environment variable name
setenv subtest_list_common ""
setenv subtest_list_non_replic ""
setenv subtest_list_non_replic "gtm8877"
setenv subtest_list_replic ""


Expand Down
57 changes: 57 additions & 0 deletions v63005/outref/gtm8877.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# NO RESTRICTIONS
original input
original input

# FILTER RETURNING -1 (attempting to use the pipe device after the open fails will cause an IONOTOPEN error)
%YDB-E-COMMFILTERERR, Error executing the command filter for gtmzsystem. RESTRICTEDOP
%YDB-E-RESTRICTEDOP, Attempt to perform a restricted operation: echo 'original input'
%YDB-E-COMMFILTERERR, Error executing the command filter for gtmpipeopen. RESTRICTEDOP
%YDB-E-RESTRICTEDOP, Attempt to perform a restricted operation: echo 'original input'
ZSTATUS=pipeopenfn+4^gtm8877,%YDB-E-IONOTOPEN, Attempt to USE an I/O device which has not been opened

# Checking the syslog
##TEST_AWK%YDB-E-COMMFILTERERR, Error executing the command filter for gtmzsystem. RESTRICTEDOP -- generated from 0x.*
##TEST_AWK%YDB-E-RESTRICTEDOP, Attempt to perform a restricted operation: echo 'original input' -- generated from 0x.*
##TEST_AWK%YDB-E-COMMFILTERERR, Error executing the command filter for gtmpipeopen. RESTRICTEDOP -- generated from 0x.*
##TEST_AWK%YDB-E-RESTRICTEDOP, Attempt to perform a restricted operation: echo 'original input' -- generated from 0x.*

# FILTER RETURNING A STRING
filtered output
filtered output

# FILTER INVOCATION IN A TP TRANSACTION
filtered output
filtered output

# TRIGGERING A RESTART OF A TP TRANSACTION WITHIN A FILTER
Attempting to trigger a restart
restart occured outside of filter function
$trestart = 1
restart occured within filter function
$trestart = 2
restart occured within filter function
$trestart = 3

# FILTER WITH ZSYSTEM AND PIPE_OPEN RESTRICTED
ZSTATUS=zsystemfn+2^gtm8877,%YDB-E-RESTRICTEDOP, Attempt to perform a restricted operation: ZSYSTEM
ZSTATUS=pipeopenfn+3^gtm8877,%YDB-E-RESTRICTEDOP, Attempt to perform a restricted operation: OPEN PIPE

# RECURSIVE FILTERS
ZSTATUS=reczsystfilter+2^gtm8877,%YDB-E-NOFILTERNEST, Filter nesting not allowed
%YDB-E-COMMFILTERERR, Error executing the command filter for gtmzsystem. 150374450,reczsystfilter+3^gtm8877,%YDB-E-NOTEXTRINSIC, QUIT/ZHALT does not return to an extrinsic function: argument not allowed
ZSTATUS=recpipefilter+3^gtm8877,%YDB-E-NOFILTERNEST, Filter nesting not allowed
%YDB-E-COMMFILTERERR, Error executing the command filter for gtmpipeopen. 150374450,recpipefilter+4^gtm8877,%YDB-E-NOTEXTRINSIC, QUIT/ZHALT does not return to an extrinsic function: argument not allowed
ZSTATUS=pipeopenfn+4^gtm8877,%YDB-E-IONOTOPEN, Attempt to USE an I/O device which has not been opened

# Checking the syslog
##TEST_AWK%YDB-E-NOFILTERNEST, Filter nesting not allowed -- generated from 0x.*
##TEST_AWK%YDB-E-NOFILTERNEST, Filter nesting not allowed -- generated from 0x.*
##TEST_AWK%YDB-E-COMMFILTERERR, Error executing the command filter for gtmzsystem. 150374450,reczsystfilter\+3\^gtm8877,%YDB-E-NOTEXTRINSIC, QUIT/ZHALT does not return to an extrinsic function: argument not allowed -- generated from 0x.*
##TEST_AWK%YDB-E-COMMFILTERERR, Error executing the command filter for gtmpipeopen. 150374450,recpipefilter\+4\^gtm8877,%YDB-E-NOTEXTRINSIC, QUIT/ZHALT does not return to an extrinsic function: argument not allowed -- generated from 0x.*

# No label specified in filters
ZSTATUS=zsystemfn+2^gtm8877,%YDB-E-RESTRICTEDOP, Attempt to perform a restricted operation: ZSYSTEM
ZSTATUS=pipeopenfn+3^gtm8877,%YDB-E-RESTRICTEDOP, Attempt to perform a restricted operation: OPEN PIPE
# Checking the syslog
##TEST_AWK%YDB-E-RESTRICTSYNTAX, Syntax error in file ##TEST_PATH##/ydb_temp_dist/restrict.txt at line number 1. All facilities restricted for process. -- generated from 0x.*
##TEST_AWK%YDB-E-RESTRICTSYNTAX, Syntax error in file ##TEST_PATH##/ydb_temp_dist/restrict.txt at line number 1. All facilities restricted for process. -- generated from 0x.*
1 change: 1 addition & 0 deletions v63005/outref/outref.txt
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
v63005 test starts...
##SUSPEND_OUTPUT REPLIC
PASS from gtm8877
##ALLOW_OUTPUT REPLIC
##SUSPEND_OUTPUT NON_REPLIC
##ALLOW_OUTPUT NON_REPLIC
Expand Down
120 changes: 120 additions & 0 deletions v63005/u_inref/gtm8877.csh
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
#!/usr/local/bin/tcsh -f
#################################################################
# #
# Copyright (c) 2018 YottaDB LLC. and/or its subsidiaries. #
# All rights reserved. #
# #
# This source code contains the intellectual property #
# of its copyright holder(s), and is made available #
# under a license. If you do not know the terms of #
# the license, please stop and do not read further. #
# #
#################################################################
#
#
#
source $gtm_tst/com/copy_ydb_dist_dir.csh ydb_temp_dist
setenv gtm_dist $ydb_dist
$gtm_tst/com/dbcreate.csh mumps 1 >>& dbcreate.out

echo "# NO RESTRICTIONS"
$ydb_dist/mumps -run zsystemfn^gtm8877
$ydb_dist/mumps -run pipeopenfn^gtm8877
echo ""


# FILTER RETURNING -1
cat > $ydb_dist/restrict.txt << EOF
ZSYSTEM_FILTER:filterfn^gtm8877
PIPE_FILTER:filterfn^gtm8877
EOF
chmod -w $ydb_dist/restrict.txt
set t = `date +"%b %e %H:%M:%S"`
echo "# FILTER RETURNING -1 (attempting to use the pipe device after the open fails will cause an IONOTOPEN error)"
$ydb_dist/mumps -run tpzsystemfn^gtm8877
$ydb_dist/mumps -run pipeopenfn^gtm8877
echo "# Checking the syslog"
$gtm_tst/com/getoper.csh "$t" "" getoper.txt
$grep RESTRICTEDOP getoper.txt |& sed 's/.*%YDB-E/%YDB-E/'
echo ""
sleep 1

# FILTER RETURNING A STRING
rm $ydb_dist/restrict.txt
cat > $ydb_dist/restrict.txt << EOF
ZSYSTEM_FILTER:filterfn2^gtm8877
PIPE_FILTER:filterfn2^gtm8877
EOF
chmod -w $ydb_dist/restrict.txt
echo "# FILTER RETURNING A STRING"
$ydb_dist/mumps -run zsystemfn^gtm8877
$ydb_dist/mumps -run pipeopenfn^gtm8877
echo ""

# TRIGGERING A FILTER IN A TP TRANSACTION
echo "# FILTER INVOCATION IN A TP TRANSACTION"
$ydb_dist/mumps -run tpzsystemfn^gtm8877
$ydb_dist/mumps -run tppipeopenfn^gtm8877
echo ""


# TRIGGERING A FILTER IN A TP TRANSACTION
echo "# TRIGGERING A RESTART OF A TP TRANSACTION WITHIN A FILTER"
rm $ydb_dist/restrict.txt
cat > $ydb_dist/restrict.txt << EOF
PIPE_FILTER:tpfilter^gtm8877
EOF
chmod -w $ydb_dist/restrict.txt
$ydb_dist/mumps -run tprestartfn^gtm8877
echo ""


# FILTERS WHILE RESTRICTING ZSYSTEM AND PIPE_OPEN
rm $ydb_dist/restrict.txt
cat > $ydb_dist/restrict.txt << EOF
ZSYSTEM
PIPE_OPEN
ZSYSTEM_FILTER:filterfn2^gtm8877
PIPE_FILTER:filterfn2^gtm8877
EOF
chmod -w $ydb_dist/restrict.txt
echo "# FILTER WITH ZSYSTEM AND PIPE_OPEN RESTRICTED"
$ydb_dist/mumps -run zsystemfn^gtm8877
$ydb_dist/mumps -run pipeopenfn^gtm8877
echo ""

# RECURSIVE FILTERS
rm $ydb_dist/restrict.txt
cat > $ydb_dist/restrict.txt << EOF
ZSYSTEM_FILTER:reczsystfilter^gtm8877
PIPE_FILTER:recpipefilter^gtm8877
EOF
chmod -w $ydb_dist/restrict.txt
echo "# RECURSIVE FILTERS"
set t = `date +"%b %e %H:%M:%S"`
$ydb_dist/mumps -run zsystemfn^gtm8877
$ydb_dist/mumps -run pipeopenfn^gtm8877
echo "# Checking the syslog"
$gtm_tst/com/getoper.csh "$t" "" getoper.txt
$grep FILTERNEST getoper.txt |& sed 's/.*%YDB-E/%YDB-E/'
$grep COMMFILTERERR getoper.txt |& sed 's/.*%YDB-E-COMMFILTERERR/%YDB-E-COMMFILTERERR/'
echo ""
sleep 1

# NO LABEL SPECIFIED IN FILTER
rm $ydb_dist/restrict.txt
cat > $ydb_dist/restrict.txt << EOF
ZSYSTEM_FILTER
PIPE_FILTER
EOF
set t = `date +"%b %e %H:%M:%S"`
chmod -w $ydb_dist/restrict.txt
echo "# No label specified in filters"
$ydb_dist/mumps -run zsystemfn^gtm8877
$ydb_dist/mumps -run pipeopenfn^gtm8877
echo "# Checking the syslog"
# Avoid extra RESTRICTSYNTAX errors from getoper.csh (which does mumps -run) by removing restrict.txt before the call
rm $ydb_dist/restrict.txt
$gtm_tst/com/getoper.csh "$t" "" getoper.txt
$grep RESTRICTSYNTAX getoper.txt |& sed 's/.*%YDB-E/%YDB-E/'
$gtm_tst/com/dbcheck.csh >>& dbcheck.out

0 comments on commit 31dbb38

Please sign in to comment.