Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend the select operation to allow more fine grained sweep selection #2012

Closed
t-b opened this issue Feb 18, 2024 · 7 comments · Fixed by #2109
Closed

Extend the select operation to allow more fine grained sweep selection #2012

t-b opened this issue Feb 18, 2024 · 7 comments · Fixed by #2109

Comments

@t-b
Copy link
Collaborator

t-b commented Feb 18, 2024

The select operation 1 has the following signature:

select([array channels, array sweeps[, string mode[, string clampMode]]])

now we would like to add more control over sweep selection. Not only should it be able to select via displayed and clampmode but also via stimulus set and if it passed or not.

Sweep passing state is a LBN entry, see PSQ_FMT_LBN_SWEEP_PASS 2 as input for CreateAnaFuncLBNKey.

Not all analysis functions have that entry, only DA, SP, RA, CR, PB, SE, VM, AR so from the complete list

PSQ_PipetteInBath PB
PSQ_SealEvaluation SE
PSQ_AccessResistanceSmoke AR
PSQ_TrueRestingMembranePotential VM
PSQ_DaScale (sub threshold) DA (sub)
PSQ_SquarePulse (long pulse) SP
PSQ_Rheobase (long pulse) RB
PSQ_DaScale (supra threshold) DA (supra)
PSQ_SquarePulse (short pulse) SP
PSQ_Rheobase (short pulse) RB
PSQ_Ramp RA
PSQ_Chirp CR

and that means PSQ_Rheobase needs to be covered differently (see AD_FillWaves line 344, but note #2017).
Idea:

select([array channels, array sweeps, selectXXX(), ...])

where all selectXXX operations would be and-ed.

selectOptVis([all | displayed]) # default: displayed
selectOptCM([all | ic | vc | izero]) # default: all
selectOptStimset([name1, name2]) # name supports wildcards
selectOptIVSCCSweepQC(passed | failed)
selectOptIVSCCSetQC(passed | failed)

As users would also like to select multiple options OR'ed (e.g. passing sweeps from stimset A and failing sweeps from stimset B) we propose to allow an array of select() for all operations which accept a select statement (tp, data, psxKernel, psxStats, epochs, labnotebook).

@t-b
Copy link
Collaborator Author

t-b commented Feb 22, 2024

All operations will accept an array of select statements. Operations which accept a range will drop that argument. tp will warn when range is set in the select statement.

select will then return multiple datasets, one or multiple ranges (Nx2) and the sweepNo-channelType-channelNo tripplet (Nx3).

  1. Unify arguments
$so1  = selectVis(displayed)
$so2  = selectCM(ic, vc)
$so3  = selectStimset(name1, name2)
$so4  = channels(AD6)
$so5  = sweeps()
$so6  = selectRange() # accepts what formerly data accepted in the first argument
$so7  = selectIVSCCSweepQC(passed | failed)
$so8  = selectIVSCCSetQC(passed | failed)

// select will show all displayed sweeps from channel 6 with ic/vc clampMode from stimsets name1/name2
select($so1, $so2, $so3, $so4, $so5, ...)
Approaches not choosen 2. Combined array
$so1 = selectOptVis()
$so2 = selectOptCM(ic)
select(channels(AD), sweeps(), [$so1, $so2])
  1. Key-Value pairs
$so1 = selectOpt("vis=all", "clampMode=vc", "stimset=setA*")
$so2 = selectOpt("vis=all", "clampMode=ic", "stimset=setB*")

select(channels(AD), sweeps(), selectJoin($so1, $so2)) # or with []
  1. Ordered arguments
$so = selectOpt(vis, all, clampMode,[vc,ic],stimset,setA*)
select(channels(AD), sweeps(), $so)
</details>

@t-b

This comment was marked as outdated.

@timjarsky
Copy link
Collaborator

@t-b I'd also like to add an option to select all baseline or all pulse epochs, for example. We could also give the user a chance to apply some QC. e.g., check for spikes in baseline data?

@t-b
Copy link
Collaborator Author

t-b commented Mar 16, 2024

And it would be nice if we could recursively do selections:

sel = select(sweeps())
sel1 = select($sel, selectRange(E1))

This helps in disecting a bunch of sweeps (an RAC?) into two selections, one for each SCI for example.

@MichaelHuth
Copy link
Collaborator

MichaelHuth commented May 15, 2024

Thought regarding selectRange(E1):

select itself can not determine if ranges are good as this is depending on sweep / channelNumber / channelType. Thus, it makes no sense to think of any AND way to combine ranges when multiple are specified.

e.g.
select(selectRange("E*", "U*"), selectRange("*1", "*2"))

Currently the implementation just gathers all ranges specified.
Another option would be to allow only a single selectRange per select.

As select does not evaluate the ranges itself this information must be carried over to e.g. data. The idea is to make a composite return with two datasets, where the first dataset is the classic Nx3 select wave and the second dataset the range specification (as returned from SFH_EvaluateRange).
The whole composite would have the SELECT data type.

Regarding other entries appearing multiple times in select:
For selectStimset, sweeps, channels the SetIntersection is calculated. String inputs such as selectStimset and channels are evaluated as is for the intersection. Any wildcard patterns are not evaluated.
This also applies to channels in a sense, as {NaN, NaN} is a channels wildcard.

@MichaelHuth
Copy link
Collaborator

behavior for select:

  • if no select argument is set, then non-appearing filter arguments are default.
  • if select+ argument is set, then non-appearing filter arguments are ignored/off.
  • select returns always only the ranges that are specified in this select operation

@MichaelHuth
Copy link
Collaborator

  • allow only a single selectRange specification per select call. This solves the 1 range vs exactly N ranges, we should not have any other number of ranges
  • operations that have a select argument: use a generic function that parses that argument as either single select or array of selects, return waveref wave of selectComp
  • SFH_GetSweepsForFormula: iterates independently over each selectComp of the waveref input, handle selects and ranges as before per selectComp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants