Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revise the use of temporary files in Stat-Analysis #2698

Open
7 of 21 tasks
JohnHalleyGotway opened this issue Sep 28, 2023 · 0 comments
Open
7 of 21 tasks

Revise the use of temporary files in Stat-Analysis #2698

JohnHalleyGotway opened this issue Sep 28, 2023 · 0 comments
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle component: code optimization Code optimization issue priority: medium Medium Priority requestor: METplus Team METplus Development Team type: enhancement Improve something that it is currently doing
Milestone

Comments

@JohnHalleyGotway
Copy link
Collaborator

JohnHalleyGotway commented Sep 28, 2023

Describe the Enhancement

As discussed in the Contributor's Guide documentation, the Stat-Analysis Tool writes a temporary file. It reads all the input files, applies any common filtering logic requested, and writes the result to a temporary file. All subsequent jobs read data from that temp file, apply any additional filtering options, and then perform the requested operation. The temp file is always written, even if only one job is run on the command line or a config file is used without any common filtering options.

This issue is to revise that logic to minimize the use of that temp file. If only one job is specified (on the command line or via a config file), there is no need to write to/read from a config file. In addition, if no common filtering logic is specified in the config file, we might be able to avoid writing the temp file. So investigate that to see if it's feasible to skip the config file there too.

Time Estimate

2 days

Sub-Issues

Consider breaking the enhancement down into sub-issues.
None needed

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Review default alert labels
  • Select component(s)
  • Select priority
  • Select requestor(s)

Milestone and Projects

  • Select Milestone as the next official version or Backlog of Development Ideas
  • For the next official version, select the MET-X.Y.Z Development project

Define Related Issue(s)

Consider the impact to the other METplus components.

Enhancement Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of develop.
    Branch name: feature_<Issue Number>_<Description>
  • Complete the development and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into develop.
    Pull request: feature <Issue Number> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Development issue
    Select: Milestone as the next official version
    Select: MET-X.Y.Z Development project for development toward the next official release
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Close this issue.
@JohnHalleyGotway JohnHalleyGotway added type: enhancement Improve something that it is currently doing priority: medium Medium Priority alert: NEED ACCOUNT KEY Need to assign an account key to this issue component: code optimization Code optimization issue requestor: METplus Team METplus Development Team labels Sep 28, 2023
@JohnHalleyGotway JohnHalleyGotway added this to the MET 12.0.0 milestone Sep 28, 2023
@JohnHalleyGotway JohnHalleyGotway changed the title Revise the use of temporary files in PB2NC Revise the use of temporary files in Stat-Analysis Sep 29, 2023
@JohnHalleyGotway JohnHalleyGotway added the alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle label Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle component: code optimization Code optimization issue priority: medium Medium Priority requestor: METplus Team METplus Development Team type: enhancement Improve something that it is currently doing
Projects
Status: 🟢 Ready
Development

No branches or pull requests

1 participant