Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Produce explanations of why a verified program is correct #23

Open
RyanGlScott opened this issue Jan 17, 2022 · 1 comment
Open

Produce explanations of why a verified program is correct #23

RyanGlScott opened this issue Jan 17, 2022 · 1 comment
Labels
enhancement New feature or request

Comments

@RyanGlScott
Copy link
Collaborator

Currently, programs that copilot-verifier marks as correct are indicated by a lack of error messages being printed. This, however, doesn't give a sense to the user as to why a program is correct. We should investigate what it would take to equip copilot-verifier with the ability to produce human-readable explanations of what the verifier did in the course of proving a program is correct.

In very broad strokes, I imagine this would require taking ahold of the ProvedGoals that Crux returns after simulation and analyzing them to determine what explanations to produce. There are likely some finer details to work around, however.

@RyanGlScott RyanGlScott added the enhancement New feature or request label Jan 17, 2022
RyanGlScott added a commit that referenced this issue Mar 15, 2023
This adds a `logSmtInteractions` flag to `VerifierOptions` that, when enabled,
will dump each SMT solver interaction into its own file under `results/`. See
the Haddocks for `logSmtInteractions` for the full details. Implementing this
requires bumping the `crucible` submodule to bring in the changes from
GaloisInc/crucible#1067, which adds support for logging offline SMT solver
interactions.

This adds some information that will be needed for #23.
@RyanGlScott
Copy link
Collaborator Author

RyanGlScott commented Mar 15, 2023

The first step towards producing a cogent explanation of why a proof is correct is to plumb the necessary information into places that are easy to access. Currently, the following information is tucked away under the hood of the verifier, and we should bring it to the surface:

  • Undefined behavior checks in Crucible
  • Other Crucible assertions
  • What4 formulae for the bullet points above
  • Raw SMT solver interactions corresponding to the What4 formulae (Support logging raw SMT solver interactions #36)
  • Equality checks between Copilot stream values and C array values
  • Equality checks between trigger arguments on the Copilot and C side
  • Checks that the trigger functions are invoked at the same times on the Copilot and C side
  • Copilot line number information (Display Copilot-related line number information #37)
  • Anything else?

It might make sense to open smaller issues for each of these subtasks as we work on them.

RyanGlScott added a commit that referenced this issue Mar 15, 2023
This adds a `logSmtInteractions` flag to `VerifierOptions` that, when enabled,
will dump each SMT solver interaction into its own file under `results/`. See
the Haddocks for `logSmtInteractions` for the full details. Implementing this
requires bumping the `crucible` submodule to bring in the changes from
GaloisInc/crucible#1067, which adds support for logging offline SMT solver
interactions.

This adds some information that will be needed for #23.
RyanGlScott added a commit that referenced this issue Mar 15, 2023
This adds a `logSmtInteractions` flag to `VerifierOptions` that, when enabled,
will dump each SMT solver interaction into its own file under `results/`. See
the Haddocks for `logSmtInteractions` for the full details. Implementing this
requires bumping the `crucible` submodule to bring in the changes from
GaloisInc/crucible#1067, which adds support for logging offline SMT solver
interactions.

This adds some information that will be needed for #23.
RyanGlScott added a commit that referenced this issue Mar 22, 2023
This addresses several bullet points in #23.
RyanGlScott added a commit that referenced this issue Mar 30, 2023
This addresses several bullet points in #23.
RyanGlScott added a commit that referenced this issue Mar 31, 2023
This addresses several bullet points in #23.
RyanGlScott added a commit that referenced this issue Apr 10, 2023
This addresses several bullet points in #23.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Development

No branches or pull requests

1 participant