fix: Remove uses of `:|` #618

atomb · 2024-08-21T15:17:51Z

Issue #, if available:

Description of changes:

Removes uses of :| in Dafny code, and updates the libraries submodule to a version that does not use it, either. This will eliminate failures when compiling with Dafny 4.8.0 and -definiteAssignment:3.

Squash/merge commit message, if applicable:

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

In conjunction with the update of the libraries submodule, this will allow all code to be compiled with `-definiteAssignment:3` without Failures with the upcoming Dafny 4.8.0.

texastony · 2024-08-21T16:14:14Z

@atomb I put this PR in draft. Once verification and other CI is passing, you can mark it ready for review.

robin-aws · 2024-08-21T21:23:59Z

StandardLibrary/src/StandardLibrary.dfy

@@ -335,7 +335,7 @@ module StandardLibrary {
   * The function is compilable, but will not exhibit enviable performance.
   */

-  function method {:tailrecursion} SetToOrderedSequence<T(!new,==)>(s: set<seq<T>>, less: (T, T) -> bool): (q: seq<seq<T>>)
+  function SetToOrderedSequence<T(!new,==)>(s: set<seq<T>>, less: (T, T) -> bool): (q: seq<seq<T>>)


This will be breaking for consumers of the MPL, since you're changing it from compiled to ghost, but there are compiled references to it already, e.g. https://github.com/aws/aws-database-encryption-sdk-dynamodb/blob/main/DynamoDbEncryption/dafny/DynamoDbEncryption/src/DynamoToStruct.dfy#L1091

Are you relying on assuming we can replace all such references with ComputeSetToSequence[2]?

Ah, I removed internal compiled references to it but didn't realize it was part of the exported API.

I suspect we should be able to replace them all, though it's possible there are complexities I'm not thinking of.

I think the two instances in that file are the only place it's used in a compiled context, and I think it can be replaced by the Compute version there.

While we can logically replace them, we cannot assume that the code on our customers computer will ~~not~~ be updated.

i.e: Our customers will still have references to the old version, and rely on the references being provided by the MPL.

Unless both of the references in the DB-ESDK, and any references in the ESDK, are only in ghost code and have ALWAYS been in only in ghost code.

@atomb Could you make this a function by method instead and use the ComputeSetToSequence2 implementation in the by method body?

Actually I don't think you have to change this anyway, I don't get an error when I do dafny verify src/StandardLibrary.dfy --function-syntax:3 --enforce-determinism.

Since this :| is in an expression context it already has to be provably deterministic.

Oh, you are indeed correct. I've changed it back.

josecorella

starred long and hard at the change and believe this is a fine change.

texastony

Blocking until breaking change issue is resolved.

error

Breaking change has been resolved

seebees

These changes look good.

seebees · 2024-08-22T19:09:22Z

...yptographicMaterialProviders/dafny/AwsCryptographicMaterialProviders/src/CMMs/DefaultCMM.dfy

-                      && key in input.reproducedEncryptionContext.value
+                      && key in input.reproducedEncryptionContext.value.Keys
                      && key in input.encryptionContext
-                      && key !in keysSet
-                      :: input.reproducedEncryptionContext.value[key] == input.encryptionContext[key]
+                      && key !in keysSet'
+                      :: input.encryptionContext[key] == input.reproducedEncryptionContext.value[key]


Slight nit, adjusting the verification makes it a little hard to verify the lack of change :)
This way would have made it clear that the only actual change was to change the invariant to use the ghost value.
and that everything else was additions.

&& key in input.reproducedEncryptionContext.value && key in input.encryptionContext && key !in keysSet' :: input.reproducedEncryptionContext.value[key] == input.encryptionContext[key]

Good point. Those changes are an artifact of attempting to reduce brittleness, before I realized that it was actually missing information leading to failure here. They're not necessary, and I'd be happy to undo them.

seebees · 2024-08-22T19:14:52Z

...lProviders/dafny/AwsCryptographicMaterialProviders/src/CMMs/RequiredEncryptionContextCMM.dfy

-      var keySeq := [];
-      while keySet != {}
-        invariant |keySeq| + |keySet| == |inputKeys|
-        invariant forall k <- keySeq
-                    :: k in inputKeys
-      {
-        var key :| key in keySet;
-        keySeq := keySeq + [key];
-        keySet := keySet - {key};
-      }
+      var keySeq := SortedSets.ComputeSetToSequence(keySet);


Similarly here, I think that we could see the diff more clearly as invariant by writing it this way.
(I checked the ensures clause)
This way the invariant changes to an assert.

var keySeq := SortedSets.ComputeSetToSequence(inputKeys); assert |keySeq| == |inputKeys|; assert forall k <- keySeq :: k in inputKeys;

seebees · 2024-08-22T19:17:41Z

...MaterialProviders/dafny/AwsCryptographicMaterialProviders/src/CanonicalEncryptionContext.dfy

-    var keys := SetToOrderedSequence(encryptionContext.Keys, UInt.UInt8Less);
+    var keys := SortedSets.ComputeSetToOrderedSequence2(encryptionContext.Keys, UInt.UInt8Less);


Looking at SortedSets.ComputeSetToOrderedSequence2 I can see ensures res == SetToOrderedSequence(s, less) so this MUST be the same.

seebees · 2024-08-22T19:19:57Z

AwsCryptographyPrimitives/test/TestRSA.dfy

These testing changes look fine.

seebees

Wonderful!

Removes uses of :| in Dafny code, and updates the libraries submodule to a version that does not use it, either. This will eliminate failures when compiling with Dafny 4.8.0 and -definiteAssignment:3.

* add ECDH error message for Rust ([#574](#574)) ([473a34a](473a34a)) * **DDB-Model:** DDB Supports 100 actions per Transaction ([#692](#692)) ([8a67843](8a67843)) * GetCurrentTimeStamp returns ISO8601 format ([#575](#575)) ([c07a51f](c07a51f)) * maintain order in test vectors for languages with parallel tests ([#641](#641)) ([8c8a38f](8c8a38f)) * Remove 4.4 DDB and KMS patches, abstract test to work on later Dafny versions ([#611](#611)) ([d51d648](d51d648)) * Remove uses of `:|` ([#618](#618)) ([f12fe5b](f12fe5b)) * test vector help text ([#657](#657)) ([0fedaf1](0fedaf1)) * bump dafny verification and code gen to dafny 4.8.0 ([#520](#520)) ([e16539e](e16539e)) * **post-release:** Change back to 1.5.1-SNAPSHOT ([09cd9a4](09cd9a4))

Removes uses of :| in Dafny code, and updates the libraries submodule to a version that does not use it, either. This will eliminate failures when compiling with Dafny 4.8.0 and -definiteAssignment:3.

Update libraries submodule

3732a7a

atomb requested a review from a team as a code owner August 21, 2024 15:17

atomb temporarily deployed to Duvet CI August 21, 2024 15:17 — with GitHub Actions Inactive

fix: Remove uses of :|

b7e3b48

In conjunction with the update of the libraries submodule, this will allow all code to be compiled with `-definiteAssignment:3` without Failures with the upcoming Dafny 4.8.0.

atomb had a problem deploying to Duvet CI August 21, 2024 16:07 — with GitHub Actions Error

atomb changed the title ~~Update libraries submodule~~ fix: Remove uses of :| Aug 21, 2024

atomb temporarily deployed to Duvet CI August 21, 2024 16:10 — with GitHub Actions Inactive

texastony marked this pull request as draft August 21, 2024 16:13

Fix one more use of :|

3263c42

atomb temporarily deployed to Duvet CI August 21, 2024 16:27 — with GitHub Actions Inactive

Call function form of ComputeSetToOrderedSequence

1a240ee

atomb temporarily deployed to Duvet CI August 21, 2024 16:34 — with GitHub Actions Inactive

Fix proof of DecryptMaterials'

53d3508

atomb temporarily deployed to Duvet CI August 21, 2024 20:42 — with GitHub Actions Inactive

atomb marked this pull request as ready for review August 21, 2024 21:16

robin-aws reviewed Aug 21, 2024

View reviewed changes

josecorella previously approved these changes Aug 21, 2024

View reviewed changes

texastony previously requested changes Aug 21, 2024

View reviewed changes

Make SetToOrderedSequence compilable again

a79e9f5

atomb temporarily deployed to Duvet CI August 22, 2024 15:04 — with GitHub Actions Inactive

seebees previously approved these changes Aug 22, 2024

View reviewed changes

Reduce and clarify difference in verification

05d7945

atomb dismissed seebees’s stale review via 05d7945 August 22, 2024 20:16

atomb temporarily deployed to Duvet CI August 22, 2024 20:16 — with GitHub Actions Inactive

Fix formatting

55c9f38

atomb temporarily deployed to Duvet CI August 22, 2024 20:42 — with GitHub Actions Inactive

seebees approved these changes Aug 22, 2024

View reviewed changes

robin-aws merged commit f12fe5b into main Aug 22, 2024
70 checks passed

robin-aws deleted the new-libraries branch August 22, 2024 21:17

josecorella mentioned this pull request Sep 10, 2024

chore(release): 1.6.0 #703

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Remove uses of `:|` #618

fix: Remove uses of `:|` #618

atomb commented Aug 21, 2024 •

edited

Loading

texastony commented Aug 21, 2024

robin-aws Aug 21, 2024

atomb Aug 21, 2024

atomb Aug 21, 2024

texastony Aug 21, 2024 •

edited

Loading

robin-aws Aug 22, 2024

robin-aws Aug 22, 2024

atomb Aug 22, 2024

josecorella left a comment

texastony left a comment

seebees left a comment

seebees Aug 22, 2024

atomb Aug 22, 2024

seebees Aug 22, 2024

seebees Aug 22, 2024

seebees Aug 22, 2024

seebees left a comment

		var keys := SetToOrderedSequence(encryptionContext.Keys, UInt.UInt8Less);
		var keys := SortedSets.ComputeSetToOrderedSequence2(encryptionContext.Keys, UInt.UInt8Less);

fix: Remove uses of :| #618

fix: Remove uses of :| #618

Conversation

atomb commented Aug 21, 2024 • edited Loading

texastony commented Aug 21, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

texastony Aug 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

josecorella left a comment

Choose a reason for hiding this comment

texastony left a comment

Choose a reason for hiding this comment

seebees left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

seebees left a comment

Choose a reason for hiding this comment

fix: Remove uses of `:|` #618

fix: Remove uses of `:|` #618

atomb commented Aug 21, 2024 •

edited

Loading

texastony Aug 21, 2024 •

edited

Loading