ranged-based for for user-defined types

proposals/p1885.md

@@ -43,6 +43,8 @@ SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
         -   [Alternative views](#alternative-views)
     -   [Large elements and Optional](#large-elements-and-optional)
     -   [C++ interoperability](#c-interoperability)
+        -   [Iterating over C++ types in Carbon](#iterating-over-c-types-in-carbon)
+        -   [Iterating over Carbon types in C++](#iterating-over-carbon-types-in-c)
     -   [Inversion of control](#inversion-of-control)
 -   [Rationale](#rationale)
 -   [Alternatives considered](#alternatives-considered)
@@ -250,7 +252,7 @@ interface Iterate {
 A naive Carbon implementation of the for loop could be:
 
 ```carbon
-let cursor: range.(Iterate.CursorType) = range.(Iterate.NewCursor)();
+var cursor: range.(Iterate.CursorType) = range.(Iterate.NewCursor)();
 var iter: Optional(range.(Iterate.ElementType)) = range.(Iterate.Next)(&cursor);
 
 // A. Possible implementation
@@ -582,13 +584,55 @@ for (v: my_dict.(Iterate.ElementType) in my_dict.reversed) {}
 Large elements would have a negative impact if the `Optional` cannot leverage
 unused bit patterns within the type, nor present a view instead of a copy.
 
-Such opportunities for `Optional` exist, and could be addressed in the future.
+If this proves difficult, we can let implementers of the container choose
+whether to expose values or pointers to values. When electing to "pointers to
+values", an adapter can be provided to support transforming the range of
+pointers into a range a values. This choice allows supporting either iteration
+over container r-values (`ElementType = T`), or efficient iteration
+(`ElementType = T*`), but not both. While acceptable as an intermediate
+solution, a better alternative will be needed in the future.
+
+```carbon
+adapter IterateByValue(template T:! type) for T {
+  impl as Iterate where
+      // Speculative syntax. Alternatively, a cursor type parameter
+      // would avoid the need for deduction.
+      where .CursorType = T.CursorType
+        and .ElementType = T.CursorType.(Deref.Result) {
+        ...
+  }
+}
+
+// Used like:
+var frames: NetworkFramesContainerByPtr = ...
+for (i: NetworkFrame in frames as IterateByValue(NetworkFramesContainerByPtr))
+{
+  ...
+}
+```
+
+Alternatively, we can allow the binding in the `for` loop to be an `addr`
+pattern, implemented by calling into a separate interface on the container. This
+puts the choice of using values or pointers to values in the hands of the `for`
+loop author. While this deviates from the ergonomics of `let` (where the
+compiler can choose to copy or alias the value depending on what is more
+efficient), this option would provide parity with C++ `for` loops (where the
+author chooses between value and reference).
 
 ### C++ interoperability
 
-The limitation of this approach is the need to interoperate with the C++
-iterator model. This could be implemented by having the cursor be an iterator
-type:
+#### Iterating over C++ types in Carbon
+
+For reference, range-based `for` loops in C++ requires:
+
+-   `begin()` and `end()` methods or free functions
+-   supporting pre-increment operator `++` and indirection operator `*`
+
+See section 8.6.4 of N4860 for more details.
+
+A limitation of the approach proposed in this document is the need for
+adaptation to interoperate with the C++ iterator model. This could be
+implemented by having the cursor be an iterator type:
 
 -   `NewCursor` providing the "start iterator"
 -   `Next` doing the "increment, bounds check, and returning value"
@@ -657,6 +701,91 @@ for (i: i32 in v as CppIterate(Cpp.std.vector(i32))) { ... }
 These two options could allow interoperability with a compatible C++ type,
 either implicitly (template approach) or explicitly (adapter approach).
 
+#### Iterating over Carbon types in C++
+
+C++ needs to provide `begin()` and `end()` methods or free functions, in
+addition to `*` and `++` operators.
+
+The `Iterate` interface does not expose a way to have a cursor nor an iterator
+to the past-last element. An option would be to have `end()` return a predefined
+invalid value, to satisfy iterator comparison when `Next()` returns an empty
+`Optional`.
+
+Below is sample implementation to iterate on a Carbon `Iterate`. Note that we
+need to copy and cache the last value, to allow for `*it` to be called. This
+puts a requirement on `ElementType` to be copyable for this interoperability to
+be usable.
+
+```cpp
+namespace Carbon {
+// A C++ input iterator for any type that implements the Carbon `Iterate` interface.
+template <typename Iterate>
+class IterateIterator {
+ public:
+  // Returns an "Invalid" iterator representing `end()`
+  static auto Invalid(const Iterate& container) -> IterateIterator<Iterate> {
+    return IterateIterator<Iterate>(container, std::nullopt);
+  }
+
+  IterateIterator(const Iterate& container,
+                  std::optional<typename Iterate::CursorType> cursor)
+      : container_(&container), cursor_(cursor) {
+    if (cursor_) {
+      next();
+    }
+  }
+  IterateIterator(const IterateIterator& other)
+      : container_(other.container_), cursor_(other.cursor_),
+      value_(other) {}
+
+  auto operator*() const -> const typename Iterate::ElementType& {
+    assert(cursor_);
+    return value_;
+  }
+  auto operator++() -> IterateIterator<Iterate>& {
+    next();
+    return *this;
+  }
+  auto operator==(const IterateIterator<Iterate>& rhs) -> bool {
+    return container_ == rhs.container_ && cursor_ == rhs.cursor_;
+  }
+  auto operator!=(const IterateIterator<Iterate>& rhs) -> bool {
+    return !(*this == rhs);
+  }
+
+ private:
+  void next() {
+    assert(cursor_);
+    if (const auto v = container_->Next(*cursor_); v) {
+      value_ = *v;
+    } else {
+      cursor_ = std::nullopt;
+    }
+  }
+
+  const Iterate* container_;
+  std::optional<typename Iterate::CursorType> cursor_;
+  typename Iterate::ElementType value_;
+};
+
+// A type implementing `Iterate` with CursorType `C`, and
+// ElementType `T` could have `begin()` and `end()` methods
+// defined as such:
+class IterateType ... {
+  auto begin() -> IterateIterator<Iterable<C, T>> {
+    return IterateIterator<Iterable<C, T>>(*this, NewCursor());
+  }
+  auto end() -> IterateIterator<Iterable<C, T>> {
+    return IterateIterator<Iterable<C, T>>::Invalid(*this);
+  }
+};
+}  // namespace Carbon
+
+// Usage:
+Carbon::MyContainer container;
+for (auto s : container) { ...}
+```
+
 ### Inversion of control
 
 Using an `Optional` has downsides, discussed in the