Docs and refactoring from discussions on PR ethereum#2457

- Add documentation
- Refactor out the provider flag since there is no support for sending transactions w/ CCIP Read support
- Refactor some around the way the ``eth_call`` is made, add flag directly to ``eth_call`` and make ``durin_call()`` and internal ``_durin_call()``
- Refactor data back into post request after linting issues went away
- Make CCIP Read max redirects configurable and add tests
- OffchainLookup should extend from ContractLogicError
- Add some more testing and refactor some more; use web3 ValidationError instead of importing from eth_utils
- Add MultipleFailedRequests if multiple failed requests for similar / same data

docs/contracts.rst

@@ -743,6 +743,20 @@ Positional and keyword arguments supplied to the contract function subclass
 will be used to find the contract function by signature,
 and forwarded to the contract function when applicable.
 
+`EIP-3668 <https://eips.ethereum.org/EIPS/eip-3668>`_ introduced support for the ``OffchainLookup`` revert /
+CCIP Read support. The ``ccip_read_enabled`` flag is set to ``True`` for calls by default, as recommended in EIP-3668.
+If raising the ``OffchainLookup`` revert is preferred, the flag may be set to ``False`` on a per-call basis.
+
+.. code-block:: python
+
+        >>> myContract.functions.revertsWithOffchainLookup(myData).call(ccip_read_enabled=False)
+        *** web3.exceptions.OffchainLookup
+
+Disabling CCIP Read support can be useful if a transaction needs to be sent to the callback function. In such cases,
+"preflighting" with an ``eth_call``, handling the ``OffchainLookup``, and sending the data via a transaction may be
+necessary. See :ref:`ccip-read-example` in the examples section for how to preflight a transaction with a contract call.
+
+
 Methods
 ~~~~~~~~~~
 

docs/examples.rst

@@ -624,6 +624,44 @@ When someone has an allowance they can transfer those tokens using the
 .. _ERC20: https://github.com/ethereum/EIPs/blob/7f4f0377730f5fc266824084188cc17cf246932e/EIPS/eip-20.md
 
 
+.. _ccip-read-example:
+
+CCIP Read support for offchain lookup
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Contract calls support CCIP Read by default via a ``ccip_read_enabled`` flag that is set to a default value of ``True``.
+The following should work by default without raising the ``OffchainLookup`` and instead handling it appropriately as
+per the specification outlined in `EIP-3668 <https://eips.ethereum.org/EIPS/eip-3668>`_.
+
+.. code-block:: python
+
+    myContract.functions.revertsWithOffchainLookup(myData).call()
+
+If the offchain lookup requires the user to send a transaction rather than make a call, this may be handled
+appropriately in the following way:
+
+.. code-block:: python
+
+    from web3 import Web3, WebsocketProvider
+    from web3.utils import handle_offchain_lookup
+
+    w3 = Web3(WebsocketProvider(...))
+
+    myContract = w3.eth.contract(address=...)
+    myData = b'data for offchain lookup function call'
+
+    # preflight with an `eth_call` and handle the exception
+    try:
+        myContract.functions.revertsWithOffchainLookup(myData).call(ccip_read_enabled=False)
+    except OffchainLookup as ocl:
+        tx = {'to': myContract.address, 'from': my_account}
+        data_for_callback_function = handle_offchain_lookup(ocl.payload)
+        tx['data'] = data_for_callback_function
+
+        # send the built transaction with `eth_sendTransaction` or sign and send with `eth_sendRawTransaction`
+        tx_hash = w3.eth.send_transaction(tx)
+
+
 Contract Unit Tests in Python
 -----------------------------
 

docs/index.rst

@@ -67,6 +67,7 @@ Table of Contents
     web3.miner
     web3.geth
     web3.parity
+    web3.utils
     gas_price
     ens
     constants

docs/web3.eth.rst

@@ -1064,7 +1064,7 @@ The following methods are available on the ``web3.eth`` namespace.
     .. warning:: Deprecated: This property is deprecated in favor of
       :meth:`~web3.eth.Eth.sign_typed_data()`
 
-.. py:method:: Eth.call(transaction, block_identifier=web3.eth.default_block, state_override=None)
+.. py:method:: Eth.call(transaction, block_identifier=web3.eth.default_block, state_override=None, ccip_read_enabled=True)
 
     * Delegates to ``eth_call`` RPC Method
 
@@ -1090,6 +1090,15 @@ The following methods are available on the ``web3.eth`` namespace.
     View their `usage documentation <https://geth.ethereum.org/docs/rpc/ns-eth#3-object---state-override-set>`_
     for a list of possible parameters.
 
+    `EIP-3668 <https://eips.ethereum.org/EIPS/eip-3668>`_ introduced support for the ``OffchainLookup`` revert / CCIP
+    Read support. In order to properly handle a call to a contract function that reverts with an ``OffchainLookup``
+    error for offchain data retrieval, the ``ccip_read_enabled`` flag has been added to the ``eth_call`` method.
+    ``ccip_read_enabled`` is set to ``True`` by default for calls, as recommended in EIP-3668. Therefore, calls to
+    contract functions that revert with an ``OffchainLookup`` will be handled appropriately by default. If the
+    ``ccip_read_enabled`` flag is set to ``False``, the call will raise the ``OffchainLookup`` instead of properly
+    handling the exception according to EIP-3668. This may be useful for "preflighting" a transaction call (see
+    :ref:`ccip-read-example` within the examples section).
+
 
 .. py:method:: Eth.fee_history(block_count, newest_block, reward_percentiles=None)
 

docs/web3.utils.rst

@@ -0,0 +1,17 @@
+Utils
+=====
+
+.. py:module:: web3.utils
+
+The ``utils`` module houses public utility and helper functions.
+
+
+.. py:method:: Utils.handle_offchain_lookup(offchain_lookup_payload, transaction)
+
+    Handle ``OffchainLookup`` reverts on contract function calls manually. For an example, see :ref:`ccip-read-example`
+    within the examples section.
+
+
+.. py:method:: Utils.async_handle_offchain_lookup(offchain_lookup_payload, transaction)
+
+    The async version of the ``handle_offchain_lookup()`` utility method described above.

tests/core/contracts/test_offchain_lookup.py

@@ -4,19 +4,20 @@
     decode_abi,
 )
 
+from web3._utils.module_testing.module_testing_utils import (
+    mock_offchain_lookup_request_response,
+)
 from web3._utils.module_testing.offchain_lookup_contract import (
     OFFCHAIN_LOOKUP_ABI,
     OFFCHAIN_LOOKUP_BYTECODE,
     OFFCHAIN_LOOKUP_BYTECODE_RUNTIME,
 )
-from web3._utils.module_testing.utils import (
-    mock_offchain_lookup_request_response,
-)
-from web3._utils.type_conversion_utils import (
+from web3._utils.type_conversion import (
     to_hex_if_bytes,
 )
 from web3.exceptions import (
     TooManyRequests,
+    ValidationError,
 )
 
 # "test offchain lookup" as an abi-encoded string
@@ -64,6 +65,20 @@ def test_offchain_lookup_functionality(
     assert decode_abi(['string'], response)[0] == 'web3py'
 
 
+def test_offchain_lookup_raises_for_improperly_formatted_rest_request_response(
+    offchain_lookup_contract, monkeypatch,
+):
+    normalized_address = to_hex_if_bytes(offchain_lookup_contract.address)
+    mock_offchain_lookup_request_response(
+        monkeypatch,
+        mocked_request_url=f'https://web3.py/gateway/{normalized_address}/{OFFCHAIN_LOOKUP_TEST_DATA}.json',  # noqa: E501
+        mocked_json_data=WEB3PY_AS_HEXBYTES,
+        json_data_field='not_data',
+    )
+    with pytest.raises(ValidationError, match="missing 'data' field"):
+        offchain_lookup_contract.caller.testOffchainLookup(OFFCHAIN_LOOKUP_TEST_DATA)
+
+
 @pytest.mark.parametrize('status_code_non_4xx_error', [100, 300, 500, 600])
 def test_eth_call_offchain_lookup_tries_next_url_for_non_4xx_error_status_and_tests_POST(
     offchain_lookup_contract, monkeypatch, status_code_non_4xx_error,

tests/core/web3-module/test_providers.py

@@ -23,3 +23,7 @@ def test_auto_provider_none():
     w3.toHex(0) == '0x0'
 
     type(w3.provider) == AutoProvider
+
+
+def test_provider_default_value_for_ccip_read_redirect(w3):
+    assert w3.provider.ccip_read_max_redirects == 4

tests/ens/test_offchain_resolution.py

@@ -1,18 +1,13 @@
 import pytest
 
-from eth_abi.abi import (
-    decode_abi,
-)
-from eth_utils import (
-    to_checksum_address,
-)
 import requests
 
 from ens.utils import (
     ens_encode_name,
 )
 from web3.exceptions import (
     OffchainLookup,
+    ValidationError,
 )
 
 # the encoded calldata for the initiating ``addr(namehash(name))`` call
@@ -106,43 +101,25 @@ def mock_get(*args, **_):
         return MockHttpBadFormatResponse(*args)
 
     monkeypatch.setattr(requests.Session, 'get', mock_get)
-    with pytest.raises(Exception, match=(
+    with pytest.raises(ValidationError, match=(
         "Improperly formatted response for offchain lookup HTTP request - missing 'data' field."
     )):
         ens.address('offchainexample.eth')
 
 
-def test_offchain_resolver_function_call_with_ccip_read_enabled(ens, monkeypatch):
-    def mock_get(*args, **kwargs):
-        return MockHttpSuccessResponse('get', *args, **kwargs)
-
-    monkeypatch.setattr(requests.Session, 'get', mock_get)
-
+def test_offchain_resolver_function_call_raises_with_ccip_read_disabled(ens, monkeypatch):
     offchain_resolver = ens.resolver('offchainexample.eth')
 
-    # set global ccip_read_enabled flag on provider to False
-    ens.w3.provider.ccip_read_enabled = False
-
-    # should fail here with global provider flag set to False
+    # should fail here with `ccip_read_enabled` flag set to False
     with pytest.raises(OffchainLookup):
         offchain_resolver.functions.resolve(
             ens_encode_name('offchainexample.eth'),
             ENCODED_ADDR_CALLDATA,
-        ).call()
-
-    # pass flag on specific call should work
-    resolved_via_function_call = offchain_resolver.functions.resolve(
-        ens_encode_name('offchainexample.eth'),
-        ENCODED_ADDR_CALLDATA,
-    ).call(ccip_read_enabled=True)
+        ).call(ccip_read_enabled=False)
 
     # pass flag on specific call via ContractCaller is also an option
-    resolved_via_caller = offchain_resolver.caller(ccip_read_enabled=True).resolve(
-        ens_encode_name('offchainexample.eth'),
-        ENCODED_ADDR_CALLDATA,
-    )
-
-    assert resolved_via_caller == resolved_via_function_call
-
-    decoded_result = decode_abi(['address'], resolved_via_caller)[0]
-    assert to_checksum_address(decoded_result) == EXPECTED_RESOLVED_ADDRESS
+    with pytest.raises(OffchainLookup):
+        offchain_resolver.caller(ccip_read_enabled=False).resolve(
+            ens_encode_name('offchainexample.eth'),
+            ENCODED_ADDR_CALLDATA,
+        )

web3/_utils/async_transactions.py

@@ -13,9 +13,6 @@
 from eth_typing import (
     URI,
 )
-from eth_utils import (
-    ValidationError,
-)
 
 from web3._utils.request import (
     async_get_json_from_client_response,
@@ -26,6 +23,9 @@
     to_bytes_if_hex,
     to_hex_if_bytes,
 )
+from web3.exceptions import (
+    ValidationError,
+)
 from eth_utils.toolz import (
     curry,
     merge,
@@ -183,7 +183,7 @@ async def async_handle_offchain_lookup(
         result = await async_get_json_from_client_response(response)
 
         if 'data' not in result.keys():
-            raise Exception(
+            raise ValidationError(
                 "Improperly formatted response for offchain lookup HTTP request - missing 'data' "
                 "field."
             )