Skip to content

Commit

Permalink
Add getPayloadBodiesByRangeV1 and getPayloadBodiesByHash engine metho…
Browse files Browse the repository at this point in the history
…ds (hyperledger#4980)

* Add engine get payload body methods and test

Signed-off-by: Zhenyang Shi <[email protected]>
Signed-off-by: Gabriel Fukushima <[email protected]>

* Add header

Signed-off-by: Zhenyang Shi <[email protected]>

* Update result struct & add test

Signed-off-by: Zhenyang Shi <[email protected]>

* Change constant to use upper case

Signed-off-by: Gabriel Fukushima <[email protected]>

* Add PayloadBody class and withdrawals to response of methods

Signed-off-by: Gabriel Fukushima <[email protected]>

* Add unit tests

Signed-off-by: Gabriel Fukushima <[email protected]>

* Add changelog

Signed-off-by: Gabriel Fukushima <[email protected]>

* spotless

Signed-off-by: Gabriel Fukushima <[email protected]>

* Add check to prevent returning trailing null results past the head

Signed-off-by: Gabriel Fukushima <[email protected]>

* Add test to check trailing null post head scenario

Signed-off-by: Gabriel Fukushima <[email protected]>

* Split tests into pre and post shanghai

Signed-off-by: Gabriel Fukushima <[email protected]>

* spotless

Signed-off-by: Gabriel Fukushima <[email protected]>

* Rename methods

Signed-off-by: Gabriel Fukushima <[email protected]>

* Use getName() to log method name

Signed-off-by: Gabriel Fukushima <[email protected]>

* spotless

Signed-off-by: Gabriel Fukushima <[email protected]>

* Rename variable

Signed-off-by: Gabriel Fukushima <[email protected]>

* Call constructor directly

Signed-off-by: Gabriel Fukushima <[email protected]>

* Fix ByHash json parsing
Signed-off-by: Simon Dudley <[email protected]>

* Fix json parsing again
Signed-off-by: Simon Dudley <[email protected]>

* Add check to prevent unnecessary queries

Signed-off-by: Gabriel Fukushima <[email protected]>

* Refactor method

Signed-off-by: Gabriel Fukushima <[email protected]>

* Add new error code for EngineGetPayloadBodies methods

Signed-off-by: Gabriel Fukushima <[email protected]>

* Add return error for request above the API limit

Signed-off-by: Gabriel Fukushima <[email protected]>

* Add constructor for empty response

Signed-off-by: Gabriel Fukushima <[email protected]>

* add check for number of blocks requested and for requests of post head

Signed-off-by: Gabriel Fukushima <[email protected]>

* Add test to check error code when request exceeds API limits

Signed-off-by: Gabriel Fukushima <[email protected]>

* add constant for max blocks allowed per request

Signed-off-by: Gabriel Fukushima <[email protected]>

* spotless

Signed-off-by: Gabriel Fukushima <[email protected]>

* Fix some nits

Signed-off-by: Gabriel Fukushima <[email protected]>

* Add invalid params check

Signed-off-by: Gabriel Fukushima <[email protected]>

* Add tests for invalid params check

Signed-off-by: Gabriel Fukushima <[email protected]>

* Fix test and spotless

Signed-off-by: Gabriel Fukushima <[email protected]>

* Revert "Fix json parsing again"

This reverts commit 558d325.

Signed-off-by: Gabriel Fukushima <[email protected]>

* Revert "Fix ByHash json parsing Signed-off-by: Simon Dudley <[email protected]>"

This reverts commit 663e11e

Signed-off-by: Gabriel Fukushima <[email protected]>

* Use UnsignedLongParameter to cast params of the request

Signed-off-by: Gabriel Fukushima <[email protected]>

* Add optional withdrawals to the NewPayload log (hyperledger#5021)

Signed-off-by: Simon Dudley <[email protected]>

* kubernetes and errorprone - update versions (hyperledger#5013)

* update errorprone and kubernetes versions
* fixed errorprone issues in prod cod
* fixed errorprone issues in test code

---------

Signed-off-by: Sally MacFarlane <[email protected]>

* Rename JsonRpcService to EngineJsonRpcService (hyperledger#5036)

* rename JsonRpcService to EngineJsonRpcService

Signed-off-by: Daniel Lehrner <[email protected]>

* Params should be single item of array type, not outer array of strings (hyperledger#5037)

Signed-off-by: Simon Dudley <[email protected]>

* Add EIP-2537 (BLS12-381 curve precompiles) to Cancun (hyperledger#5017)

Add the BLS curve precompiles into the registry for cancun.  All of the
curve precompiles have been here since berlin, so this is just wiring
them in.

Signed-off-by: Danno Ferrin <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>

* Use UnsignedLongParameter to cast params of the request

Signed-off-by: Gabriel Fukushima <[email protected]>

* Add optional withdrawals to the NewPayload log (hyperledger#5021)

Signed-off-by: Simon Dudley <[email protected]>

* kubernetes and errorprone - update versions (hyperledger#5013)

* update errorprone and kubernetes versions
* fixed errorprone issues in prod cod
* fixed errorprone issues in test code

---------

Signed-off-by: Sally MacFarlane <[email protected]>

* Rename JsonRpcService to EngineJsonRpcService (hyperledger#5036)

* rename JsonRpcService to EngineJsonRpcService

Signed-off-by: Daniel Lehrner <[email protected]>

* Params should be single item of array type, not outer array of strings (hyperledger#5037)

Signed-off-by: Simon Dudley <[email protected]>

* Add EIP-2537 (BLS12-381 curve precompiles) to Cancun (hyperledger#5017)

Add the BLS curve precompiles into the registry for cancun.  All of the
curve precompiles have been here since berlin, so this is just wiring
them in.

Signed-off-by: Danno Ferrin <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>

* Convert start and count from hex to match JSON-RPC Spec standard

Signed-off-by: Gabriel Fukushima <[email protected]>

---------

Signed-off-by: Zhenyang Shi <[email protected]>
Signed-off-by: Gabriel Fukushima <[email protected]>
Signed-off-by: Simon Dudley <[email protected]>
Signed-off-by: Sally MacFarlane <[email protected]>
Signed-off-by: Daniel Lehrner <[email protected]>
Signed-off-by: Danno Ferrin <[email protected]>
Co-authored-by: Zhenyang Shi <[email protected]>
Co-authored-by: Simon Dudley <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
Co-authored-by: Daniel Lehrner <[email protected]>
Co-authored-by: Danno Ferrin <[email protected]>
  • Loading branch information
6 people authored Feb 9, 2023
1 parent fd90643 commit 05bf6ab
Show file tree
Hide file tree
Showing 10 changed files with 919 additions and 1 deletion.
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ tests are updated to use EC private keys instead of RSA keys.
- Improve get account performance by using the world state updater cache [#4897](https://github.com/hyperledger/besu/pull/4897)
- Add new KZG precompile and option to override the trusted setup being used [#4822](https://github.com/hyperledger/besu/issues/4822)
- Add implementation for eth_createAccessList RPC method [#4942](https://github.com/hyperledger/besu/pull/4942)
- Add implementation for engine_exchangeCapabilities [#4997](https://github.com/hyperledger/besu/pull/4997)
- Add implementation for engine_getPayloadBodiesByRangeV1 and engine_getPayloadBodiesByHashV1 [#4980](https://github.com/hyperledger/besu/pull/4980)
- Updated reference tests to v11.3 [#4996](https://github.com/hyperledger/besu/pull/4996)
- Add DebugGetRawBlock and DebugGetRawHeader RPC methods [#5011](https://github.com/hyperledger/besu/pull/5011)
- Besu requires minimum Java 17 and up to build and run [#3320](https://github.com/hyperledger/besu/issues/3320)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ public enum RpcMethod {
ENGINE_FORKCHOICE_UPDATED_V1("engine_forkchoiceUpdatedV1"),
ENGINE_FORKCHOICE_UPDATED_V2("engine_forkchoiceUpdatedV2"),
ENGINE_EXCHANGE_TRANSITION_CONFIGURATION("engine_exchangeTransitionConfigurationV1"),
ENGINE_GET_PAYLOAD_BODIES_BY_HASH_V1("engine_getPayloadBodiesByHashV1"),
ENGINE_GET_PAYLOAD_BODIES_BY_RANGE_V1("engine_getPayloadBodiesByRangeV1"),
ENGINE_EXCHANGE_CAPABILITIES("engine_exchangeCapabilities"),

GOQUORUM_ETH_GET_QUORUM_PAYLOAD("eth_getQuorumPayload"),
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
/*
* Copyright Hyperledger Besu Contributors.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine;

import static org.hyperledger.besu.util.Slf4jLambdaHelper.traceLambda;

import org.hyperledger.besu.datatypes.Hash;
import org.hyperledger.besu.ethereum.ProtocolContext;
import org.hyperledger.besu.ethereum.api.jsonrpc.RpcMethod;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.JsonRpcRequestContext;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.ExecutionEngineJsonRpcMethod;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcError;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcErrorResponse;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcResponse;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcSuccessResponse;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.results.BlockResultFactory;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.results.EngineGetPayloadBodiesResultV1;
import org.hyperledger.besu.ethereum.chain.Blockchain;

import java.util.Arrays;
import java.util.stream.Collectors;

import io.vertx.core.Vertx;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class EngineGetPayloadBodiesByHashV1 extends ExecutionEngineJsonRpcMethod {

private static final int MAX_REQUEST_BLOCKS = 1024;
private static final Logger LOG = LoggerFactory.getLogger(EngineGetPayloadBodiesByHashV1.class);
private final BlockResultFactory blockResultFactory;

public EngineGetPayloadBodiesByHashV1(
final Vertx vertx,
final ProtocolContext protocolContext,
final BlockResultFactory blockResultFactory,
final EngineCallListener engineCallListener) {
super(vertx, protocolContext, engineCallListener);
this.blockResultFactory = blockResultFactory;
}

@Override
public String getName() {
return RpcMethod.ENGINE_GET_PAYLOAD_BODIES_BY_HASH_V1.getMethodName();
}

@Override
public JsonRpcResponse syncResponse(final JsonRpcRequestContext request) {
engineCallListener.executionEngineCalled();

final Object reqId = request.getRequest().getId();

final Hash[] blockHashes = request.getRequiredParameter(0, Hash[].class);

traceLambda(LOG, "{} parameters: blockHashes {}", () -> getName(), () -> blockHashes);

if (blockHashes.length > getMaxRequestBlocks()) {
return new JsonRpcErrorResponse(reqId, JsonRpcError.INVALID_RANGE_REQUEST_TOO_LARGE);
}

final Blockchain blockchain = protocolContext.getBlockchain();

final EngineGetPayloadBodiesResultV1 engineGetPayloadBodiesResultV1 =
blockResultFactory.payloadBodiesCompleteV1(
Arrays.stream(blockHashes).map(blockchain::getBlockBody).collect(Collectors.toList()));

return new JsonRpcSuccessResponse(reqId, engineGetPayloadBodiesResultV1);
}

protected int getMaxRequestBlocks() {
return MAX_REQUEST_BLOCKS;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
/*
* Copyright Hyperledger Besu Contributors.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine;

import static org.hyperledger.besu.util.Slf4jLambdaHelper.traceLambda;

import org.hyperledger.besu.ethereum.ProtocolContext;
import org.hyperledger.besu.ethereum.api.jsonrpc.RpcMethod;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.JsonRpcRequestContext;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.ExecutionEngineJsonRpcMethod;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.UnsignedLongParameter;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcError;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcErrorResponse;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcResponse;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcSuccessResponse;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.results.BlockResultFactory;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.results.EngineGetPayloadBodiesResultV1;
import org.hyperledger.besu.ethereum.chain.Blockchain;

import java.util.stream.Collectors;
import java.util.stream.LongStream;

import io.vertx.core.Vertx;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class EngineGetPayloadBodiesByRangeV1 extends ExecutionEngineJsonRpcMethod {
private static final Logger LOG = LoggerFactory.getLogger(EngineGetPayloadBodiesByRangeV1.class);
private static final int MAX_REQUEST_BLOCKS = 1024;
private final BlockResultFactory blockResultFactory;

public EngineGetPayloadBodiesByRangeV1(
final Vertx vertx,
final ProtocolContext protocolContext,
final BlockResultFactory blockResultFactory,
final EngineCallListener engineCallListener) {
super(vertx, protocolContext, engineCallListener);
this.blockResultFactory = blockResultFactory;
}

@Override
public String getName() {
return RpcMethod.ENGINE_GET_PAYLOAD_BODIES_BY_RANGE_V1.getMethodName();
}

@Override
public JsonRpcResponse syncResponse(final JsonRpcRequestContext request) {
engineCallListener.executionEngineCalled();

final long startBlockNumber =
request.getRequiredParameter(0, UnsignedLongParameter.class).getValue();
final long count = request.getRequiredParameter(1, UnsignedLongParameter.class).getValue();
final Object reqId = request.getRequest().getId();

traceLambda(
LOG,
"{} parameters: start block number {} count {}",
() -> getName(),
() -> startBlockNumber,
() -> count);

if (startBlockNumber < 1 || count < 1) {
return new JsonRpcErrorResponse(reqId, JsonRpcError.INVALID_PARAMS);
}

if (count > getMaxRequestBlocks()) {
return new JsonRpcErrorResponse(reqId, JsonRpcError.INVALID_RANGE_REQUEST_TOO_LARGE);
}

final Blockchain blockchain = protocolContext.getBlockchain();
final long chainHeadBlockNumber = blockchain.getChainHeadBlockNumber();

// request startBlockNumber is beyond head of chain
if (chainHeadBlockNumber < startBlockNumber) {
// Empty List of payloadBodies
return new JsonRpcSuccessResponse(reqId, new EngineGetPayloadBodiesResultV1());
}

final long upperBound = startBlockNumber + count;

// if we've received request from blocks beyond the head we exclude those from the query
final long endExclusiveBlockNumber =
chainHeadBlockNumber < upperBound ? chainHeadBlockNumber + 1 : upperBound;

EngineGetPayloadBodiesResultV1 engineGetPayloadBodiesResultV1 =
blockResultFactory.payloadBodiesCompleteV1(
LongStream.range(startBlockNumber, endExclusiveBlockNumber)
.mapToObj(
blockNumber ->
blockchain
.getBlockHashByNumber(blockNumber)
.flatMap(blockchain::getBlockBody))
.collect(Collectors.toList()));

return new JsonRpcSuccessResponse(reqId, engineGetPayloadBodiesResultV1);
}

protected int getMaxRequestBlocks() {
return MAX_REQUEST_BLOCKS;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ public enum JsonRpcError {
INVALID_TERMINAL_BLOCK(-32002, "Terminal block doesn't satisfy terminal block conditions"),
INVALID_FORKCHOICE_STATE(-38002, "Invalid forkchoice state"),
INVALID_PAYLOAD_ATTRIBUTES(-38003, "Invalid payload attributes"),

INVALID_RANGE_REQUEST_TOO_LARGE(-38004, "Too large request"),
// Miner failures
COINBASE_NOT_SET(-32010, "Coinbase not set. Unable to start mining without a coinbase"),
NO_HASHES_PER_SECOND(-32011, "No hashes being generated by the current node"),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,19 @@

import org.hyperledger.besu.datatypes.Hash;
import org.hyperledger.besu.datatypes.Wei;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.results.EngineGetPayloadBodiesResultV1.PayloadBody;
import org.hyperledger.besu.ethereum.api.query.BlockWithMetadata;
import org.hyperledger.besu.ethereum.api.query.TransactionWithMetadata;
import org.hyperledger.besu.ethereum.core.Block;
import org.hyperledger.besu.ethereum.core.BlockBody;
import org.hyperledger.besu.ethereum.core.BlockHeader;
import org.hyperledger.besu.ethereum.core.BlockValueCalculator;
import org.hyperledger.besu.ethereum.core.BlockWithReceipts;
import org.hyperledger.besu.ethereum.core.encoding.TransactionEncoder;

import java.util.ArrayList;
import java.util.List;
import java.util.Optional;
import java.util.stream.Collectors;

import com.fasterxml.jackson.databind.JsonNode;
Expand Down Expand Up @@ -115,6 +118,15 @@ public EngineGetPayloadResultV2 payloadTransactionCompleteV2(
Quantity.create(blockValue));
}

public EngineGetPayloadBodiesResultV1 payloadBodiesCompleteV1(
final List<Optional<BlockBody>> blockBodies) {
final List<PayloadBody> payloadBodies =
blockBodies.stream()
.map(maybeBody -> maybeBody.map(PayloadBody::new).orElse(null))
.collect(Collectors.toList());
return new EngineGetPayloadBodiesResultV1(payloadBodies);
}

public BlockResult transactionHash(final BlockWithMetadata<Hash, Hash> blockWithMetadata) {
return transactionHash(blockWithMetadata, false);
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
/*
* Copyright Hyperledger Besu Contributors.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.ethereum.api.jsonrpc.internal.results;

import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.WithdrawalParameter;
import org.hyperledger.besu.ethereum.core.BlockBody;
import org.hyperledger.besu.ethereum.core.encoding.TransactionEncoder;

import java.util.Collections;
import java.util.List;
import java.util.stream.Collectors;

import com.fasterxml.jackson.annotation.JsonGetter;
import com.fasterxml.jackson.annotation.JsonPropertyOrder;
import com.fasterxml.jackson.annotation.JsonValue;
import org.apache.tuweni.bytes.Bytes;

@JsonPropertyOrder({"payloadBodies"})
public class EngineGetPayloadBodiesResultV1 {

private final List<PayloadBody> payloadBodies;

public EngineGetPayloadBodiesResultV1() {
this.payloadBodies = Collections.<PayloadBody>emptyList();
}

public EngineGetPayloadBodiesResultV1(final List<PayloadBody> payloadBody) {
this.payloadBodies = payloadBody;
}

@JsonValue
public List<PayloadBody> getPayloadBodies() {
return payloadBodies;
}

public static class PayloadBody {
private final List<String> transactions;
private final List<WithdrawalParameter> withdrawals;

public PayloadBody(final BlockBody blockBody) {
this.transactions =
blockBody.getTransactions().stream()
.map(TransactionEncoder::encodeOpaqueBytes)
.map(Bytes::toHexString)
.collect(Collectors.toList());
this.withdrawals =
blockBody
.getWithdrawals()
.map(
ws ->
ws.stream()
.map(WithdrawalParameter::fromWithdrawal)
.collect(Collectors.toList()))
.orElse(null);
}

@JsonGetter(value = "transactions")
public List<String> getTransactions() {
return transactions;
}

@JsonGetter(value = "withdrawals")
public List<WithdrawalParameter> getWithdrawals() {
return withdrawals;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineExchangeTransitionConfiguration;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineForkchoiceUpdatedV1;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineForkchoiceUpdatedV2;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineGetPayloadBodiesByHashV1;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineGetPayloadBodiesByRangeV1;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineGetPayloadV1;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineGetPayloadV2;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineNewPayloadV1;
Expand Down Expand Up @@ -114,6 +116,10 @@ protected Map<String, JsonRpcMethod> create() {
engineQosTimer),
new EngineExchangeTransitionConfiguration(
consensusEngineServer, protocolContext, engineQosTimer),
new EngineGetPayloadBodiesByHashV1(
consensusEngineServer, protocolContext, blockResultFactory, engineQosTimer),
new EngineGetPayloadBodiesByRangeV1(
consensusEngineServer, protocolContext, blockResultFactory, engineQosTimer),
new EngineExchangeCapabilities(consensusEngineServer, protocolContext, engineQosTimer));
} else {
return mapOf(
Expand Down
Loading

0 comments on commit 05bf6ab

Please sign in to comment.