Add benchmark circuits #6
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
All circuits from the paper are now implemented and tested. The Bernstein-Vazirani, Hidden shift and QAOA circuits are implemented directly in qibo, while the basis change, supremacy and quantum volume are built using Cirq or qiskit and ported to qibo via OpenQASM. Note that in some cases the number of gates in the circuits is different than what is reported in the HyQuas paper table, however this may be because of difference in parameters (eg. in the QAOA the number of gates depends on the graph which is randomly generated). Here are some numbers for these circuits on qibo machine: CPU - qft - swaps=True
CPU - variational - nlayers=1, varlayer=False
CPU - bernstein-vazirani - nqubits=3
CPU - hidden-shift
CPU - qaoa - nparams=2
CPU - supremacy - depth=2, seed=123
CPU - basis-change - simulation_time=1, seed=123
CPU - quantum-volume - depth=1, seed=123
GPU - qft - swaps=True
GPU - variational - nlayers=1, varlayer=False
GPU - bernstein-vazirani - nqubits=3
GPU - hidden-shift
GPU - qaoa - nparams=2
GPU - supremacy - depth=2, seed=123
GPU - basis-change - simulation_time=1, seed=123
GPU - quantum-volume - depth=1, seed=123
In these cases it may be useful to compare with some other libraries too, for example Cirq or qiskit for CPU or Qulacs for GPU in order to verify whether Qibo is indeed slower as quoted in the HyQuas paper. |
|
@stavros11 thank you so much for all these circuits and benchmarks. It is quite reassuring to see that on CPU both simulators match, and on GPU qibotf is dominated by transfer, as expected. Do you have the json for these runs? If so could you please upload to the data folder? |
Thanks for checking. Indeed the numbers look consistent between qibojit/qibotf but as I wrote above I will try to compare with some external libraries to to verify that the overal qibo performance remains competitive in circuits other than QFT.
I added both the log files in Note that for the Hidden shift and QAOA circuits the number of gates depends on the hidden bitstring / MaxCut graph respectively, so I generated some random bitstrings and graphs which I placed in a new |
|
@stavros11 shall we enable github actions? |
Thank you for the review. Yes, that would be great. Will you do this? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #3 by adding the following circuits from Table 1 of the HyQuas paper:
Note that the depth/number of gates of some of these circuits is affected by parameters other than the number of qubits, such as the graph structure in QAOA or the target shift bitstring in Hidden shift. Care should be taken so that such parameters are kept the same when we are benchmarking different backends.