-
Notifications
You must be signed in to change notification settings - Fork 126
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
QV experiment #32
Merged
chriseclectic
merged 61 commits into
qiskit-community:main
from
dekelmeirom:qv_experiment
Jul 19, 2021
Merged
QV experiment #32
Changes from 58 commits
Commits
Show all changes
61 commits
Select commit
Hold shift + click to select a range
5832967
add qv experiment
dekelmeirom ba98eb1
add init file
dekelmeirom c3395ba
add plot
dekelmeirom 49702cf
fix lint errors and add example
dekelmeirom d709279
fixes from review
dekelmeirom 6fe2da8
add options changes
dekelmeirom 9e26cc6
fix lint error
dekelmeirom e805e23
small docstring fix
dekelmeirom 3a2df23
fix some docs
dekelmeirom f05f12e
add minimum 100 trials
dekelmeirom 6337f9c
fix threshold for qv
dekelmeirom 8ba88f6
remove comment
dekelmeirom 8d58ecc
Minor changes
gadial 79e368a
More renaming
gadial 2bb69eb
Removing add_trails as the effect is already present in re-running th…
gadial 737ec31
Removing the trials property since it can be accessed directly from e…
gadial 72d367f
Fixes to ensure the code runs
gadial 3e51727
Code restructuring to pass ideal probabilities via metadata
gadial 3c90945
Linting
gadial 43b71b7
add qv experiment
dekelmeirom 392c75d
add init file
dekelmeirom 4b77915
add plot
dekelmeirom 7536937
fix lint errors and add example
dekelmeirom f8be521
fixes from review
dekelmeirom d121c01
add options changes
dekelmeirom b4d3856
fix lint error
dekelmeirom 18112e8
small docstring fix
dekelmeirom f8f0cf2
fix some docs
dekelmeirom 2718863
add minimum 100 trials
dekelmeirom 477b866
fix threshold for qv
dekelmeirom 7b3829d
remove comment
dekelmeirom 01754b4
Rough Drag calibration (#117)
eggerdj f59b40e
Add fixed parameter to curve analysis (#153)
nkanazawa1989 c158d7e
Experiment data and analysis result classes (#113)
jyu00 99cedc9
Add full experiment metadata to curve analysis base class (#150)
nkanazawa1989 bed5340
Additional randomized benchmarking tests (#152)
gadial c96556b
Add guess function library (#154)
nkanazawa1989 b501df2
Data processor run options (#121)
eggerdj 4d1a280
Draw fit uncertainty only for T1 and RB (#163)
yaelbh 185d869
T2Ramsey tutorial (#122)
merav-aharoni f862a9f
Adding simulation data at circuit creation; no need to wait for trans…
gadial 8df495c
Merge branch 'Qiskit:main' into qv_experiment
gadial 178f286
Merge branch 'qv_experiment' into qv_experiment
gadial 20cd1c1
Removing simulation data field as it is not needed
gadial 0153465
Further simplification of the ideal probabilities generation
gadial 3d237cc
Merge pull request #1 from gadial/qv_experiment
dekelmeirom f357765
remove duplication of function
dekelmeirom e233403
run black
dekelmeirom adf517e
Merge remote-tracking branch 'upstream/main' into qv_experiment
dekelmeirom 90a774b
remove execute and enable sum trails
dekelmeirom 37241b3
fix typo
dekelmeirom eb0f1f9
add qv tests
dekelmeirom e1459da
fix lint error
dekelmeirom 7c73ad5
fix dir path
dekelmeirom 3b74b93
change test to almost equal
dekelmeirom d580add
fix assert check
dekelmeirom a0e182b
small fix
dekelmeirom f4f721f
fix qv notebook tutorial
dekelmeirom 9ddfbfc
Merge branch 'main' into qv_experiment
ShellyGarion 10b0603
update tutorial
dekelmeirom 006917a
Merge remote-tracking branch 'upstream/main' into qv_experiment
dekelmeirom File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
# This code is part of Qiskit. | ||
# | ||
# (C) Copyright IBM 2021. | ||
# | ||
# This code is licensed under the Apache License, Version 2.0. You may | ||
# obtain a copy of this license in the LICENSE.txt file in the root directory | ||
# of this source tree or at http://www.apache.org/licenses/LICENSE-2.0. | ||
# | ||
# Any modifications or derivative works of this code must retain this | ||
# copyright notice, and modified files need to carry a notice indicating | ||
# that they have been altered from the originals. | ||
|
||
"""Quantum Volume Experiment Classes.""" | ||
from .qv_experiment import QuantumVolume | ||
from .qv_analysis import QuantumVolumeAnalysis |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,266 @@ | ||
# This code is part of Qiskit. | ||
# | ||
# (C) Copyright IBM 2021. | ||
# | ||
# This code is licensed under the Apache License, Version 2.0. You may | ||
# obtain a copy of this license in the LICENSE.txt file in the root directory | ||
# of this source tree or at http://www.apache.org/licenses/LICENSE-2.0. | ||
# | ||
# Any modifications or derivative works of this code must retain this | ||
# copyright notice, and modified files need to carry a notice indicating | ||
# that they have been altered from the originals. | ||
""" | ||
Quantum Volume analysis class. | ||
""" | ||
|
||
import math | ||
import warnings | ||
from typing import Optional | ||
import numpy as np | ||
|
||
from qiskit_experiments.base_analysis import BaseAnalysis | ||
from qiskit_experiments.base_analysis import AnalysisResult | ||
from qiskit_experiments.analysis import plotting | ||
|
||
|
||
class QuantumVolumeAnalysis(BaseAnalysis): | ||
"""Quantum Volume Analysis class.""" | ||
|
||
# pylint: disable = arguments-differ | ||
def _run_analysis( | ||
self, | ||
experiment_data, | ||
plot: bool = True, | ||
ax: Optional["plotting.pyplot.AxesSubplot"] = None, | ||
): | ||
"""Run analysis on circuit data. | ||
Args: | ||
experiment_data (ExperimentData): the experiment data to analyze. | ||
plot: If True generate a plot of fitted data. | ||
ax: Optional, matplotlib axis to add plot to. | ||
Returns: | ||
tuple: A pair ``(analysis_result, figures)`` where | ||
``analysis_results`` may be a single or list of | ||
AnalysisResult objects, and ``figures`` may be | ||
None, a single figure, or a list of figures. | ||
""" | ||
depth = experiment_data.experiment.num_qubits | ||
data = experiment_data.data() | ||
num_trials = len(data) | ||
heavy_output_prob_exp = [] | ||
|
||
for data_trial in data: | ||
heavy_output = self._calc_ideal_heavy_output( | ||
data_trial["metadata"]["ideal_probabilities"], data_trial["metadata"]["depth"] | ||
) | ||
heavy_output_prob_exp.append( | ||
self._calc_exp_heavy_output_probability(data_trial, heavy_output) | ||
) | ||
|
||
analysis_result = AnalysisResult( | ||
self._calc_quantum_volume(heavy_output_prob_exp, depth, num_trials) | ||
) | ||
|
||
if plot and plotting.HAS_MATPLOTLIB: | ||
ax = self._format_plot(ax, analysis_result) | ||
figures = [ax.get_figure()] | ||
else: | ||
figures = None | ||
return [analysis_result], figures | ||
|
||
@staticmethod | ||
def _calc_ideal_heavy_output(probabilities_vector, depth): | ||
""" | ||
Calculate the bit strings of the heavy output for the ideal simulation | ||
Args: | ||
ideal_data (dict): the simulation result of the ideal circuit | ||
Returns: | ||
list: the bit strings of the heavy output | ||
""" | ||
|
||
format_spec = "{0:0%db}" % depth | ||
# Keys are bit strings and values are probabilities of observing those strings | ||
all_output_prob_ideal = { | ||
format_spec.format(b): float(np.real(probabilities_vector[b])) | ||
for b in range(2 ** depth) | ||
} | ||
|
||
median_probabilities = float(np.real(np.median(probabilities_vector))) | ||
heavy_strings = list( | ||
filter( | ||
lambda x: all_output_prob_ideal[x] > median_probabilities, | ||
list(all_output_prob_ideal.keys()), | ||
) | ||
) | ||
return heavy_strings | ||
|
||
@staticmethod | ||
def _calc_exp_heavy_output_probability(data, heavy_outputs): | ||
""" | ||
Calculate the probability of measuring heavy output string in the data | ||
Args: | ||
data (dict): the result of the circuit exectution | ||
heavy_outputs (list): the bit strings of the heavy output from the ideal simulation | ||
Returns: | ||
int: heavy output probability | ||
""" | ||
circ_shots = sum(data["counts"].values()) | ||
|
||
# Calculate the number of heavy output counts in the experiment | ||
heavy_output_counts = sum([data["counts"].get(value, 0) for value in heavy_outputs]) | ||
|
||
# Calculate the experimental heavy output probability | ||
return heavy_output_counts / circ_shots | ||
|
||
@staticmethod | ||
def _calc_z_value(mean, sigma): | ||
"""Calculate z value using mean and sigma. | ||
|
||
Args: | ||
mean (float): mean | ||
sigma (float): standard deviation | ||
|
||
Returns: | ||
float: z_value in standard normal distibution. | ||
""" | ||
|
||
if sigma == 0: | ||
# Assign a small value for sigma if sigma = 0 | ||
sigma = 1e-10 | ||
warnings.warn("Standard deviation sigma should not be zero.") | ||
|
||
z_value = (mean - 2 / 3) / sigma | ||
|
||
return z_value | ||
|
||
@staticmethod | ||
def _calc_confidence_level(z_value): | ||
"""Calculate confidence level using z value. | ||
|
||
Accumulative probability for standard normal distribution | ||
in [-z, +infinity] is 1/2 (1 + erf(z/sqrt(2))), | ||
where z = (X - mu)/sigma = (hmean - 2/3)/sigma | ||
|
||
Args: | ||
z_value (float): z value in in standard normal distibution. | ||
|
||
Returns: | ||
float: confidence level in decimal (not percentage). | ||
""" | ||
|
||
confidence_level = 0.5 * (1 + math.erf(z_value / 2 ** 0.5)) | ||
|
||
return confidence_level | ||
|
||
def _calc_quantum_volume(self, heavy_output_prob_exp, depth, trials): | ||
""" | ||
Calc the quantum volume of the analysed system. | ||
quantum volume is determined by the largest successful depth. | ||
A depth is successful if it has 'mean heavy-output probability' > 2/3 with confidence | ||
level > 0.977 (corresponding to z_value = 2), and at least 100 trials have been ran. | ||
we assume the error (standard deviation) of the heavy output probability is due to a | ||
binomial distribution. standard deviation for binomial distribution is sqrt(np(1-p)), | ||
where n is the number of trials and p is the success probability. | ||
|
||
Returns: | ||
dict: quantum volume calculations - | ||
the quantum volume, | ||
whether the results passed the threshold, | ||
the confidence of the result, | ||
the heavy output probability for each trial, | ||
the mean heavy output probability, | ||
the error of the heavy output probability, | ||
the depth of the circuit, | ||
the number of trials ran | ||
""" | ||
quantum_volume = 1 | ||
success = False | ||
|
||
mean_hop = np.mean(heavy_output_prob_exp) | ||
sigma_hop = (mean_hop * ((1.0 - mean_hop) / trials)) ** 0.5 | ||
z = 2 | ||
threshold = 2 / 3 + z * sigma_hop | ||
z_value = self._calc_z_value(mean_hop, sigma_hop) | ||
confidence_level = self._calc_confidence_level(z_value) | ||
# Must have at least 100 trials | ||
if trials < 100: | ||
warnings.warn("Must use at least 100 trials to consider Quantum Volume as successful.") | ||
if mean_hop > threshold and trials >= 100: | ||
quantum_volume = 2 ** depth | ||
success = True | ||
|
||
result = { | ||
"quantum volume": quantum_volume, | ||
"qv success": success, | ||
"confidence": confidence_level, | ||
"heavy output probability": heavy_output_prob_exp, | ||
"mean hop": mean_hop, | ||
"sigma": sigma_hop, | ||
"depth": depth, | ||
"trials": trials, | ||
} | ||
return result | ||
|
||
@staticmethod | ||
def _format_plot(ax, analysis_result): | ||
""" | ||
Format the QV plot | ||
Args: | ||
ax: matplotlib axis to add plot to. | ||
analysis_result: the results of the experimnt | ||
Returns: | ||
AxesSubPlot: the matplotlib axes containing the plot. | ||
""" | ||
trial_list = np.arange(1, analysis_result["trials"] + 1) # x data | ||
|
||
hop_accumulative = np.cumsum(analysis_result["heavy output probability"]) / trial_list | ||
two_sigma = 2 * (hop_accumulative * (1 - hop_accumulative) / trial_list) ** 0.5 | ||
|
||
# Plot inidivual HOP as scatter | ||
ax = plotting.plot_scatter( | ||
trial_list, | ||
analysis_result["heavy output probability"], | ||
ax=ax, | ||
s=3, | ||
zorder=3, | ||
label="Individual HOP", | ||
) | ||
# Plot accumulative HOP | ||
ax.plot(trial_list, hop_accumulative, color="r", label="Cumulative HOP") | ||
# Plot two-sigma shaded area | ||
ax = plotting.plot_errorbar( | ||
trial_list, | ||
hop_accumulative, | ||
two_sigma, | ||
ax=ax, | ||
fmt="none", | ||
ecolor="lightgray", | ||
elinewidth=20, | ||
capsize=0, | ||
alpha=0.5, | ||
label="2$\\sigma$", | ||
) | ||
# Plot 2/3 success threshold | ||
ax.axhline(2 / 3, color="k", linestyle="dashed", linewidth=1, label="Threshold") | ||
|
||
ax.set_ylim( | ||
max(hop_accumulative[-1] - 4 * two_sigma[-1], 0), | ||
min(hop_accumulative[-1] + 4 * two_sigma[-1], 1), | ||
) | ||
|
||
ax.set_xlabel("Number of Trials", fontsize=14) | ||
ax.set_ylabel("Heavy Output Probability", fontsize=14) | ||
|
||
ax.set_title( | ||
"Quantum Volume experiment for depth " | ||
+ str(analysis_result["depth"]) | ||
+ " - accumulative hop", | ||
fontsize=14, | ||
) | ||
|
||
# Re-arrange legend order | ||
handles, labels = ax.get_legend_handles_labels() | ||
handles = [handles[1], handles[2], handles[0], handles[3]] | ||
labels = [labels[1], labels[2], labels[0], labels[3]] | ||
ax.legend(handles, labels) | ||
return ax |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These kwarg options should now be added to the Analysis class
_default_options
, and then the signature of this function can be the same as the base classdef _run_analysis(self, experiment_data, **options)
.You can then access these values within the function using
self.options.plot
andself.options.ax