move and enable visual tests (#9961)

* move and enable visual tests * change directory for linux visual * diff visual tests and assert immediately * use autoformatting * fix lint errors * reformat after lint fix * remove unused var lint err * fix test image reference paths * archive and publish image tests on failure * add results to archive on failure * fix format issue * fix graph result naming * add new source of truth for image tests * update state city graph reference * feedback cleanup * remove unused import * update state city ref * visual tests run as separate job * add visual test job to azure pipelines * add virtual env to visual tests * add bad ref for testing * fix barrier reference * add bad graph ref for testing * fix graph references * rounding error * update state city ref * unify failure dir, force failure * refactor basedon review * formatting, update test discovery * import order per linting * add docstring to vis utilities * add dev requirements to image tests * update references * Remove unnecessary Azure default parameter * Remove out-of-date references to Binder * Remove executable bit from Python script --------- Co-authored-by: Luciano Bello <bel@zurich.ibm.com> Co-authored-by: Jake Lishman <jake.lishman@ibm.com>
Qiskit · Jul 11, 2023 · fb9d5d8 · fb9d5d8
1 parent e9c86a5
commit fb9d5d8
Show file tree

Hide file tree

Showing 205 changed files with 1,599 additions and 564 deletions.
diff --git a/.azure/test-linux.yml b/.azure/test-linux.yml
@@ -177,6 +177,7 @@ jobs:
             image_tests/bin/python -m pip install -U \
               -c constraints.txt \
               -r requirements.txt \
+              -r requirements-dev.txt \
               -e ".[visualization]"
             sudo apt-get update
             sudo apt-get install -y graphviz pandoc
@@ -185,10 +186,47 @@ jobs:
           env:
             SETUPTOOLS_ENABLE_FEATURES: "legacy-editable"
 
-        - bash: image_tests/bin/python -m unittest discover -v test/ipynb
+        - bash: image_tests/bin/python -m unittest discover -v test/visual
           displayName: 'Run image test'
           env:
             # Needed to suppress a warning in jupyter-core 5.x by eagerly migrating to
             # a new internal interface that will be the default in jupyter-core 6.x.
             # This variable should become redundant on release of jupyter-core 6.
             JUPYTER_PLATFORM_DIRS: 1
+
+        - task: ArchiveFiles@2
+          displayName: Archive visual test failure diffs
+          inputs:
+            rootFolderOrFile: 'test/visual/mpl/visual_test_failures'
+            includeRootFolder: false
+            archiveType: tar
+            archiveFile: '$(Build.ArtifactStagingDirectory)/visual_test_failures.tar.gz'
+            verbose: true
+          condition: failed()
+
+        - task: ArchiveFiles@2
+          displayName: Archive circuit results
+          inputs:
+            rootFolderOrFile: 'test/visual/mpl/circuit/circuit_results'
+            archiveType: tar
+            archiveFile: '$(Build.ArtifactStagingDirectory)/circuit_results.tar.gz'
+            verbose: true
+          condition: failed()
+
+        - task: ArchiveFiles@2
+          displayName: Archive graph results
+          inputs:
+            rootFolderOrFile: 'test/visual/mpl/graph/graph_results'
+            archiveType: tar
+            archiveFile: '$(Build.ArtifactStagingDirectory)/graph_results.tar.gz'
+            verbose: true
+          condition: failed()
+
+        - task: PublishBuildArtifacts@1
+          displayName: 'Publish image test failure diffs'
+          inputs:
+            pathtoPublish: '$(Build.ArtifactStagingDirectory)'
+            artifactName: 'image_test_failure_img_diffs'
+            Parallel: true
+            ParallelCount: 8
+          condition: failed()
diff --git a/qiskit/visualization/state_visualization.py b/qiskit/visualization/state_visualization.py
@@ -515,6 +515,9 @@ def plot_state_city(
     min_dzi = np.min(dzi)
     max_dzi = np.max(dzi)
 
+    # There seems to be a rounding error in which some zero bars are negative
+    dzr = np.clip(dzr, 0, None)
+
     if ax1 is not None:
         fc1 = generate_facecolors(xpos, ypos, zpos, dx, dy, dzr, color[0])
         for idx, cur_zpos in enumerate(zpos):

diff --git a/test/ipynb/mpl/circuit/__init__.py b/test/ipynb/mpl/circuit/__init__.py
diff --git a/test/ipynb/mpl/circuit/references/6095.png b/test/ipynb/mpl/circuit/references/6095.png
diff --git a/test/ipynb/mpl/circuit/references/barrier_label.png b/test/ipynb/mpl/circuit/references/barrier_label.png
diff --git a/test/ipynb/mpl/circuit/references/big_gates.png b/test/ipynb/mpl/circuit/references/big_gates.png
diff --git a/test/ipynb/mpl/circuit/references/bit_conditional_bundle.png b/test/ipynb/mpl/circuit/references/bit_conditional_bundle.png
diff --git a/test/ipynb/mpl/circuit/references/bit_conditional_no_bundle.png b/test/ipynb/mpl/circuit/references/bit_conditional_no_bundle.png
diff --git a/test/ipynb/mpl/circuit/references/bw.png b/test/ipynb/mpl/circuit/references/bw.png
diff --git a/test/ipynb/mpl/circuit/references/calibrations.png b/test/ipynb/mpl/circuit/references/calibrations.png
diff --git a/test/ipynb/mpl/circuit/references/calibrations_with_control_gates.png b/test/ipynb/mpl/circuit/references/calibrations_with_control_gates.png
diff --git a/test/ipynb/mpl/circuit/references/calibrations_with_rzz_and_rxx.png b/test/ipynb/mpl/circuit/references/calibrations_with_rzz_and_rxx.png
diff --git a/test/ipynb/mpl/circuit/references/calibrations_with_swap_and_reset.png b/test/ipynb/mpl/circuit/references/calibrations_with_swap_and_reset.png
diff --git a/test/ipynb/mpl/circuit/references/cnot.png b/test/ipynb/mpl/circuit/references/cnot.png
diff --git a/test/ipynb/mpl/circuit/references/cond_bits_reverse.png b/test/ipynb/mpl/circuit/references/cond_bits_reverse.png
diff --git a/test/ipynb/mpl/circuit/references/creg_initial_false.png b/test/ipynb/mpl/circuit/references/creg_initial_false.png
diff --git a/test/ipynb/mpl/circuit/references/creg_initial_true.png b/test/ipynb/mpl/circuit/references/creg_initial_true.png
diff --git a/test/ipynb/mpl/circuit/references/cswap_rzz.png b/test/ipynb/mpl/circuit/references/cswap_rzz.png
diff --git a/test/ipynb/mpl/circuit/references/ctrl_labels.png b/test/ipynb/mpl/circuit/references/ctrl_labels.png
diff --git a/test/ipynb/mpl/circuit/references/cz.png b/test/ipynb/mpl/circuit/references/cz.png
diff --git a/test/ipynb/mpl/circuit/references/empty_circut.png b/test/ipynb/mpl/circuit/references/empty_circut.png
diff --git a/test/ipynb/mpl/circuit/references/figwidth.png b/test/ipynb/mpl/circuit/references/figwidth.png
diff --git a/test/ipynb/mpl/circuit/references/fold_4.png b/test/ipynb/mpl/circuit/references/fold_4.png
diff --git a/test/ipynb/mpl/circuit/references/fold_minus1.png b/test/ipynb/mpl/circuit/references/fold_minus1.png
diff --git a/test/ipynb/mpl/circuit/references/fold_with_conditions.png b/test/ipynb/mpl/circuit/references/fold_with_conditions.png
diff --git a/test/ipynb/mpl/circuit/references/ghz_to_gate.png b/test/ipynb/mpl/circuit/references/ghz_to_gate.png
diff --git a/test/ipynb/mpl/circuit/references/global_phase.png b/test/ipynb/mpl/circuit/references/global_phase.png
diff --git a/test/ipynb/mpl/circuit/references/idle_wires_barrier.png b/test/ipynb/mpl/circuit/references/idle_wires_barrier.png
diff --git a/test/ipynb/mpl/circuit/references/init_reset.png b/test/ipynb/mpl/circuit/references/init_reset.png
diff --git a/test/ipynb/mpl/circuit/references/instruction_1q_1c.png b/test/ipynb/mpl/circuit/references/instruction_1q_1c.png
diff --git a/test/ipynb/mpl/circuit/references/instruction_3q_3c_circ1.png b/test/ipynb/mpl/circuit/references/instruction_3q_3c_circ1.png
diff --git a/test/ipynb/mpl/circuit/references/instruction_3q_3c_circ2.png b/test/ipynb/mpl/circuit/references/instruction_3q_3c_circ2.png
diff --git a/test/ipynb/mpl/circuit/references/instruction_3q_3c_circ3.png b/test/ipynb/mpl/circuit/references/instruction_3q_3c_circ3.png
diff --git a/test/ipynb/mpl/circuit/references/iqx-dark_color.png b/test/ipynb/mpl/circuit/references/iqx-dark_color.png
diff --git a/test/ipynb/mpl/circuit/references/iqx_color.png b/test/ipynb/mpl/circuit/references/iqx_color.png
diff --git a/test/ipynb/mpl/circuit/references/long_name.png b/test/ipynb/mpl/circuit/references/long_name.png
diff --git a/test/ipynb/mpl/circuit/references/meas_condition.png b/test/ipynb/mpl/circuit/references/meas_condition.png
diff --git a/test/ipynb/mpl/circuit/references/measure_cond_bits_false.png b/test/ipynb/mpl/circuit/references/measure_cond_bits_false.png
diff --git a/test/ipynb/mpl/circuit/references/measure_cond_bits_right.png b/test/ipynb/mpl/circuit/references/measure_cond_bits_right.png
diff --git a/test/ipynb/mpl/circuit/references/measure_cond_bits_true.png b/test/ipynb/mpl/circuit/references/measure_cond_bits_true.png
diff --git a/test/ipynb/mpl/circuit/references/measure_cond_false.png b/test/ipynb/mpl/circuit/references/measure_cond_false.png
diff --git a/test/ipynb/mpl/circuit/references/measure_cond_true.png b/test/ipynb/mpl/circuit/references/measure_cond_true.png
diff --git a/test/ipynb/mpl/circuit/references/multi_underscore_false.png b/test/ipynb/mpl/circuit/references/multi_underscore_false.png
diff --git a/test/ipynb/mpl/circuit/references/multi_underscore_true.png b/test/ipynb/mpl/circuit/references/multi_underscore_true.png
diff --git a/test/ipynb/mpl/circuit/references/no_barriers.png b/test/ipynb/mpl/circuit/references/no_barriers.png
diff --git a/test/ipynb/mpl/circuit/references/no_op_circut.png b/test/ipynb/mpl/circuit/references/no_op_circut.png
diff --git a/test/ipynb/mpl/circuit/references/one_bit_regs.png b/test/ipynb/mpl/circuit/references/one_bit_regs.png
diff --git a/test/ipynb/mpl/circuit/references/partial_layout.png b/test/ipynb/mpl/circuit/references/partial_layout.png
diff --git a/test/ipynb/mpl/circuit/references/pauli_clifford.png b/test/ipynb/mpl/circuit/references/pauli_clifford.png
diff --git a/test/ipynb/mpl/circuit/references/pi_in_param_expr.png b/test/ipynb/mpl/circuit/references/pi_in_param_expr.png
diff --git a/test/ipynb/mpl/circuit/references/plot_barriers_false.png b/test/ipynb/mpl/circuit/references/plot_barriers_false.png
diff --git a/test/ipynb/mpl/circuit/references/plot_barriers_true.png b/test/ipynb/mpl/circuit/references/plot_barriers_true.png
diff --git a/test/ipynb/mpl/circuit/references/plot_partial_barrier.png b/test/ipynb/mpl/circuit/references/plot_partial_barrier.png
diff --git a/test/ipynb/mpl/circuit/references/r_gates.png b/test/ipynb/mpl/circuit/references/r_gates.png
diff --git a/test/ipynb/mpl/circuit/references/reg_conditional.png b/test/ipynb/mpl/circuit/references/reg_conditional.png
diff --git a/test/ipynb/mpl/circuit/references/registerless_one_bit.png b/test/ipynb/mpl/circuit/references/registerless_one_bit.png
diff --git a/test/ipynb/mpl/circuit/references/reverse_bits.png b/test/ipynb/mpl/circuit/references/reverse_bits.png
diff --git a/test/ipynb/mpl/circuit/references/reverse_bits_cond_false.png b/test/ipynb/mpl/circuit/references/reverse_bits_cond_false.png
diff --git a/test/ipynb/mpl/circuit/references/reverse_bits_cond_true.png b/test/ipynb/mpl/circuit/references/reverse_bits_cond_true.png
diff --git a/test/ipynb/mpl/circuit/references/scale_default.png b/test/ipynb/mpl/circuit/references/scale_default.png
diff --git a/test/ipynb/mpl/circuit/references/scale_double.png b/test/ipynb/mpl/circuit/references/scale_double.png
diff --git a/test/ipynb/mpl/circuit/references/scale_half.png b/test/ipynb/mpl/circuit/references/scale_half.png
diff --git a/test/ipynb/mpl/circuit/references/sidetext_condition.png b/test/ipynb/mpl/circuit/references/sidetext_condition.png
diff --git a/test/ipynb/mpl/circuit/references/style_custom_gates.png b/test/ipynb/mpl/circuit/references/style_custom_gates.png
diff --git a/test/ipynb/mpl/circuit/references/subfont.png b/test/ipynb/mpl/circuit/references/subfont.png
diff --git a/test/ipynb/mpl/circuit/references/textbook_color.png b/test/ipynb/mpl/circuit/references/textbook_color.png
diff --git a/test/ipynb/mpl/circuit/references/user_ax.png b/test/ipynb/mpl/circuit/references/user_ax.png
diff --git a/test/ipynb/mpl/circuit/references/user_style.png b/test/ipynb/mpl/circuit/references/user_style.png
diff --git a/test/ipynb/mpl/circuit/references/wide_params.png b/test/ipynb/mpl/circuit/references/wide_params.png
diff --git a/test/ipynb/mpl/circuit/references/wire_order.png b/test/ipynb/mpl/circuit/references/wire_order.png
diff --git a/test/ipynb/mpl/graph/references/16_qubit_gate_map.png b/test/ipynb/mpl/graph/references/16_qubit_gate_map.png
diff --git a/test/ipynb/mpl/graph/references/1_qubit_gate_map.png b/test/ipynb/mpl/graph/references/1_qubit_gate_map.png
diff --git a/test/ipynb/mpl/graph/references/27_qubit_gate_map.png b/test/ipynb/mpl/graph/references/27_qubit_gate_map.png
diff --git a/test/ipynb/mpl/graph/references/5_qubit_gate_map.png b/test/ipynb/mpl/graph/references/5_qubit_gate_map.png
diff --git a/test/ipynb/mpl/graph/references/65_qubit_gate_map.png b/test/ipynb/mpl/graph/references/65_qubit_gate_map.png
diff --git a/test/ipynb/mpl/graph/references/7_qubit_gate_map.png b/test/ipynb/mpl/graph/references/7_qubit_gate_map.png
diff --git a/test/ipynb/mpl/graph/references/bloch_multivector.png b/test/ipynb/mpl/graph/references/bloch_multivector.png
diff --git a/test/ipynb/mpl/graph/references/bloch_multivector_figsize_improvements.png b/test/ipynb/mpl/graph/references/bloch_multivector_figsize_improvements.png
diff --git a/test/ipynb/mpl/graph/references/coupling_map.png b/test/ipynb/mpl/graph/references/coupling_map.png
diff --git a/test/ipynb/mpl/graph/references/figsize.png b/test/ipynb/mpl/graph/references/figsize.png
diff --git a/test/ipynb/mpl/graph/references/font_color.png b/test/ipynb/mpl/graph/references/font_color.png
diff --git a/test/ipynb/mpl/graph/references/hinton.png b/test/ipynb/mpl/graph/references/hinton.png
diff --git a/test/ipynb/mpl/graph/references/histogram.png b/test/ipynb/mpl/graph/references/histogram.png
diff --git a/test/ipynb/mpl/graph/references/histogram_2_sets_with_rest.png b/test/ipynb/mpl/graph/references/histogram_2_sets_with_rest.png
diff --git a/test/ipynb/mpl/graph/references/histogram_color.png b/test/ipynb/mpl/graph/references/histogram_color.png
diff --git a/test/ipynb/mpl/graph/references/histogram_desc_value_sort.png b/test/ipynb/mpl/graph/references/histogram_desc_value_sort.png
diff --git a/test/ipynb/mpl/graph/references/histogram_hamming.png b/test/ipynb/mpl/graph/references/histogram_hamming.png
diff --git a/test/ipynb/mpl/graph/references/histogram_legend.png b/test/ipynb/mpl/graph/references/histogram_legend.png
diff --git a/test/ipynb/mpl/graph/references/histogram_multiple_colors.png b/test/ipynb/mpl/graph/references/histogram_multiple_colors.png
diff --git a/test/ipynb/mpl/graph/references/histogram_title.png b/test/ipynb/mpl/graph/references/histogram_title.png
diff --git a/test/ipynb/mpl/graph/references/histogram_value_sort.png b/test/ipynb/mpl/graph/references/histogram_value_sort.png
diff --git a/test/ipynb/mpl/graph/references/histogram_with_rest.png b/test/ipynb/mpl/graph/references/histogram_with_rest.png
diff --git a/test/ipynb/mpl/graph/references/line_color.png b/test/ipynb/mpl/graph/references/line_color.png
diff --git a/test/ipynb/mpl/graph/references/paulivec.png b/test/ipynb/mpl/graph/references/paulivec.png
diff --git a/test/ipynb/mpl/graph/references/qsphere.png b/test/ipynb/mpl/graph/references/qsphere.png
diff --git a/test/ipynb/mpl/graph/references/qubit_color.png b/test/ipynb/mpl/graph/references/qubit_color.png
diff --git a/test/ipynb/mpl/graph/references/qubit_labels.png b/test/ipynb/mpl/graph/references/qubit_labels.png
diff --git a/test/ipynb/mpl/graph/references/qubit_size.png b/test/ipynb/mpl/graph/references/qubit_size.png
diff --git a/test/ipynb/mpl/graph/references/state_city.png b/test/ipynb/mpl/graph/references/state_city.png