This is a Vivado 2023.2 starter project for the XCKU15P FPGA on the Innova-2 SmartNIC MNV303212A-ADLT that uses the OpenCAPI connector for PCIe and I2C.
The GTY Transceivers connected to the OpenCAPI Connector are in a column that does not contain the Configuration Block so it is impossible for the FPGA to be programmed within the 100ms PCIe power-up time limit.
Motherboard boot must be delayed to allow the FPGA to configure itself before PCIe devices are enumerated. This can be accomplished by toggling the POWER button, then pressing and holding the RESET button for a second before releasing it. Or, connect a capacitor across the reset pins of an ATX motherboard's Front Panel Header.
Currently testing using a Second Revision OpenCAPI-to-PCIe adapter. PCIe x8 using the OpenCAPI connector works but requires a high quality cable and uses a PCIe Lane to Transceiver Channel ordering that Vivado complains about.
Refer to this tutorial for detailed instructions on generating a similar project from scratch.
Related Projects: OpenCAPI-to-PCIe, OpenCAPI_Breakout, PCIe_x8_Breakout.
An offset is used for AXI_Lite to simplify memory-mapping the IIC (I2C) Block.
| Block | Address (Hex) | Size |
|---|---|---|
M_AXI BRAM_CTRL_1 |
0x00000000 | 128K |
M_AXI_LITE BRAM_CTRL_0 |
0x40040000 | 64K |
M_AXI_LITE GPIO_3 |
0x40050000 | 64K |
M_AXI_LITE IIC_0 |
0x40060000 | 64K |
Refer to the innova2_flex_xcku15p_notes project's instructions on Loading a User Image. Binary Memory Configuration Bitstream Files are included in this project's Releases.
wget https://github.com/mwrnd/innova2_xdma_opencapi/releases/download/v0.1/innova2_xdma_opencapi_bitstream.zip
unzip innova2_xdma_opencapi_bitstream.zip
sha256sum *bin
echo 4b95eea589159197ab7c973038cfea28e6a4fe04385a92a70a736d428d8ab882 should be Checksum of innova2_xdma_opencapi_primary.bin
echo f22c0fefabcf8eb9ced863bf11f3b252b336faa0f2778f6cac1588dd74e08daf should be Checksum of innova2_xdma_opencapi_secondary.bin
After programming the bitstream and rebooting, the design should show up as Memory controller: Xilinx Corporation Device 9038 under lspci. It shows up at PCIe Bus Address 01:00 for me but this depends on the PCIe connector you plug your board into.
sudo lspci -tv | grep -i "Mellanox\|0000\|Xilinx\|1d"
Using a 3M 8ES8-1DF21-0.75 cable and a Second Revision OpenCAPI-to-PCIe adapter the PCIe Link Status is usually excellent:
sudo lspci -v -d 10ee: ; sudo lspci -vvv -d 10ee: | grep LnkSta
dmesg | grep -i xdma provides details on how Xilinx's PCIe XDMA driver has loaded.
Using a 3M 8ES8-1DF21-0.75 cable:
PCIe Link Status is usually excellent:
Using an SFPCables.com SFF-8654 to SFF-8654 8i cable:
PCIe Link Status is downgraded:
I am working on a third revision of the OpenCAPI-to-PCIe adapter to improve signal integrity.
The OpenCAPI-to-PCIe adapter routes the OpenCAPI I2C signals to an external connector which has the same pin ordering as a TC74 Temperature Sensor. Note 3.3V is from the PCIe connector.
Complete system:
innova2_xdma_opencapi_iic_tc74_test.c is a simple program to read the temperature and configuration registers of the sensor. Refer to these notes on AXI IIC (I2C) over XDMA for more information.
Compile and run the TC74 test program:
make
sudo ./innova2_xdma_opencapi_iic_tc74_test
After spraying the sensor with some Freeze Spray the temperature drops to below 0C. The data is in 2's-complement format.
There are only two I2C registers in the TC74:
Run the source command from the Vivado 2023.2 Tcl Console:
cd innova2_xdma_opencapi
dir
source innova2_xdma_opencapi.tcl
Wait for Synthesis and Implementation runs to finish. The programming files innova2_xdma_opencapi_primary.bin and innova2_xdma_opencapi_secondary.bin will be generated.
Resources used for the design:
