module built using petalinux

[DarrenChengdu]

Hi jeremy,

I use petalinux v2016.3 to contain the source code of ezdma.c in my created module. Once the linux on board booted, I typed "insmod /lib/modules/'uname -r'/extra/ezdma.ko" or "modprobe ezdma", but the terminal outputs:
"ezdma: allocated chrdev region: Major: 244 , Minor: 0-8
probing ezdma
ezdma: couldn't find dma channel: loop_tx, deferring...
ezdma: loop_tx (TX) available
ezdma: couldn't find dma channel: loop_rx, deferring...
ezdma: loop_rx (RX) available"

I debug the source and found some errors occured in the code segment:
/* Get the named DMA channel */
p_info->chan = dma_request_slave_channel(
&pdev->dev, p_dma_name);

    if ( !p_info->chan )
    {
        printk( KERN_WARNING KBUILD_MODNAME 
                ": couldn't find dma channel: %s, deferring...\n",
                p_info->name);

        outer_rv = -EPROBE_DEFER;
    }

the function "dma_request_slave_channel( &pdev->dev, p_dma_name)" returned an unexpected value assigned to "p_info->chan". I can not find either "loop_tx" or "loop_rx" in dir "/dev/". I wondered what's the problem, and what steps might I forgot to exec? Thank you!

[jeremytrimble]

@DarrenChengdu,

The first things I think of would be:

1. Make sure that the AXI DMA driver is loaded (either compiled into the kernel or added as a module).
2. Make sure your devicetree contains both the nodes for the AXI DMA core, and for ezdma (and that the ezdma node properly refers to the AXI DMA's node).

Notes:
If the AXI DMA driver is compiled as a module and its "compatible" attribute is correct, the OS will usually load the module automatically at boot.

If you're not sure about whether the device tree is correct you could post it here and I could take a look to see if anything obvious jumps out at me. A .dts or .dtb file is fine.

[DarrenChengdu]

@jeremytrimble My hw configuration (block design) completely follows the post at http://www.fpgadeveloper.com/2014/08/using-the-axi-dma-in-vivado.html. I exported the hw file .hdf in vivado to create a petalinux project, and then added the "ezdma" module and "speed_test" app. I replaced petalinux-generated files(ezdma.c and speed_test.c) with yours, and modified the generated device-tree file "pl.dtsi" as follows: / { amba_pl: amba_pl { #address-cells = <1>; #size-cells = <1>; compatible = "simple-bus"; ranges ; axi_dma_0: dma@4040000<dma/ezdma@40400000> { #dma-cells = <1>; clock-names = "s_axi_lite_aclk", "m_axi_sg_aclk", "m_axi_mm2s_aclk", "m_axi_s2mm_aclk"; clocks = <&clkc 15>, <&clkc 15>, <&clkc 15>, <&clkc 15>; compatible = "xlnx,axi-dma-1.00.a"; interrupt-parent = <&intc>; interrupts = <0 29 4 0 30 4>; reg = <0x40400000 0x10000>; xlnx,addrwidth = <0x20>; xlnx,include-sg ; dma-channel@4040000<dma-channel/ezdma@40400000> { compatible = "xlnx,axi-dma-mm2s-channel"; dma-channels = <0x1>; interrupts = <0 29 4>; xlnx,datawidth = <0x20>; xlnx,device-id = <0x0>; }; dma-channel@4040003<dma-channel/ezdma@40400030> { compatible = "xlnx,axi-dma-s2mm-channel"; dma-channels = <0x1>; interrupts = <0 30 4>; xlnx,datawidth = <0x20>; xlnx,device-id = <0x0>; }; }; ezdma0 { compatible = "ezdma"; dmas = <&axi_dma_0 0 &axi_dma_0 1>; dma-names = "loop_tx", "loop_rx"; ezdma,dirs = <2 1>; }; }; }; I can not find any differences from couterpart code segment in the recommended method. About the device tree details please check the system.dtb file in attachment. By the way, petalinux tools used kernel 4.6.0, and another post said api function "dma_request_channel()" was not supported anymore in the xlnx kernel 4.0 (https://forums.xilinx.com/t5/Embedded-Linux/AXI-DMA-request-channel-problem/m-p/679058#M15239), but I am not sure!

[DarrenChengdu]

By not using petalinux, I just succeeded in inserting ezdma module into linux kernel, and now character files "loop_tx" and "loop_rx" could be oberved in dir “/dev”. Then I tried to test speed of ezdma:
bash' arm-xilinx-linux-gnueabi-gcc ezdma_speed_test.c -lrt -o speed_test
bash' ./speed_test
I find "write(tx_fd, tx_buf, PACKET_SIZE)" and "read (rx_fd, rx_buf, PACKET_SIZE)" in "ezdma_speed_test.c " can only pass once and the former is dump in the next loop. I am trying to solve this problem. Any suggestions? ^_^

[jeremytrimble]

@DarrenChengdu,

Glad to hear you got it loaded. As for the issue with the loop only executing once -- do you see any complaints from the AXI DMA driver in dmesg? What does the output of speed_test look like?

[DarrenChengdu]

@jeremytrimble I added several printed info into the source code as follows:
while ( i < NUM_TRIALS )
{
assert( PACKET_SIZE == write(tx_fd, tx_buf, PACKET_SIZE) );
printf("write complete!\n");
assert( PACKET_SIZE == read (rx_fd, rx_buf, PACKET_SIZE) );
printf("read complete!\n");
//{
//int rv = read (rx_fd, rx_buf, PACKET_SIZE);
//printf("read returned %d\n", rv);
//}

    {
        int j;
        for (j = 0; j < PACKET_SIZE; ++j)
        {
            if ( rx_buf[j] != tx_buf[j] )
            {
                printf("ERROR IN DATA\n");
                printf("  @ j=%d: rx_buf[%d]: %u, tx_buf[%d]: %u\n",
                    j, j, rx_buf[j], j, tx_buf[j]);
                return 2;
            }
        }

    }

    tx_buf[i % PACKET_SIZE] += 5;  // modify data each time

    i++;
printf("%d\n", i);
}

The output in terminal is:
write complete!
read complete!
1
write complete!

It seems blocked at "read (rx_fd, rx_buf, PACKET_SIZE)" among the second loop. When I modify PACKET_SIZE=1024, the test runs successfully, and achieves ~8.2 MBytes/s throughput. How to improve the data rate of transferred? Another report shows a data rate of 270MB/s transferred from the PS to the PL and back could be achieved. (https://tobygomersallwordpresscom.wordpress.com/blog/)

[jeremytrimble]

Darren,

I'm not sure why the test is only succeeding when the packet size is reduced -- how deep is your FIFO configured in the PL?

Performance-wise, the throughput is dependent on a number of factors as I'm sure you know. Clock speeds, which AXI PS/PL interfaces you're using (and how deep you have the associated FIFOs configured) are a few factors that play a role.

Another thing to note is that in the case of ezdma_speed_test.c, the "throughput" being measured is based on a "round trip" (e.g. data being sent into the PL by the write() call, and then read back out by the read() call), and the write() and read() happen synchronously (in the same thread). Decoupling the write() and read() (performing them in separate threads or separate processes, if appropriate for your application) could speed things up considerably.

Also, in order to use simple write() and read() calls, ezdma has to "map" the associated memory for each read() or write() call -- this "mapping" process basically makes the memory visible to the DMA core, and sometimes has to be done on a page-by-page basis, which incurs some overhead for every read()/write() call. More optimized DMA schemes are able to reduce this overhead but make the process much more complicated (see the "IIO" subsystem in the kernel).

[hawkroot-ua]

I have the same problem in petalinux:
"ezdma: allocated chrdev region: Major: 244 , Minor: 0-8
probing ezdma
ezdma: couldn't find dma channel: loop_tx, deferring...
ezdma: loop_tx (TX) available
ezdma: couldn't find dma channel: loop_rx, deferring...
ezdma: loop_rx (RX) available"