Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compiling errors #3

Closed
dq2b25 opened this issue May 3, 2017 · 9 comments
Closed

Compiling errors #3

dq2b25 opened this issue May 3, 2017 · 9 comments

Comments

@dq2b25
Copy link

dq2b25 commented May 3, 2017

I follow the instructions to install F-stack step by step, but I got some errors when compiling nginx.

root@ubuntu:~/downloads/f-stack/app/nginx-1.11.10# make -j 8
make -f objs/Makefile
make[1]: Entering directory /root/downloads/f-stack/app/nginx-1.11.10' cc -o objs/nginx \ objs/src/core/nginx.o \ objs/src/core/ngx_log.o \ objs/src/core/ngx_palloc.o \ objs/src/core/ngx_array.o \ objs/src/core/ngx_list.o \ objs/src/core/ngx_hash.o \ objs/src/core/ngx_buf.o \ objs/src/core/ngx_queue.o \ objs/src/core/ngx_output_chain.o \ objs/src/core/ngx_string.o \ objs/src/core/ngx_parse.o \ objs/src/core/ngx_parse_time.o \ objs/src/core/ngx_inet.o \ objs/src/core/ngx_file.o \ objs/src/core/ngx_crc32.o \ objs/src/core/ngx_murmurhash.o \ objs/src/core/ngx_md5.o \ objs/src/core/ngx_sha1.o \ objs/src/core/ngx_rbtree.o \ objs/src/core/ngx_radix_tree.o \ objs/src/core/ngx_slab.o \ objs/src/core/ngx_times.o \ objs/src/core/ngx_shmtx.o \ objs/src/core/ngx_connection.o \ objs/src/core/ngx_cycle.o \ objs/src/core/ngx_spinlock.o \ objs/src/core/ngx_rwlock.o \ objs/src/core/ngx_cpuinfo.o \ objs/src/core/ngx_conf_file.o \ objs/src/core/ngx_module.o \ objs/src/core/ngx_resolver.o \ objs/src/core/ngx_open_file_cache.o \ objs/src/core/ngx_crypt.o \ objs/src/core/ngx_proxy_protocol.o \ objs/src/core/ngx_syslog.o \ objs/src/event/ngx_event.o \ objs/src/event/ngx_event_timer.o \ objs/src/event/ngx_event_posted.o \ objs/src/event/ngx_event_accept.o \ objs/src/event/ngx_event_connect.o \ objs/src/event/ngx_event_pipe.o \ objs/src/os/unix/ngx_time.o \ objs/src/os/unix/ngx_errno.o \ objs/src/os/unix/ngx_alloc.o \ objs/src/os/unix/ngx_files.o \ objs/src/os/unix/ngx_socket.o \ objs/src/os/unix/ngx_recv.o \ objs/src/os/unix/ngx_readv_chain.o \ objs/src/os/unix/ngx_udp_recv.o \ objs/src/os/unix/ngx_send.o \ objs/src/os/unix/ngx_writev_chain.o \ objs/src/os/unix/ngx_udp_send.o \ objs/src/os/unix/ngx_udp_sendmsg_chain.o \ objs/src/os/unix/ngx_channel.o \ objs/src/os/unix/ngx_shmem.o \ objs/src/os/unix/ngx_process.o \ objs/src/os/unix/ngx_daemon.o \ objs/src/os/unix/ngx_setaffinity.o \ objs/src/os/unix/ngx_setproctitle.o \ objs/src/os/unix/ngx_posix_init.o \ objs/src/os/unix/ngx_user.o \ objs/src/os/unix/ngx_dlopen.o \ objs/src/os/unix/ngx_process_cycle.o \ objs/src/os/unix/ngx_linux_init.o \ objs/src/event/modules/ngx_epoll_module.o \ objs/src/os/unix/ngx_linux_sendfile_chain.o \ objs/src/event/modules/ngx_kqueue_module.o \ objs/src/event/modules/ngx_ff_module.o \ objs/src/core/ngx_regex.o \ objs/src/http/ngx_http.o \ objs/src/http/ngx_http_core_module.o \ objs/src/http/ngx_http_special_response.o \ objs/src/http/ngx_http_request.o \ objs/src/http/ngx_http_parse.o \ objs/src/http/modules/ngx_http_log_module.o \ objs/src/http/ngx_http_request_body.o \ objs/src/http/ngx_http_variables.o \ objs/src/http/ngx_http_script.o \ objs/src/http/ngx_http_upstream.o \ objs/src/http/ngx_http_upstream_round_robin.o \ objs/src/http/ngx_http_file_cache.o \ objs/src/http/ngx_http_write_filter_module.o \ objs/src/http/ngx_http_header_filter_module.o \ objs/src/http/modules/ngx_http_chunked_filter_module.o \ objs/src/http/modules/ngx_http_range_filter_module.o \ objs/src/http/modules/ngx_http_gzip_filter_module.o \ objs/src/http/ngx_http_postpone_filter_module.o \ objs/src/http/modules/ngx_http_ssi_filter_module.o \ objs/src/http/modules/ngx_http_charset_filter_module.o \ objs/src/http/modules/ngx_http_userid_filter_module.o \ objs/src/http/modules/ngx_http_headers_filter_module.o \ objs/src/http/ngx_http_copy_filter_module.o \ objs/src/http/modules/ngx_http_not_modified_filter_module.o \ objs/src/http/modules/ngx_http_static_module.o \ objs/src/http/modules/ngx_http_autoindex_module.o \ objs/src/http/modules/ngx_http_index_module.o \ objs/src/http/modules/ngx_http_auth_basic_module.o \ objs/src/http/modules/ngx_http_access_module.o \ objs/src/http/modules/ngx_http_limit_conn_module.o \ objs/src/http/modules/ngx_http_limit_req_module.o \ objs/src/http/modules/ngx_http_geo_module.o \ objs/src/http/modules/ngx_http_map_module.o \ objs/src/http/modules/ngx_http_split_clients_module.o \ objs/src/http/modules/ngx_http_referer_module.o \ objs/src/http/modules/ngx_http_rewrite_module.o \ objs/src/http/modules/ngx_http_proxy_module.o \ objs/src/http/modules/ngx_http_fastcgi_module.o \ objs/src/http/modules/ngx_http_uwsgi_module.o \ objs/src/http/modules/ngx_http_scgi_module.o \ objs/src/http/modules/ngx_http_memcached_module.o \ objs/src/http/modules/ngx_http_empty_gif_module.o \ objs/src/http/modules/ngx_http_browser_module.o \ objs/src/http/modules/ngx_http_upstream_hash_module.o \ objs/src/http/modules/ngx_http_upstream_ip_hash_module.o \ objs/src/http/modules/ngx_http_upstream_least_conn_module.o \ objs/src/http/modules/ngx_http_upstream_keepalive_module.o \ objs/src/http/modules/ngx_http_upstream_zone_module.o \ objs/ngx_modules.o \ -ldl -lpthread -lcrypt -lpcre -lz \ -Wl,-E objs/src/os/unix/ngx_process_cycle.o: In function ngx_single_process_cycle':
/root/downloads/f-stack/app/nginx-1.11.10/src/os/unix/ngx_process_cycle.c:350: undefined reference to ff_run' objs/src/event/modules/ngx_ff_module.o: In function ff_mod_init':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:88: undefined reference to ff_init' objs/src/event/modules/ngx_ff_module.o: In function socket':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:102: undefined reference to ff_socket' objs/src/event/modules/ngx_ff_module.o: In function accept':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:203: undefined reference to ff_accept' objs/src/event/modules/ngx_ff_module.o: In function select':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:278: undefined reference to ff_select' objs/src/event/modules/ngx_ff_module.o: In function recv':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:158: undefined reference to ff_recv' objs/src/event/modules/ngx_ff_module.o: In function read':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:169: undefined reference to ff_read' objs/src/event/modules/ngx_ff_module.o: In function bind':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:114: undefined reference to ff_bind' objs/src/event/modules/ngx_ff_module.o: In function connect':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:125: undefined reference to ff_connect' objs/src/event/modules/ngx_ff_module.o: In function send':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:136: undefined reference to ff_send' objs/src/event/modules/ngx_ff_module.o: In function write':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:147: undefined reference to ff_write' objs/src/event/modules/ngx_ff_module.o: In function listen':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:180: undefined reference to ff_listen' objs/src/event/modules/ngx_ff_module.o: In function setsockopt':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:192: undefined reference to ff_setsockopt' objs/src/event/modules/ngx_ff_module.o: In function accept4':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:219: undefined reference to ff_accept' objs/src/event/modules/ngx_ff_module.o: In function close':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:230: undefined reference to ff_close' objs/src/event/modules/ngx_ff_module.o: In function writev':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:241: undefined reference to ff_writev' objs/src/event/modules/ngx_ff_module.o: In function readv':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:252: undefined reference to ff_readv' objs/src/event/modules/ngx_ff_module.o: In function ioctl':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:263: undefined reference to ff_ioctl' objs/src/event/modules/ngx_ff_module.o: In function kqueue':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:287: undefined reference to ff_kqueue' objs/src/event/modules/ngx_ff_module.o: In function kevent':
/root/downloads/f-stack/app/nginx-1.11.10/src/event/modules/ngx_ff_module.c:294: undefined reference to ff_kevent' collect2: error: ld returned 1 exit status make[1]: *** [objs/nginx] Error 1 make[1]: Leaving directory /root/downloads/f-stack/app/nginx-1.11.10'
make: *** [build] Error 2

@whl739
Copy link
Member

whl739 commented May 3, 2017

Did you follow these steps:
export FF_PATH=/data/f-stack
export FF_DPDK=/data/f-stack/dpdk/x86_64-native-linuxapp-gcc
./configure --prefix=/usr/local/nginx_fstack --with-ff_module

It seemed that ngx didn't link libfstack.a.

@dq2b25
Copy link
Author

dq2b25 commented May 3, 2017

I did the steps you mentioned above, but still got the errors.
These undefined functions are declared in /data/f-stack/lib/ff_api.h , but the compiler cannot find their definitions.

@whl739
Copy link
Member

whl739 commented May 3, 2017

Here is the right prints :
................................
objs/src/http/modules/ngx_http_upstream_zone_module.o
objs/ngx_modules.o
-ldl -lpthread -lcrypt -lpcre -lz -L/data/whl/f-stack/lib -L/data/whl/f-stack/dpdk/x86_64-native-linuxapp-gcc/lib -Wl,--whole-archive,-lfstack,--no-whole-archive -g -Wl,--no-as-needed -fvisibility=default -pthread -lm -lrt -Wl,--whole-archive -lrte_pmd_vmxnet3_uio -lrte_pmd_i40e -lrte_pmd_ixgbe -lrte_pmd_e1000 -lrte_pmd_ring -Wl,--whole-archive -lrte_hash -lrte_kvargs -Wl,-lrte_mbuf -lethdev -lrte_eal -Wl,-lrte_mempool -lrte_ring -lrte_cmdline -lrte_cfgfile -lrte_kni -lrte_timer -Wl,-lrte_pmd_virtio -Wl,--no-whole-archive -lrt -lm -ldl -lm -lcrypto \

Obviously it didn't link the fstack libs.
What does ./configure --with-ff_module output?

@dq2b25
Copy link
Author

dq2b25 commented May 3, 2017

Thanks, I paste the output as follows:

root@ubuntu:/data/f-stack/app/nginx-1.11.10# ./configure --prefix=/usr/local/nginx_fstack --with-ff_module
checking for OS

  • Linux 4.2.0-27-generic x86_64
    checking for C compiler ... found
  • using GNU C compiler
  • gcc version: 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
    checking for gcc -pipe switch ... found
    checking for -Wl,-E switch ... found
    checking for gcc builtin atomic operations ... found
    checking for C99 variadic macros ... found
    checking for gcc variadic macros ... found
    checking for gcc builtin 64 bit byteswap ... found
    checking for unistd.h ... found
    checking for inttypes.h ... found
    checking for limits.h ... found
    checking for sys/filio.h ... not found
    checking for sys/param.h ... found
    checking for sys/mount.h ... found
    checking for sys/statvfs.h ... found
    checking for crypt.h ... found
    checking for Linux specific features
    checking for epoll ... found
    checking for EPOLLRDHUP ... found
    checking for EPOLLEXCLUSIVE ... not found
    checking for O_PATH ... found
    checking for sendfile() ... found
    checking for sendfile64() ... found
    checking for sys/prctl.h ... found
    checking for prctl(PR_SET_DUMPABLE) ... found
    checking for sched_setaffinity() ... found
    checking for crypt_r() ... found
    checking for sys/vfs.h ... found
    checking for nobody group ... not found
    checking for nogroup group ... found
    checking for poll() ... found
    checking for /dev/poll ... not found
    checking for kqueue ... not found
    checking for crypt() ... not found
    checking for crypt() in libcrypt ... found
    checking for F_READAHEAD ... not found
    checking for posix_fadvise() ... found
    checking for O_DIRECT ... found
    checking for F_NOCACHE ... not found
    checking for directio() ... not found
    checking for statfs() ... found
    checking for statvfs() ... found
    checking for dlopen() ... not found
    checking for dlopen() in libdl ... found
    checking for sched_yield() ... found
    checking for SO_SETFIB ... not found
    checking for SO_REUSEPORT ... found
    checking for SO_ACCEPTFILTER ... not found
    checking for SO_BINDANY ... not found
    checking for IP_BIND_ADDRESS_NO_PORT ... not found
    checking for IP_TRANSPARENT ... found
    checking for IP_BINDANY ... not found
    checking for IP_RECVDSTADDR ... not found
    checking for IP_PKTINFO ... found
    checking for IPV6_RECVPKTINFO ... found
    checking for TCP_DEFER_ACCEPT ... found
    checking for TCP_KEEPIDLE ... found
    checking for TCP_FASTOPEN ... found
    checking for TCP_INFO ... found
    checking for eventfd() ... found
    checking for int size ... 4 bytes
    checking for long size ... 8 bytes
    checking for long long size ... 8 bytes
    checking for void * size ... 8 bytes
    checking for uint32_t ... found
    checking for uint64_t ... found
    checking for sig_atomic_t ... found
    checking for sig_atomic_t size ... 4 bytes
    checking for socklen_t ... found
    checking for in_addr_t ... found
    checking for in_port_t ... found
    checking for rlim_t ... found
    checking for uintptr_t ... uintptr_t found
    checking for system byte ordering ... little endian
    checking for size_t size ... 8 bytes
    checking for off_t size ... 8 bytes
    checking for time_t size ... 8 bytes
    checking for AF_INET6 ... found
    checking for setproctitle() ... not found
    checking for pread() ... found
    checking for pwrite() ... found
    checking for pwritev() ... found
    checking for sys_nerr ... found
    checking for localtime_r() ... found
    checking for posix_memalign() ... found
    checking for memalign() ... found
    checking for mmap(MAP_ANON|MAP_SHARED) ... found
    checking for mmap("/dev/zero", MAP_SHARED) ... found
    checking for System V shared memory ... found
    checking for POSIX semaphores ... not found
    checking for POSIX semaphores in libpthread ... found
    checking for struct msghdr.msg_control ... found
    checking for ioctl(FIONBIO) ... found
    checking for struct tm.tm_gmtoff ... found
    checking for struct dirent.d_namlen ... not found
    checking for struct dirent.d_type ... found
    checking for sysconf(_SC_NPROCESSORS_ONLN) ... found
    checking for openat(), fstatat() ... found
    checking for getaddrinfo() ... found
    checking for PCRE library ... found
    checking for PCRE JIT support ... found
    checking for zlib library ... found
    creating objs/Makefile
    ./configure: 31: auto/make: CORE_LIBS+= -L/data/f-stack/lib -L/data/f-stack/dpdk/x86_64-native-linuxapp-gcc/lib -Wl,--whole-archive,-lfstack,--no-whole-archive: not found
    ./configure: 32: auto/make: CORE_LIBS+= -g -Wl,--no-as-needed -fvisibility=default -pthread -lm -lrt: not found
    ./configure: 33: auto/make: CORE_LIBS+= -Wl,--whole-archive -lrte_pmd_vmxnet3_uio -lrte_pmd_i40e -lrte_pmd_ixgbe -lrte_pmd_e1000 -lrte_pmd_ring: not found
    ./configure: 34: auto/make: CORE_LIBS+= -Wl,--whole-archive -lrte_hash -lrte_kvargs -Wl,-lrte_mbuf -lethdev -lrte_eal -Wl,-lrte_mempool: not found
    ./configure: 35: auto/make: CORE_LIBS+= -lrte_ring -lrte_cmdline -lrte_cfgfile -lrte_kni -lrte_timer -Wl,-lrte_pmd_virtio: not found
    ./configure: 36: auto/make: CORE_LIBS+= -Wl,--no-whole-archive -lrt -lm -ldl -lm -lcrypto: not found

Configuration summary

  • using system PCRE library
  • OpenSSL library is not used
  • using system zlib library

nginx path prefix: "/usr/local/nginx_fstack"
nginx binary file: "/usr/local/nginx_fstack/sbin/nginx"
nginx modules path: "/usr/local/nginx_fstack/modules"
nginx configuration prefix: "/usr/local/nginx_fstack/conf"
nginx configuration file: "/usr/local/nginx_fstack/conf/nginx.conf"
nginx pid file: "/usr/local/nginx_fstack/logs/nginx.pid"
nginx error log file: "/usr/local/nginx_fstack/logs/error.log"
nginx http access log file: "/usr/local/nginx_fstack/logs/access.log"
nginx http client request body temporary files: "client_body_temp"
nginx http proxy temporary files: "proxy_temp"
nginx http fastcgi temporary files: "fastcgi_temp"
nginx http uwsgi temporary files: "uwsgi_temp"
nginx http scgi temporary files: "scgi_temp"

@whl739
Copy link
Member

whl739 commented May 4, 2017

./configure: 31: auto/make: CORE_LIBS+= -L/data/f-stack/lib -L/data/f-stack/dpdk/x86_64-native-linuxapp-gcc/lib -Wl,--whole-archive,-lfstack,--no-whole-archive: not found

It's really strange that line 31 of auto/make CORE_LIBS+=" -L$FF_PATH/lib -L$FF_DPDK/lib -Wl,--whole-archive,-lfstack,--no-whole-archive" was treated as a command.

@yulintao
Copy link

yulintao commented May 8, 2017

auto/make CORE_LIBS="$CORE_LIBS -L$FF_PATH/lib -L$FF_DPDK/lib -Wl,--whole-archive,-lfstack,--no-whole-archive"
can be help!

@frameyl
Copy link

frameyl commented May 8, 2017

I have the same issue. The root cause is my root shell linked to 'sh' instead of 'bash'. It seems the ubuntu has the issue but Fedora not. You can run the command as:
bash ./configure --prefix=/usr/local/nginx_fstack --with-ff_module

@frameyl
Copy link

frameyl commented May 8, 2017

I actually has another issue on Ubuntu during compiling. The default awk of ubuntu is mawk instead of gawk, which also causing compiling issue. It can be wrote in help or FQA docs.

@whl739
Copy link
Member

whl739 commented May 8, 2017

@frameyl
Thank you for your feedback and comments! I will test on Ubuntu and add some docs.

@whl739 whl739 closed this as completed in 82aa761 May 8, 2017
tanjianfeng added a commit to tanjianfeng/f-stack that referenced this issue Dec 7, 2020
Two kinds of mbuf are used in f-stack: freebsd mbuf and dpdk mbuf.

freebsd mbufs are metadata used in freebsd stack, and their data
pointers (m_data) point to dpdk mbuf's data (buf_addr). And they have
their own chain, like this:

  bsd_mbuf1 -> bsd_mbuf2 -> bsd_mbuf3
      \            \            \
    dpdk_mbuf1 -> dpdk_mbuf2 -> dpdk_mbuf3

Considering the map relationship,

- m_freem() is corresponding to rte_pktmbuf_free(), is to free the whole
  chain of mbufs.
- m_free() is corresponding to rte_pktmbuf_free_seg(), is to free the
  specified mbuf segment.

The current implementation in f-stack uses rte_pktmbuf_free() for
m_free(). This leads to mbufs, which are still in use, be freed
unexpectedly. For example, if the bsd_mbuf1 is trimed into zero length,
bsd will invoke m_free() to free the specified segment, however, the
whole mbuf chain is freed by calling rte_pktmbuf_free().

  #0 rte_pktmbuf_free (m=0x22006fb480)
  F-Stack#1 in ff_dpdk_pktmbuf_free (m=0x22006fb480)
  F-Stack#2 in ff_mbuf_ext_free (m=0x7ffff7f82800, arg1=0x22006fb480, arg2=0x0)
  F-Stack#3 in mb_free_ext (m=0x7ffff7f82800)
  F-Stack#4 in m_free (m=0x7ffff7f82800)
  F-Stack#5 in sbcompress (sb=, m=0x7ffff7f82800, n=)
  F-Stack#6 in sbappendstream_locked (sb=, m=0x7ffff7f82800, flags=0)

The fix is straightforward. Use the correct API for segment free.

Reported-by: Yong-Hao Zou <yonghaoz1994@gmail.com>
Signed-off-by: Jianfeng Tan <henry.tjf@antgroup.com>
tanjianfeng added a commit to tanjianfeng/f-stack that referenced this issue Dec 7, 2020
Two kinds of mbuf are used in f-stack: freebsd mbuf and dpdk mbuf.

freebsd mbufs are metadata used in freebsd stack, and their data
pointers (m_data) point to dpdk mbuf's data (buf_addr). And they have
their own chain, like this:

  bsd_mbuf1 -> bsd_mbuf2 -> bsd_mbuf3
      \            \            \
    dpdk_mbuf1 -> dpdk_mbuf2 -> dpdk_mbuf3

Considering the map relationship,

- m_freem() is corresponding to rte_pktmbuf_free(), is to free the whole
  chain of mbufs.
- m_free() is corresponding to rte_pktmbuf_free_seg(), is to free the
  specified mbuf segment.

The current implementation in f-stack uses rte_pktmbuf_free() for
m_free(). This leads to mbufs, which are still in use, be freed
unexpectedly. For example, if the bsd_mbuf1 is trimed into zero length,
bsd will invoke m_free() to free the specified segment, however, the
whole mbuf chain is freed by calling rte_pktmbuf_free().

  #0 rte_pktmbuf_free (m=0x22006fb480)
  F-Stack#1 in ff_dpdk_pktmbuf_free (m=0x22006fb480)
  F-Stack#2 in ff_mbuf_ext_free (m=0x7ffff7f82800, arg1=0x22006fb480, arg2=0x0)
  F-Stack#3 in mb_free_ext (m=0x7ffff7f82800)
  F-Stack#4 in m_free (m=0x7ffff7f82800)
  F-Stack#5 in sbcompress (sb=, m=0x7ffff7f82800, n=)
  F-Stack#6 in sbappendstream_locked (sb=, m=0x7ffff7f82800, flags=0)

The fix is straightforward. Use the correct API for segment free.

Reported-by: Yong-Hao Zou <yonghaoz1994@gmail.com>
Signed-off-by: Jianfeng Tan <henry.tjf@antgroup.com>
tanjianfeng added a commit to tanjianfeng/f-stack that referenced this issue Dec 21, 2020
Two kinds of mbuf are used in f-stack: freebsd mbuf and dpdk mbuf.

freebsd mbufs are metadata used in freebsd stack, and their data
pointers (m_data) point to dpdk mbuf's data (buf_addr). And they have
their own chain, like this:

  bsd_mbuf1 -> bsd_mbuf2 -> bsd_mbuf3
      \            \            \
    dpdk_mbuf1 -> dpdk_mbuf2 -> dpdk_mbuf3

Considering the map relationship,

- m_freem() is corresponding to rte_pktmbuf_free(), is to free the whole
  chain of mbufs.
- m_free() is corresponding to rte_pktmbuf_free_seg(), is to free the
  specified mbuf segment.

The current implementation in f-stack uses rte_pktmbuf_free() for
m_free(). This leads to mbufs, which are still in use, be freed
unexpectedly. For example, if the bsd_mbuf1 is trimed into zero length,
bsd will invoke m_free() to free the specified segment, however, the
whole mbuf chain is freed by calling rte_pktmbuf_free().

  #0 rte_pktmbuf_free (m=0x22006fb480)
  F-Stack#1 in ff_dpdk_pktmbuf_free (m=0x22006fb480)
  F-Stack#2 in ff_mbuf_ext_free (m=0x7ffff7f82800, arg1=0x22006fb480, arg2=0x0)
  F-Stack#3 in mb_free_ext (m=0x7ffff7f82800)
  F-Stack#4 in m_free (m=0x7ffff7f82800)
  F-Stack#5 in sbcompress (sb=, m=0x7ffff7f82800, n=)
  F-Stack#6 in sbappendstream_locked (sb=, m=0x7ffff7f82800, flags=0)

The fix is straightforward. Use the correct API for segment free.

Reported-by: Yong-Hao Zou <yonghaoz1994@gmail.com>
Signed-off-by: Jianfeng Tan <henry.tjf@antgroup.com>
jfb8856606 pushed a commit that referenced this issue Mar 6, 2021
Two kinds of mbuf are used in f-stack: freebsd mbuf and dpdk mbuf.

freebsd mbufs are metadata used in freebsd stack, and their data
pointers (m_data) point to dpdk mbuf's data (buf_addr). And they have
their own chain, like this:

  bsd_mbuf1 -> bsd_mbuf2 -> bsd_mbuf3
      \            \            \
    dpdk_mbuf1 -> dpdk_mbuf2 -> dpdk_mbuf3

Considering the map relationship,

- m_freem() is corresponding to rte_pktmbuf_free(), is to free the whole
  chain of mbufs.
- m_free() is corresponding to rte_pktmbuf_free_seg(), is to free the
  specified mbuf segment.

The current implementation in f-stack uses rte_pktmbuf_free() for
m_free(). This leads to mbufs, which are still in use, be freed
unexpectedly. For example, if the bsd_mbuf1 is trimed into zero length,
bsd will invoke m_free() to free the specified segment, however, the
whole mbuf chain is freed by calling rte_pktmbuf_free().

  #0 rte_pktmbuf_free (m=0x22006fb480)
  #1 in ff_dpdk_pktmbuf_free (m=0x22006fb480)
  #2 in ff_mbuf_ext_free (m=0x7ffff7f82800, arg1=0x22006fb480, arg2=0x0)
  #3 in mb_free_ext (m=0x7ffff7f82800)
  #4 in m_free (m=0x7ffff7f82800)
  #5 in sbcompress (sb=, m=0x7ffff7f82800, n=)
  #6 in sbappendstream_locked (sb=, m=0x7ffff7f82800, flags=0)

The fix is straightforward. Use the correct API for segment free.

Reported-by: Yong-Hao Zou <yonghaoz1994@gmail.com>
Signed-off-by: Jianfeng Tan <henry.tjf@antgroup.com>
jfb8856606 pushed a commit that referenced this issue Mar 6, 2021
Two kinds of mbuf are used in f-stack: freebsd mbuf and dpdk mbuf.

freebsd mbufs are metadata used in freebsd stack, and their data
pointers (m_data) point to dpdk mbuf's data (buf_addr). And they have
their own chain, like this:

  bsd_mbuf1 -> bsd_mbuf2 -> bsd_mbuf3
      \            \            \
    dpdk_mbuf1 -> dpdk_mbuf2 -> dpdk_mbuf3

Considering the map relationship,

- m_freem() is corresponding to rte_pktmbuf_free(), is to free the whole
  chain of mbufs.
- m_free() is corresponding to rte_pktmbuf_free_seg(), is to free the
  specified mbuf segment.

The current implementation in f-stack uses rte_pktmbuf_free() for
m_free(). This leads to mbufs, which are still in use, be freed
unexpectedly. For example, if the bsd_mbuf1 is trimed into zero length,
bsd will invoke m_free() to free the specified segment, however, the
whole mbuf chain is freed by calling rte_pktmbuf_free().

  #0 rte_pktmbuf_free (m=0x22006fb480)
  #1 in ff_dpdk_pktmbuf_free (m=0x22006fb480)
  #2 in ff_mbuf_ext_free (m=0x7ffff7f82800, arg1=0x22006fb480, arg2=0x0)
  #3 in mb_free_ext (m=0x7ffff7f82800)
  #4 in m_free (m=0x7ffff7f82800)
  #5 in sbcompress (sb=, m=0x7ffff7f82800, n=)
  #6 in sbappendstream_locked (sb=, m=0x7ffff7f82800, flags=0)

The fix is straightforward. Use the correct API for segment free.

Reported-by: Yong-Hao Zou <yonghaoz1994@gmail.com>
Signed-off-by: Jianfeng Tan <henry.tjf@antgroup.com>
jfb8856606 pushed a commit that referenced this issue Mar 6, 2021
Two kinds of mbuf are used in f-stack: freebsd mbuf and dpdk mbuf.

freebsd mbufs are metadata used in freebsd stack, and their data
pointers (m_data) point to dpdk mbuf's data (buf_addr). And they have
their own chain, like this:

  bsd_mbuf1 -> bsd_mbuf2 -> bsd_mbuf3
      \            \            \
    dpdk_mbuf1 -> dpdk_mbuf2 -> dpdk_mbuf3

Considering the map relationship,

- m_freem() is corresponding to rte_pktmbuf_free(), is to free the whole
  chain of mbufs.
- m_free() is corresponding to rte_pktmbuf_free_seg(), is to free the
  specified mbuf segment.

The current implementation in f-stack uses rte_pktmbuf_free() for
m_free(). This leads to mbufs, which are still in use, be freed
unexpectedly. For example, if the bsd_mbuf1 is trimed into zero length,
bsd will invoke m_free() to free the specified segment, however, the
whole mbuf chain is freed by calling rte_pktmbuf_free().

  #0 rte_pktmbuf_free (m=0x22006fb480)
  #1 in ff_dpdk_pktmbuf_free (m=0x22006fb480)
  #2 in ff_mbuf_ext_free (m=0x7ffff7f82800, arg1=0x22006fb480, arg2=0x0)
  #3 in mb_free_ext (m=0x7ffff7f82800)
  #4 in m_free (m=0x7ffff7f82800)
  #5 in sbcompress (sb=, m=0x7ffff7f82800, n=)
  #6 in sbappendstream_locked (sb=, m=0x7ffff7f82800, flags=0)

The fix is straightforward. Use the correct API for segment free.

Reported-by: Yong-Hao Zou <yonghaoz1994@gmail.com>
Signed-off-by: Jianfeng Tan <henry.tjf@antgroup.com>
jfb8856606 pushed a commit that referenced this issue Jul 5, 2022
Two kinds of mbuf are used in f-stack: freebsd mbuf and dpdk mbuf.

freebsd mbufs are metadata used in freebsd stack, and their data
pointers (m_data) point to dpdk mbuf's data (buf_addr). And they have
their own chain, like this:

  bsd_mbuf1 -> bsd_mbuf2 -> bsd_mbuf3
      \            \            \
    dpdk_mbuf1 -> dpdk_mbuf2 -> dpdk_mbuf3

Considering the map relationship,

- m_freem() is corresponding to rte_pktmbuf_free(), is to free the whole
  chain of mbufs.
- m_free() is corresponding to rte_pktmbuf_free_seg(), is to free the
  specified mbuf segment.

The current implementation in f-stack uses rte_pktmbuf_free() for
m_free(). This leads to mbufs, which are still in use, be freed
unexpectedly. For example, if the bsd_mbuf1 is trimed into zero length,
bsd will invoke m_free() to free the specified segment, however, the
whole mbuf chain is freed by calling rte_pktmbuf_free().

  #0 rte_pktmbuf_free (m=0x22006fb480)
  #1 in ff_dpdk_pktmbuf_free (m=0x22006fb480)
  #2 in ff_mbuf_ext_free (m=0x7ffff7f82800, arg1=0x22006fb480, arg2=0x0)
  #3 in mb_free_ext (m=0x7ffff7f82800)
  #4 in m_free (m=0x7ffff7f82800)
  #5 in sbcompress (sb=, m=0x7ffff7f82800, n=)
  #6 in sbappendstream_locked (sb=, m=0x7ffff7f82800, flags=0)

The fix is straightforward. Use the correct API for segment free.

Reported-by: Yong-Hao Zou <yonghaoz1994@gmail.com>
Signed-off-by: Jianfeng Tan <henry.tjf@antgroup.com>
jfb8856606 pushed a commit that referenced this issue Sep 2, 2022
Two kinds of mbuf are used in f-stack: freebsd mbuf and dpdk mbuf.

freebsd mbufs are metadata used in freebsd stack, and their data
pointers (m_data) point to dpdk mbuf's data (buf_addr). And they have
their own chain, like this:

  bsd_mbuf1 -> bsd_mbuf2 -> bsd_mbuf3
      \            \            \
    dpdk_mbuf1 -> dpdk_mbuf2 -> dpdk_mbuf3

Considering the map relationship,

- m_freem() is corresponding to rte_pktmbuf_free(), is to free the whole
  chain of mbufs.
- m_free() is corresponding to rte_pktmbuf_free_seg(), is to free the
  specified mbuf segment.

The current implementation in f-stack uses rte_pktmbuf_free() for
m_free(). This leads to mbufs, which are still in use, be freed
unexpectedly. For example, if the bsd_mbuf1 is trimed into zero length,
bsd will invoke m_free() to free the specified segment, however, the
whole mbuf chain is freed by calling rte_pktmbuf_free().

  #0 rte_pktmbuf_free (m=0x22006fb480)
  #1 in ff_dpdk_pktmbuf_free (m=0x22006fb480)
  #2 in ff_mbuf_ext_free (m=0x7ffff7f82800, arg1=0x22006fb480, arg2=0x0)
  #3 in mb_free_ext (m=0x7ffff7f82800)
  #4 in m_free (m=0x7ffff7f82800)
  #5 in sbcompress (sb=, m=0x7ffff7f82800, n=)
  #6 in sbappendstream_locked (sb=, m=0x7ffff7f82800, flags=0)

The fix is straightforward. Use the correct API for segment free.

Reported-by: Yong-Hao Zou <yonghaoz1994@gmail.com>
Signed-off-by: Jianfeng Tan <henry.tjf@antgroup.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants