[bug report] angsd sites breaks due to compiler flags

[isinaltinkaya]

Bug report

Current behavior

The same angsd source code built in two environments can change the output as following:

Env1:

	-> Total number of sites analyzed: 10
	-> Number of sites retained after filtering: 10 

Env2 (conda activated):

	-> Total number of sites analyzed: 10
	-> Number of sites retained after filtering: 0

ANGSD version

	-> angsd version: 0.937-83-g09ef3d1-dirty (htslib: 1.14-9-ge769401)

This issue is not version specific, and is the same when tested 0.939-10-g21ed01c (htslib: 1.14-9-ge769401).

Details

Conda environment may set some environment variables related to compiler flags such as CXXFLAGS and CFLAGS under the hood, and this can even affect the value of variables after deactivating the environment (conda#10195).

# Env 1
$ echo ${CPPFLAGS}
  # not set

# Env 2
$ echo ${CPPFLAGS}
-DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/pfs488/lundbeck/scratch/pfs488/miniconda3/include

Note that NDEBUG in -DNDEBUG turns off asserts. Makefile:61 then Makefile:64 causes Makefile to compile cpp files with this flag. prep_sites.cpp contains a lot of assert statements which probably causes the issue with sites when asserts are disabled.

Possible solutions

We can add the following statement to Makefile to filter out the flag:

CPPFLAGS:=$(filter-out -DNDEBUG,$(CPPFLAGS))

Related issues

Possibly related to issues #520 #466 #405
May be related to angsd bioconda package-related issues, too.

[isinaltinkaya]

For replicating this conda behavior:

conda create --name testenv;
conda activate testenv;
echo ${CPPFLAGS};
conda install gcc_linux-64;
echo ${CPPFLAGS};

Before conda install:

(testenv) echo ${CPPFLAGS}                                                         
                                                                                   
(testenv) echo ${CFLAGS}                                                           
                                                                                   
(testenv) echo ${CXXFLAGS}                                                         
                                                                                   
(testenv) echo ${LDFLAGS}                                                          
   

After conda install:

(testenv) echo ${CPPFLAGS}                                                        
-DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/pfs488/lundbeck/scratch/pfs488/miniconda3/envs/testenv/include
(testenv) echo ${CFLAGS}                                                          
-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/pfs488/lundbeck/scratch/pfs488/miniconda3/envs/testenv/include
(testenv) echo ${CXXFLAGS}                                                        
                                                                                   
(testenv) echo ${LDFLAGS}                                                         
-Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,/home/pfs488/lundbeck/scratch/pfs488/miniconda3/envs/testenv/lib -Wl,-rpath-link,/home/pfs488/lundbeck/scratch/pfs488/miniconda3/envs/testenv/lib -L/home/pfs488/lundbeck/scratch/pfs488/miniconda3/envs/testenv/lib