On DeepSpeed Transformer Kernel Attention Mask

[hezq06]

Hi, I'm trying to use DeepSpeed transformer kernel to build a GPT-like system by attention masking. After checking the source code, there is an attention_mask parameter that seems to be able to play with. However, there is no documentation about this part. The bing_bert example uses it only to do input masking. Is it able to do attention masking with the attention_mask parameter?