Segatron: Segment-Aware Transformer for Language Modeling and Understanding

Published in AAAI 2021, 2021

He Bai, Peng Shi, Jimmy Lin, Yuqing Xie, Luchen Tan, Kun Xiong, Wen Gao, Ming Li

This paper is about position embedding of Transformers.