They consist of two main components: an encoder and a decoder. The encoder takes in the input sequence and encodes it into a fixed-length representation, often referred to as the context vector.