What Is Rotary Positional Embedding?
Rotary positional embedding rotates query and key features by position so their dot product contains relative-distance information.
Don't Just Read it – Explore It
Rotary positional embedding rotates query and key features by position so their dot product contains relative-distance information.
Linear attention rearranges or approximates attention calculations so cost grows roughly linearly rather than quadratically with sequence length.
FlashAttention computes exact attention in memory-efficient tiles, reducing costly movement between fast and slow GPU memory.
Mamba is a selective state-space architecture that lets sequence inputs control how information enters, persists in, and leaves the hidden state.
A state-space model tracks a compact hidden state that updates over a sequence and produces outputs from that evolving memory.
A bi-encoder converts two inputs independently into vectors that can be compared quickly with a similarity function.
Hard negative mining selects incorrect examples that closely resemble correct matches so a model learns finer distinctions.
A cross-encoder processes two inputs together so attention can model detailed interactions before predicting their relationship.
Retrieval evaluation measures how well a search system finds and orders relevant items for representative queries.
Query expansion adds related terms or concepts to a search request so the system can retrieve relevant material that uses different wording.
A search reranker applies a more detailed model to a small candidate set and produces a better final ordering.
Reciprocal rank fusion combines several result lists by rewarding items that appear near the top of one or more rankings.