TurboQuant’s Emergence at ICLR 2026 Highlights Key Differences with EDEN Vector Quantization

The International Conference on Learning Representations (ICLR) 2026, a premier event for artificial intelligence and machine learning research, recently showcased significant advancements in model compression and efficient vector quantization. Among the most talked-about developments was TurboQuant, an online vector quantization method that garnered considerable attention from researchers and industry professionals alike. However, this novel approach shares striking similarities with EDEN, a quantization method that has been progressively developed and refined since its initial introduction. A detailed analysis reveals that TurboQuant’s primary variants, TurboQuant-mse and TurboQuant-prod, represent either a degenerate case or a less efficient alternative to the established EDEN framework, particularly concerning optimized scaling strategies.

TurboQuant’s debut at ICLR 2026 followed a series of publications outlining its capabilities. The method was presented with two main configurations: TurboQuant-mse, designed to minimize Mean Squared Error (MSE), and TurboQuant-prod, which aims for unbiased estimation. These presentations sparked immediate comparisons within the research community, given the existing body of work on EDEN.

EDEN, first introduced as the 1-bit quantization method DRIVE at NeurIPS 2021, was later generalized to support arbitrary bit-widths at ICML 2022. Co-authored by a team including Ran Ben-Basat, Yaniv Ben-Itzhak, Gal Mendelson, Michael Mitzenmacher, Shay Vargaftik, and the author of the original piece, EDEN’s core innovation lies in its sophisticated use of random rotation and analytically derived scaling factors.

A comprehensive comparative study [5] has since demonstrated that TurboQuant-mse is, in essence, a simplified version of EDEN, specifically a case where the optimal scaling factor is not computed. This oversight leads to consistently lower performance for TurboQuant-mse compared to its EDEN counterparts across various configurations.

The EDEN Quantization Process: A Deeper Dive

To understand the distinctions, it’s crucial to examine how EDEN quantizes a d-dimensional vector, such as a gradient update, an embedding, or a key-value cache entry, into a compressed representation using a limited number of bits per coordinate. EDEN employs a four-step process:

1. Random Rotation: The input vector x is rotated using a random orthogonal matrix. This step is critical as it transforms the distribution of the vector’s components.
2. Coordinate-wise Quantization: Each rotated coordinate is then quantized to a lower bit-width. This is typically achieved by mapping continuous values to a discrete set of representative values.
3. Scaling: A scaling factor S is applied to the quantized vector. This is a pivotal step where EDEN’s analytical optimization comes into play.
4. Dequantization (Implicit): For reconstruction, the scaled and quantized vector is effectively de-rotated.

Historically, prior research, such as Suresh et al. (2017) [6], utilized rotation primarily to reduce the dynamic range of vector coordinates – the difference between the largest and smallest values. However, EDEN was among the first quantization schemes to leverage a more profound property of random rotation: the resulting coordinates follow a predictable distribution. This insight allows EDEN to employ a deterministic quantizer in conjunction with a closed-form scaling factor. Depending on the specific application, this scale S can either minimize the Mean Squared Error (MSE) or ensure an unbiased estimate of the original vector. Both scaling strategies are derived analytically, offering an asymptotic reduction in MSE compared to previous methods.

The two primary variants of EDEN, EDEN-biased and EDEN-unbiased, differ fundamentally in their choice of the scaling factor S:

• EDEN-biased: This variant optimizes S to minimize the expected Mean Squared Error (MSE) between the original vector and its quantized approximation. This is particularly relevant for applications where reconstruction accuracy is paramount.
• EDEN-unbiased: This variant selects S to ensure that the expected value of the reconstructed vector equals the original vector, i.e., E[x̂] = x. This is crucial for applications involving the averaging of many quantized vectors, such as in distributed training or certain retrieval tasks.

TurboQuant-mse: A Simplified Approach

When compared to EDEN, TurboQuant-mse aligns with EDEN’s methodology in most respects, with one critical divergence: the derivation and application of the optimal scaling factor S. While TurboQuant-mse also targets MSE minimization, it notably omits the optimized scaling step. Instead, it effectively sets S=1, meaning no analytical scaling is applied after the initial rotation and quantization.

The pseudocode comparison in Figure 1 illustrates this divergence. The three algorithms – EDEN-biased, EDEN-unbiased, and TurboQuant-mse – are identical up to step 5, where the choice of S dictates their behavior.

Figure 1: EDEN's pseudocode instantiated for EDEN-biased, EDEN-unbiased, and TurboQuant-mse. The three are identical except at step 5: the choice of S. Image by author [5].

The Significance of Optimal Scaling

The impact of employing a properly derived scaling factor S becomes more pronounced as the bit-width of quantization increases. At a minimal bit-width of b=1, the performance difference between EDEN and TurboQuant-mse might be marginal. However, at more practical bit-widths commonly used for embeddings and KV caches, such as b=4 bits and for vectors of dimension d=128, EDEN-biased demonstrates a tangible advantage. In these scenarios, EDEN-biased reduces MSE by approximately 2.25% over TurboQuant-mse.

Across a wide range of dimensions, from 16 to 4096, and for all tested bit-widths (b ∈ 1, 2, 3, 4), EDEN-biased consistently exhibits lower vector-normalized MSE (vNMSE). The vNMSE metric, defined as E[||x – x̂||²] / ||x||², normalizes the error by the squared norm of the original vector, providing a relative measure of quantization quality.

Figure 2 visually depicts this performance gap. As the dimensionality of the vectors increases significantly, the optimal scaling factor S in EDEN approaches 1, causing the performance of EDEN-biased and TurboQuant-mse to converge. However, at practical dimensions commonly encountered in machine learning applications (e.g., 128 to 1024), the performance advantage of EDEN-biased remains substantial.

Figure 2: vNMSE vs. dimension comparing EDEN-biased and TurboQuant-mse across bit-widths b ∈ 1,2,3,4 (panels left to right). EDEN-biased (which optimizes the scale factor S) achieves lower error than TurboQuant-mse (which fixes S=1) at every tested dimension. The curves converge at high dimension as the optimal S approaches 1. Image by author [5].

Unbiased Compression: EDEN’s Edge in Accuracy

The performance comparisons for the biased (MSE-minimizing) variants are compelling, but the advantages of EDEN become even more significant when considering unbiased compression. Many applications, including distributed training, approximate attention mechanisms, and inner-product retrieval systems, rely on the property that the expected value of the reconstructed vector equals the original vector (E[x̂] = x). This is essential because these systems often aggregate results from multiple quantized vectors.

While EDEN-unbiased achieves this unbiased property using the same single-pass rotation and quantization algorithm as EDEN-biased, by simply selecting a different, bias-correcting scale S, TurboQuant’s unbiased counterpart, TurboQuant-prod, adopts a different strategy. TurboQuant-prod allocates (b-1) bits to the standard TurboQuant-mse quantization process and reserves the remaining single bit for a Quantized Johnson-Lindenstrauss (QJL) correction applied to the residual error. The QJL method, while related to EDEN at b=1, is characterized by higher variance.

The comparative results highlight a substantial performance gap, with EDEN-unbiased outperforming TurboQuant-prod across all tested configurations. This superiority stems from three core structural advantages inherent in EDEN’s single-pass design:

• Optimized Scaling: EDEN-unbiased directly computes an analytically optimal scale S for bias correction. This is more efficient and accurate than the heuristic approach of reserving bits for post-hoc correction.
• Full Bit-Width Utilization: EDEN utilizes the full bit-width allocated for quantization to represent the rotated vector, whereas TurboQuant-prod dedicates a portion of these bits to a separate correction mechanism.
• Synergy between Rotation and Scaling: EDEN’s approach ensures that the rotation and the subsequent scaling are intrinsically linked and optimized together, leading to a more cohesive and effective compression.

These compounding factors result in a remarkable outcome: 1-bit, 2-bit, and 3-bit EDEN-unbiased are each more accurate than 2-bit, 3-bit, and 4-bit TurboQuant-prod, respectively. This implies that by switching to EDEN, practitioners can achieve comparable or superior accuracy while reducing the bit-width per coordinate by one full bit. Figure 3 visually underscores this advantage.

Figure 3: vNMSE vs. dimension comparing EDEN-unbiased and TurboQuant-prod across bit-widths b ∈ 1,2,3,4 (panels left to right). EDEN-unbiased achieves lower error at every dimension. The gap is large enough that EDEN with b bits often outperforms TurboQuant-prod with b + 1 bits. Image by author [5].

Performance on Standard Benchmarks

The findings are not confined to theoretical analyses; they extend to practical performance on standard benchmarks used in the Approximate Nearest Neighbor (ANN) search domain. TurboQuant’s own published evaluations on datasets such as Stanford’s GloVe pre-trained word vectors and Qdrant’s dbpedia-entities-openai3-text-embedding-3-large embeddings reveal a consistent trend.

Using TurboQuant’s evaluation code, EDEN-biased consistently achieves lower MSE than TurboQuant-mse. Similarly, EDEN-unbiased demonstrates markedly lower inner-product error compared to TurboQuant-prod. Crucially, in nearest-neighbor recall evaluations on both datasets, EDEN consistently outperforms TurboQuant across various bit-width configurations (2 and 4 bits per coordinate), as illustrated in Figure 4.

Figure 4: Nearest-neighbor recall on GloVe and OpenAI3 embeddings at 2 and 4 bits per coordinate. EDEN-unbiased outperforms TurboQuant-prod across all four settings. Image by author [5].

Takeaway: The Undeniable Value of Optimal Scaling

The analysis strongly suggests that the core differentiator between EDEN and TurboQuant lies in the strategic implementation of scaling. EDEN’s scale factor S bridges the gap between the known distribution of rotated vectors and an analytically optimal quantizer. TurboQuant-mse retains EDEN’s rotation and codebook but fixes S=1, rendering it a weaker special case. TurboQuant-prod attempts to compensate for this by introducing a separate 1-bit QJL stage, whereas EDEN-unbiased achieves unbiasedness and superior accuracy through a more integrated, bias-correcting scale.

The implications of these findings are significant for the field of efficient machine learning:

• Efficiency Gains: By leveraging EDEN, researchers and engineers can achieve substantial compression ratios without sacrificing accuracy, leading to reduced memory footprints and faster inference times for large models.
• Democratization of AI: More efficient models can be deployed on resource-constrained devices, broadening the accessibility of advanced AI capabilities.
• Foundation for Future Research: EDEN’s robust theoretical underpinnings and proven effectiveness provide a solid foundation for further innovations in quantization and model compression.

The EDEN framework was initially developed to address challenges in distributed mean estimation for federated and distributed training systems. Its versatility has led to its application in a variety of domains. Notable subsequent work includes embedding compression for document re-ranking (SDR, 2022) [8], adaptation for large language model (LLM) training with NVIDIA’s FP4 format (MS-EDEN in Quartet II, 2026) [10], and data-free LLM weight compression (HIGGS, 2025) [9]. Furthermore, EDEN has been utilized for KV-cache compression in LLMs (AQUA-KV, 2025) [11].

Implementations of EDEN are readily available across major deep learning frameworks. These include PyTorch and TensorFlow versions, integration within Intel’s OpenFL framework [7], and its 1-bit variant in Google’s FedJax, TensorFlow Federated, and TensorFlow Model Optimization [905].

For a complete technical comparison, including detailed experimental methodologies and all supporting figures, the comprehensive note [5] is available. The original derivations, proofs, and further extensions of the EDEN method can be found in the foundational papers [1], [2]. The ongoing discourse and comparative analyses, such as the one presented here, are vital for the continued advancement and practical application of efficient vector quantization techniques in the rapidly evolving landscape of artificial intelligence.