KV cache compression and data over algorithms

Elie Bakouch breaks down the new architecture and its KV cache compression.

The report says “the marginal return of engineering the data and environment pipeline substantially exceeds that of algorithmic novelty in post-training.” Jasper Lu points out that this is coming from a lab known for pioneering new algorithms.