Technical notes by KbWen
Notes on AI, machine learning, Python, and data engineering, plus tools I build.
Selected posts
More selected posts →
Latest posts
AI 為什麼愛附和使用者？RLHF 訓練中的阿諛奉承現象
我們來看看 Anthropic 的研究怎麼測量出 AI 助理會附和使用者，又是怎麼發現原因出在人類偏好資料和偏好模型上。
LLM sycophancy, RLHF and preference models
A look at Anthropic's paper on sycophancy in AI assistants and human preference judgments.
Sparse autoencoders and Claude's Golden Gate Bridge feature
A look at Anthropic's sparse autoencoder work on Claude 3 Sonnet through one feature, the Golden Gate Bridge.
金門大橋 Claude 與稀疏自編碼器：把單一特徵調大會發生什麼事
我們來看看 Anthropic 怎麼用稀疏自編碼器在 Claude 3 Sonnet 裡找到「金門大橋」這個特徵。
Latest by category
More posts
Copying nested lists in Python
A look at what changes when a copied Python list contains other lists.
Python 淺拷貝與巢狀串列
用 list.copy() 備份巢狀串列之後，修改資料會發生什麼事？從兩種修改方式看看淺拷貝的範圍。
RAG 混合檢索的 Reciprocal Rank Fusion
從官方範例看看文字搜尋和向量搜尋的結果，如何用 RRF 合併排序。
Reciprocal rank fusion with multiple vector fields
A closer look at the result lists that go into reciprocal rank fusion when a hybrid search uses several vector fields.
DeepSeek-V3's auxiliary-loss-free load balancing and multi-token prediction
A look at how DeepSeek-V3 keeps load even across its experts during training, and what its multi-token prediction module is for.
DeepSeek-V3 的 Multi-head Latent Attention 如何壓縮 KV 快取
我們來看看 DeepSeek-V3 如何利用 MLA 的低維潛在向量壓縮 KV 快取，解決長序列推論時的記憶體瓶頸。