File talk:Attention-qkv.png

From Wikimedia Commons, the free media repository
Jump to navigation Jump to search

Small typo in the equation

[edit]

The argument of the softmax in the equation is "x Q_w * X K_w^T" while it should be "x Q_w * (X K_w)^T". Also it may be a bit confusing that some matrix multiplications are implicit like "x Q_w" and "X K_w" while other have an explicit "*" sign. Also the "*" sign is used for many things in maths, including convolution. If an explicit multiplication sign is used, I suggest using cdot "⋅". 155.245.155.209 09:20, 22 March 2024 (UTC)[reply]

Retrieved from "https://commons.wikimedia.org/w/index.php?title=File_talk:Attention-qkv.png&oldid=862517026"