File:Attention-qkv.png

From Wikimedia Commons, the free media repository
Jump to navigation Jump to search

Original file (2,634 × 1,145 pixels, file size: 234 KB, MIME type: image/png)

Captions

Captions

calculation of softmax(qK)*V

Summary

[edit]
Description
English: calculation flow through a single attention head
Date
Source Own work
Author Numiri

Licensing

[edit]
I, the copyright holder of this work, hereby publish it under the following license:
w:en:Creative Commons
attribution share alike
This file is licensed under the Creative Commons Attribution-Share Alike 4.0 International license.
You are free:
  • to share – to copy, distribute and transmit the work
  • to remix – to adapt the work
Under the following conditions:
  • attribution – You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
  • share alike – If you remix, transform, or build upon the material, you must distribute your contributions under the same or compatible license as the original.

File history

Click on a date/time to view the file as it appeared at that time.

Date/TimeThumbnailDimensionsUserComment
current18:57, 21 August 2024Thumbnail for version as of 18:57, 21 August 20242,634 × 1,145 (234 KB)Numiri (talk | contribs)tweaked alpha and softmax
15:12, 20 August 2024Thumbnail for version as of 15:12, 20 August 20242,676 × 1,163 (203 KB)Numiri (talk | contribs)wording change. "normalized softmax" to "scaled softmax"
18:10, 18 August 2024Thumbnail for version as of 18:10, 18 August 20242,705 × 1,137 (203 KB)Numiri (talk | contribs)tweaked font style of "x" in Legend. very important.
18:04, 18 August 2024Thumbnail for version as of 18:04, 18 August 2024951 × 147 (25 KB)Numiri (talk | contribs)use concise softmax notation [ ]_xm. tweaked Legend.
15:39, 10 April 2024Thumbnail for version as of 15:39, 10 April 20242,129 × 906 (152 KB)Numiri (talk | contribs)changed Q_w to W_q. same for K and V.
03:04, 10 May 2023Thumbnail for version as of 03:04, 10 May 20231,687 × 739 (64 KB)Numiri (talk | contribs)changed (b) to (*) since we only have one footnote item now.
02:41, 10 May 2023Thumbnail for version as of 02:41, 10 May 20231,708 × 758 (65 KB)Numiri (talk | contribs)grouped attention head, expanded softmax formula
03:17, 9 May 2023Thumbnail for version as of 03:17, 9 May 20231,506 × 712 (59 KB)Numiri (talk | contribs)changed legend labels _x_, a, b, c, d
11:16, 8 May 2023Thumbnail for version as of 11:16, 8 May 20231,517 × 705 (56 KB)Numiri (talk | contribs)Uploaded own work with UploadWizard

You cannot overwrite this file.

There are no pages that use this file.

File usage on other wikis

The following other wikis use this file:

Metadata

This file contains additional information such as Exif metadata which may have been added by the digital camera, scanner, or software program used to create or digitize it. If the file has been modified from its original state, some details such as the timestamp may not fully reflect those of the original file. The timestamp is only as accurate as the clock in the camera, and it may be completely wrong.

Retrieved from "https://commons.wikimedia.org/w/index.php?title=File:Attention-qkv.png&oldid=914271700"