Skip to content

IrishCoffee/cudnnMultiHeadAttention

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 

Repository files navigation

cudnnMultiHeadAttention

This is a draft implementation of the formula softmax(QK^T/sqrt(d_k))V.

The reference paper is "Attention is All You Need" (https://arxiv.org/abs/1706.03762).

About

multi-head attention implementation

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published