Overcoming a Theoretical Limitation of Self-Attention

This code accompanies the paper: David Chiang and Peter Cholak, Overcoming a theoretical limitation of self-attention, to appear in Proc. ACL, 2022.

Tested with Python 3.9.6 and PyTorch 1.9.1.

This code is in the public domain.

Name		Name	Last commit message	Last commit date
Latest commit History 18 Commits
README.md		README.md
encoder.py		encoder.py
first.py		first.py
first_baseline.data		first_baseline.data
first_exact.py		first_exact.py
first_exact_layernorm.py		first_exact_layernorm.py
first_scaled.data		first_scaled.data
graph.py		graph.py
line.py		line.py
parity.py		parity.py
parity_exact.py		parity_exact.py
parity_exact_layernorm.py		parity_exact_layernorm.py
river.py		river.py
stats.py		stats.py

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Overcoming a Theoretical Limitation of Self-Attention

About

Languages

ndnlp/parity

Folders and files

Latest commit

History

Repository files navigation

Overcoming a Theoretical Limitation of Self-Attention

About

Resources

Stars

Watchers

Forks

Languages