Overcoming a Theoretical Limitation of Self-Attention

This code accompanies the paper: David Chiang and Peter Cholak, Overcoming a theoretical limitation of self-attention, to appear in Proc. ACL, 2022.

Tested with Python 3.9.6 and PyTorch 1.9.1.

This code is in the public domain.

Name	Name	Last commit message	Last commit date
Latest commit davidweichiang fix bug found by Paul Lintilhac Apr 26, 2023 b0cce20 · Apr 26, 2023 History 18 Commits
README.md	README.md	clean up scripts	Feb 24, 2022
encoder.py	encoder.py	clean up scripts	Feb 24, 2022
first.py	first.py	clean up scripts	Feb 24, 2022
first_baseline.data	first_baseline.data	rerun baseline becasubaseline	Oct 28, 2021
first_exact.py	first_exact.py	same for first	Oct 12, 2021
first_exact_layernorm.py	first_exact_layernorm.py	same for first	Oct 12, 2021
first_scaled.data	first_scaled.data	merge	Oct 28, 2021
graph.py	graph.py	clean up scripts	Feb 24, 2022
line.py	line.py	clean up scripts	Feb 24, 2022
parity.py	parity.py	fix bug found by Paul Lintilhac	Apr 26, 2023
parity_exact.py	parity_exact.py	clean up scripts	Feb 24, 2022
parity_exact_layernorm.py	parity_exact_layernorm.py	clean up scripts	Feb 24, 2022
river.py	river.py	clean up scripts	Feb 24, 2022
stats.py	stats.py	clean up scripts	Feb 24, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Overcoming a Theoretical Limitation of Self-Attention

About

Languages

ndnlp/parity

Folders and files

Latest commit

History

Repository files navigation

Overcoming a Theoretical Limitation of Self-Attention

About

Resources

Stars

Watchers

Forks

Languages