[WIP] Add Attention is All you need transformer and Translation example

[andr-ec]

This pull request adds the transformer from "attention is all you need"/"the annotated transformer" into the models folder. And adds a translation example for WMT'14 English-German data.
I'd love to hear any feedback, and any help would be great!

This would close #148

TODO:

• implement transformer
• add dataset
• working training loop
• clean up comments
• split dataset into eval and add eval into training loop
• add testing dataset and test loop

[googlebot]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

---

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

[andr-ec]

@googlebot I signed it!

[googlebot]

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

[brettkoonce]

SUGOI DESU NE

[brettkoonce]

might clean this out if it's not in use

[brettkoonce]

WMT == 💪!

[texasmichelle]

Thank you for putting this together! A translation example would indeed be a fantastic addition to the repo.

Between GPT-2 and BERT, we're trying to get a handle on the code duplication among transformer-based models. We'd like to reuse code among common concepts, which a translation example could also utilize. If you're up for it, we could use some help with #436 before compounding the problem by adding more transformer code.

Breaking this up into smaller PRs would also be helpful, such as the standalone dataset or at least leveraging existing components (because how many separate implementations of multi-head attention do we really need? Apropos multi-head attention, it appears you could easily use existing libraries since the code looks nearly identical, but your name is listed as author in the header so maybe there are some major changes).

[marcrasi]

Hi, there hasn't been activity on this PR for a while. We're going to close it to keep our list of open PRs small. Of course, feel free to reopen any time if you have time to come back and work on this!