Forwarded from Deep learning channel (Mohsen Fayyaz)
A new summary of The Evolved #Transformer
The recipe:
1. Take the human-designed Transformer
2. Add a new neural architecture search - PDH
3. Whisk with ~200 TPUs to get a much better model - #ET
https://www.lyrn.ai/2019/03/12/the-evolved-transformer/
The recipe:
1. Take the human-designed Transformer
2. Add a new neural architecture search - PDH
3. Whisk with ~200 TPUs to get a much better model - #ET
https://www.lyrn.ai/2019/03/12/the-evolved-transformer/
Lyrn.AI
The Evolved Transformer – Enhancing Transformer with Neural Architecture Search | Lyrn.AI
Neural architecture search (NAS) is the process of algorithmically searching for new designs of neural networks. Though researchers have developed sophisticated architectures over the years, the ability to find the most efficient ones is limited, and recently…