【架构搜索】很棒的架构搜索

A curated list of awesome architecture search and hyper-parameter optimization resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning and awesome-deep-learning-papers.

Hyper-parameter optimization has always been a popular field in the Machine Learning community, architecture search just emerges as a rising star in recent years. These are some of the awesome resources!

Table of Contents

• Architecture Search
  • Reinforcement Learning
  • Evolutionary Algorithm
  • Others
• Hyper-Parameter Search
• Contributing
• License

Architecture Search

Reinforcement Learning

• Neural Architecture Search with Reinforcement Learning [pdf]
  • Barret Zoph and Quoc V. Le. ICLR'17
• Designing Neural Network Architectures Using Reinforcement Learning [pdf] [code]
  • Bowen Baker, Otkrist Gupta, Nikhil Naik, Ramesh Raskar. ICLR'17
• Efficient Architecture Search by Network Transformation [pdf] [code]
  • Han Cai, Tianyao Chen, Weinan Zhang, Yong Yu, Jun Wang. AAAI'18
• Learning Transferable Architectures for Scalable Image Recognition [pdf] [nasnet]
  • Barret Zoph, Vijay Vasudevan, Jonathan Shlens, Quoc V. Le. Arxiv 1707
• Practical Block-wise Neural Network Architecture Generation [pdf]
  • Zhao Zhong, Junjie Yan, Wei Wu, Jing Shao, Cheng-Lin Liu. CVPR'18
• A Flexible Approach to Automated RNN Architecture Generation [pdf]
  • Martin Schrimpf, Stephen Merity, James Bradbury, Richard Socher. ICLR'18
• Efficient Neural Architecture Search via Parameter Sharing [pdf] [code (not official)] [code (official)]
  • Hieu Pham, Melody Y. Guan, Barret Zoph, Quoc V. Le, Jeff Dean. Arxiv 1802
• Path-Level Network Transformation for Efficient Architecture Search [pdf] [code]
  • Han Cai, Jiacheng Yang, Weinan Zhang, Song Han, Yong Yu. ICML'18

Evolutionary Algorithm

• Large-Scale Evolution of Image Classifiers [pdf]
  • Esteban Real, Sherry Moore, Andrew Selle, Saurabh Saxena, Yutaka Leon Suematsu, Jie Tan, Quoc Le, Alex Kurakin. ICML'17
• Genetic CNN [pdf] [code]
  • Lingxi Xie and Alan Yuille. ICCV'17
• Hierarchical Representations for Efficient Architecture Search [pdf]
  • Hanxiao Liu, Karen Simonyan, Oriol Vinyals, Chrisantha Fernando, Koray Kavukcuoglu. ICLR'18
• Regularized Evolution for Image Classifier Architecture Search [pdf]
  • Esteban Real, Alok Aggarwal, Yanping Huang, Quoc V Le. Arxiv 1802
• Weight Agnostic Neural Networks [pdf]
  • Adam Gaier, David Ha. NeurIPS'19

Others

• Neural Architecture Optimization [pdf] [code]
  • Renqian Luo, Fei Tian, Tao Qin, Enhong Chen, Tie-Yan Liu. Arxiv 1808
• DeepArchitect: Automatically Designing and Training Deep Architectures [pdf] [code]
  • Renato Negrinho and Geoff Gordon. Arxiv 1704
• SMASH: One-Shot Model Architecture Search through HyperNetworks [pdf] [code]
  • Andrew Brock, Theodore Lim, J.M. Ritchie, Nick Weston. ICLR'18
• Simple and efficient architecture search for Convolutional Neural Networks [pdf]
  • Thomas Elsken, Jan-Hendrik Metzen, Frank Hutter. ICLR'18 Workshop
• Progressive Neural Architecture Search [pdf]
  • Chenxi Liu, Barret Zoph, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, Kevin Murphy. Arxiv 1712
• DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures [pdf]
  • Jin-Dong Dong, An-Chieh Cheng, Da-Cheng Juan, Wei Wei, Min Sun. ECCV'18
• Neural Architecture Search with Bayesian Optimisation and Optimal Transport [pdf]
  • Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schneider, Barnabas Poczos, Eric Xing. Arxiv 1802
• Effective Building Block Design for Deep Convolutional Neural Networks using Search [pdf]
  • Jayanta K Dutta, Jiayi Liu, Unmesh Kurup, Mohak Shah. Arxiv 1801
• DARTS: Differentiable Architecture Search [pdf] [code]
  • Hanxiao Liu, Karen Simonyan, Yiming Yang. Arxiv 1806
• Efficient Neural Architecture Search with Network Morphism [pdf] [code]
  • Haifeng Jin, Qingquan Song, Xia Hu. Arxiv 1806
• Searching for Efficient Multi-Scale Architectures for Dense Image Prediction [pdf]
  • Liang-Chieh Chen, Maxwell D. Collins, Yukun Zhu, George Papandreou, Barret Zoph, Florian Schroff, Hartwig Adam, Jonathon Shlens. Arxiv 1809
• AMC: AutoML for Model Compression and Acceleration on Mobile Devices [pdf] [code (not official)]
  • Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, Song Han. ECCV'18
• MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks [pdf]
  • Ariel Gordon, Elad Eban, Bo Chen, Ofir Nachum, Tien-Ju Yang, Edward Choi. CVPR'18
• Weight Agnostic Neural Networks [pdf]
  • Adam Gaier, David Ha. NeurIPS'19
• Towards Modular and Programmable Architecture Search [pdf] [code]
  • Renato Negrinho, Darshan Patil, Nghia Le, Daniel Ferreira, Matthew Gormley, Geoffrey Gordon. NeurIPS'19

Hyper-Parameter Search

• Speeding up Automatic Hyperparameter Optimization of Deep Neural Networksby Extrapolation of Learning Curves [pdf] [code]
  • Tobias Domhan, Jost Tobias Springenberg, Frank Hutter. IJCAI'15
• Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization [pdf]
  • Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, Ameet Talwalkar. ICLR'17
• Learning Curve Prediction with Bayesian Neural Networks [pdf]
  • Aaron Klein, Stefan Falkner, Jost Tobias Springenberg, Frank Hutter. ICLR'17
• Accelerating Neural Architecture Search using Performance Prediction [pdf]
  • Bowen Baker, Otkrist Gupta, Ramesh Raskar, Nikhil Naik. ICLR'18 Workshop
• Hyperparameter Optimization: A Spectral Approach [pdf] [code]
  • Elad Hazan, Adam Klivans, Yang Yuan. NIPS DLTP Workshop 2017
• Population Based Training of Neural Networks [pdf]
  • Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M. Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, Chrisantha Fernando, Koray Kavukcuoglu. Arxiv 1711

原文:https://github.com/markdtw/awesome-architecture-search