change the device

AutoDL-Projects/.github/CODE-OF-CONDUCT.md

@@ -0,0 +1,76 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, sex characteristics, gender identity and expression,
+level of experience, education, socio-economic status, nationality, personal
+appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+ advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+ address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+ professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at dongxuanyi888@gmail.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see
+https://www.contributor-covenant.org/faq

AutoDL-Projects/.github/CONTRIBUTING.md

@@ -0,0 +1,33 @@
+# Contributing to AutoDL-Projects
+
+:+1::tada: First off, thanks for taking the time to contribute! :tada::+1:
+
+The following is a set of guidelines for contributing to AutoDL-Projects.
+
+## Table Of Contents
+
+[How Can I Contribute?](#how-can-i-contribute)
+  * [Reporting Bugs](#reporting-bugs)
+  * [Suggesting Enhancements](#suggesting-enhancements)
+  * [Your First Code Contribution](#your-first-code-contribution)
+
+## How Can I Contribute?
+
+### Reporting Bugs
+
+This section guides you through submitting a bug report for AutoDL-Projects.
+Following these guidelines helps maintainers and the community understand your report :pencil:, reproduce the behavior :computer: :computer:, and find related reports :mag_right:.
+
+When you are creating a bug report, please include as many details as possible.
+Fill out [the required template](https://github.com/D-X-Y/AutoDL-Projects/blob/main/.github/ISSUE_TEMPLATE/bug-report.md). The information it asks for helps us resolve issues faster.
+
+> **Note:** If you find a **Closed** issue that seems like it is the same thing that you're experiencing, open a new issue and include a link to the original issue in the body of your new one.
+
+### Suggesting Enhancements
+
+Please feel free to email me (dongxuanyi888@gmail.com).
+
+### Your First Code Contribution
+
+Please feel free to open an Pull Requests.
+

AutoDL-Projects/.github/ISSUE_TEMPLATE/bug-report.md

@@ -0,0 +1,24 @@
+---
+name: Bug Report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Please provide a small script to reproduce the behavior:
+```
+codes to reproduce the bug
+```
+Please let me know your OS, Python version, PyTorch version.
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.

AutoDL-Projects/.github/ISSUE_TEMPLATE/question.md

@@ -0,0 +1,15 @@
+---
+name: Questions w.r.t. an AutoDL algorithm
+about: Ask questions about or discuss on some algorithms
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Which Algorithm**
+Please put the title of the AutoDL algorithm that you want to ask here.
+
+**Describe the Question**
+A clear and concise description of what the bug is.
+

AutoDL-Projects/.github/workflows/test-basic.yaml

@@ -0,0 +1,67 @@
+name: Test Spaces
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - main
+
+
+jobs:
+  build:
+    strategy:
+      matrix:
+        os: [ubuntu-18.04, ubuntu-20.04, macos-latest]
+        python-version: [3.6, 3.7, 3.8, 3.9]
+
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@v2
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Lint with Black
+        run: |
+          python -m pip install black
+          python --version
+          python -m black --version
+          echo $PWD ; ls
+          python -m black ./exps -l 88 --check --diff --verbose
+          python -m black ./tests -l 88 --check --diff --verbose
+          python -m black ./xautodl/x* -l 88 --check --diff --verbose
+          python -m black ./xautodl/spaces -l 88 --check --diff --verbose
+          python -m black ./xautodl/trade_models -l 88 --check --diff --verbose
+          python -m black ./xautodl/procedures -l 88 --check --diff --verbose
+          python -m black ./xautodl/config_utils -l 88 --check --diff --verbose
+          python -m black ./xautodl/log_utils -l 88 --check --diff --verbose
+
+      - name: Install XAutoDL from source
+        run: |
+          pip install .
+
+      - name: Test Search Space
+        run: |
+          python -m pip install pytest
+          python -m pip install torch torchvision
+          python -m pip install parameterized
+          echo $PWD
+          echo "Show what we have here:"
+          ls
+          python --version
+          python -m pytest ./tests/test_import.py -s
+          python -m pytest ./tests/test_basic_space.py -s
+        shell: bash
+
+      - name: Test Math
+        run: |
+          python -m pytest ./tests/test_math*.py -s
+        shell: bash
+
+      - name: Test Synthetic Data
+        run: |
+          python -m pytest ./tests/test_synthetic*.py -s
+        shell: bash

AutoDL-Projects/.github/workflows/test-misc.yaml

@@ -0,0 +1,41 @@
+name: Test Xmisc
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - main
+
+
+jobs:
+  build:
+    strategy:
+      matrix:
+        os: [ubuntu-18.04, ubuntu-20.04, macos-latest]
+        python-version: [3.6, 3.7, 3.8, 3.9]
+
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@v2
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install XAutoDL from source
+        run: |
+          pip install .
+
+      - name: Test Xmisc
+        run: |
+          python -m pip install pytest
+          python -m pip install torch torchvision
+          python -m pip install parameterized
+          echo $PWD
+          echo "Show what we have here:"
+          ls
+          python --version
+          python -m pytest ./tests/test_misc* -s
+        shell: bash

AutoDL-Projects/.github/workflows/test-super-layer_model.yaml

@@ -0,0 +1,42 @@
+name: Test Super Model
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - main
+
+
+jobs:
+  build:
+    strategy:
+      matrix:
+        os: [ubuntu-18.04, ubuntu-20.04, macos-latest]
+        python-version: [3.6, 3.7, 3.8, 3.9]
+
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@v2
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install XAutoDL from source
+        run: |
+          pip install .
+
+      - name: Test Super Model
+        run: |
+          python -m pip install pytest
+          python -m pip install parameterized
+          python -m pip install torch torchvision
+          python -m pytest ./tests/test_super_*.py
+        shell: bash
+
+      - name: Test TAS (NeurIPS 2019)
+        run: |
+          python -m pytest ./tests/test_tas.py
+        shell: bash

AutoDL-Projects/.gitignore

@@ -0,0 +1,138 @@
+.DS_Store
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+env/
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib64/
+parts/
+sdist/
+var/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*,cover
+.hypothesis/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# IPython Notebook
+.ipynb_checkpoints
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# dotenv
+.env
+
+# virtualenv
+venv/
+ENV/
+
+# Spyder project settings
+.spyderproject
+
+# Rope project settings
+.ropeproject
+
+# Pycharm project
+.idea
+snapshots
+*.pytorch
+*.tar.bz
+data
+.*.swp
+main_main.py
+*.pdf
+*/*.pdf
+
+# Device
+scripts-nas/.nfs00*
+*/.nfs00*
+*.DS_Store
+
+# logs and snapshots
+output
+logs
+
+# snapshot
+a.pth
+cal-merge*.sh
+GPU-*.sh
+cal.sh
+aaa
+cx.sh
+
+NAS-Bench-*-v1_0.pth
+lib/NAS-Bench-*-v1_0.pth
+others/TF
+scripts-search/l2s-algos
+TEMP-L.sh
+
+.nfs00*
+*.swo
+*/*.swo
+
+# Visual Studio Code
+.vscode
+mlruns*
+outputs
+
+pytest_cache
+*.pkl
+*.pth
+
+*.tgz

AutoDL-Projects/.gitmodules

@@ -0,0 +1,9 @@
+[submodule ".latent-data/qlib"]
+	path = .latent-data/qlib
+	url = https://github.com/microsoft/qlib.git
+[submodule ".latent-data/NATS-Bench"]
+	path = .latent-data/NATS-Bench
+	url = https://github.com/D-X-Y/NATS-Bench.git
+[submodule ".latent-data/NAS-Bench-201"]
+	path = .latent-data/NAS-Bench-201
+	url = https://github.com/D-X-Y/NAS-Bench-201.git

AutoDL-Projects/CHANGE-LOG.md

@@ -0,0 +1,12 @@
+# This file shows the major updates of this repo.
+
+- [2020.04.11] [4ef9531](https://github.com/D-X-Y/AutoDL-Projects/tree/4ef9531) Add change log as `CHANGE-LOG.md`.
+- [2019.12.20] [69ca086](https://github.com/D-X-Y/AutoDL-Projects/tree/69ca086) Release NAS-Bench-201.
+- [2019.09.28] [f8f3f38](https://github.com/D-X-Y/AutoDL-Projects/tree/f8f3f38) TAS and SETN codes were publicly released.
+- [2019.01.31] [13e908f](https://github.com/D-X-Y/AutoDL-Projects/tree/13e908f) GDAS codes were publicly released.
+- [2020.07.01] [a45808b](https://github.com/D-X-Y/AutoDL-Projects/tree/a45808b) Upgrade NAS-API to the 2.0 version.
+- [2020.09.16] [7052265](https://github.com/D-X-Y/AutoDL-Projects/tree/7052265) Create NATS-BENCH.
+- [2020.10.15] [446262a](https://github.com/D-X-Y/AutoDL-Projects/tree/446262a) Update NATS-BENCH to version 1.0
+- [2020.12.20] [dae387a](https://github.com/D-X-Y/AutoDL-Projects/tree/dae387a) Update NATS-BENCH to version 1.1
+- [2021.05.18] [98fadf8](https://github.com/D-X-Y/AutoDL-Projects/tree/98fadf8) Before moving to `xautodl`
+- [2021.05.21] [df99173](https://github.com/D-X-Y/AutoDL-Projects/tree/df99173) `xautodl` is close to ready

AutoDL-Projects/LICENSE.md

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) since 2019.01.01, author: Xuanyi Dong (GitHub: https://github.com/D-X-Y)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

AutoDL-Projects/README.md

@@ -0,0 +1,163 @@
+<p align="center">
+<img src="https://xuanyidong.com/resources/images/AutoDL-log.png" width="400"/>
+</p>
+
+---------
+[![MIT licensed](https://img.shields.io/badge/license-MIT-brightgreen.svg)](LICENSE.md)
+
+Automated Deep Learning Projects (AutoDL-Projects) is an open source, lightweight, but useful project for everyone.
+This project implemented several neural architecture search (NAS) and hyper-parameter optimization (HPO) algorithms.
+中文介绍见[README_CN.md](https://github.com/D-X-Y/AutoDL-Projects/tree/main/docs/README_CN.md)
+
+**Who should consider using AutoDL-Projects**
+
+- Beginners who want to **try different AutoDL algorithms**
+- Engineers who want to **try AutoDL** to investigate whether AutoDL works on your projects
+- Researchers who want to **easily** implement and experiement **new** AutoDL algorithms.
+
+**Why should we use AutoDL-Projects**
+- Simple library dependencies
+- All algorithms are in the same codebase
+- Active maintenance
+
+## AutoDL-Projects Capabilities
+
+At this moment, this project provides the following algorithms and scripts to run them. Please see the details in the link provided in the description column.
+
+<table>
+ <tbody>
+    <tr align="center" valign="bottom">
+      <th>Type</th>
+      <th>ABBRV</th>
+      <th>Algorithms</th>
+      <th>Description</th>
+    </tr>
+    <tr> <!-- (1-st row) -->
+    <td rowspan="6" align="center" valign="middle" halign="middle"> NAS </td>
+    <td align="center" valign="middle"> TAS </td>
+    <td align="center" valign="middle"> <a href="https://arxiv.org/abs/1905.09717">Network Pruning via Transformable Architecture Search</a> </td>
+    <td align="center" valign="middle"> <a href="https://github.com/D-X-Y/AutoDL-Projects/tree/main/docs/NeurIPS-2019-TAS.md">NeurIPS-2019-TAS.md</a> </td>
+    </tr>
+    <tr> <!-- (2-nd row) -->
+    <td align="center" valign="middle"> DARTS </td>
+    <td align="center" valign="middle"> <a href="https://arxiv.org/abs/1806.09055">DARTS: Differentiable Architecture Search</a> </td>
+    <td align="center" valign="middle"> <a href="https://github.com/D-X-Y/AutoDL-Projects/tree/main/docs/ICLR-2019-DARTS.md">ICLR-2019-DARTS.md</a> </td>
+    </tr>
+    <tr> <!-- (3-nd row) -->
+    <td align="center" valign="middle"> GDAS </td>
+    <td align="center" valign="middle"> <a href="https://arxiv.org/abs/1910.04465">Searching for A Robust Neural Architecture in Four GPU Hours</a> </td>
+    <td align="center" valign="middle"> <a href="https://github.com/D-X-Y/AutoDL-Projects/tree/main/docs/CVPR-2019-GDAS.md">CVPR-2019-GDAS.md</a> </td>
+    </tr>
+    <tr> <!-- (4-rd row) -->
+    <td align="center" valign="middle"> SETN </td>
+    <td align="center" valign="middle"> <a href="https://arxiv.org/abs/1910.05733">One-Shot Neural Architecture Search via Self-Evaluated Template Network</a> </td>
+    <td align="center" valign="middle"> <a href="https://github.com/D-X-Y/AutoDL-Projects/tree/main/docs/ICCV-2019-SETN.md">ICCV-2019-SETN.md</a> </td>
+    </tr>
+    <tr> <!-- (5-th row) -->
+    <td align="center" valign="middle"> NAS-Bench-201 </td>
+    <td align="center" valign="middle"> <a href="https://openreview.net/forum?id=HJxyZkBKDr"> NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search</a> </td>
+    <td align="center" valign="middle"> <a href="https://github.com/D-X-Y/AutoDL-Projects/tree/main/docs/NAS-Bench-201.md">NAS-Bench-201.md</a> </td>
+    </tr>
+    <tr> <!-- (6-th row) -->
+    <td align="center" valign="middle"> NATS-Bench </td>
+    <td align="center" valign="middle"> <a href="https://xuanyidong.com/assets/projects/NATS-Bench"> NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size</a> </td>
+    <td align="center" valign="middle"> <a href="https://github.com/D-X-Y/NATS-Bench/blob/main/README.md">NATS-Bench.md</a> </td>
+    </tr>
+    <tr> <!-- (7-th row) -->
+    <td align="center" valign="middle"> ... </td>
+    <td align="center" valign="middle"> ENAS / REA / REINFORCE / BOHB </td>
+    <td align="center" valign="middle"> Please check the original papers </td>
+    <td align="center" valign="middle"> <a href="https://github.com/D-X-Y/AutoDL-Projects/tree/main/docs/NAS-Bench-201.md">NAS-Bench-201.md</a>  <a href="https://github.com/D-X-Y/NATS-Bench/blob/main/README.md">NATS-Bench.md</a> </td>
+    </tr>
+    <tr> <!-- (start second block) -->
+    <td rowspan="1" align="center" valign="middle" halign="middle"> HPO </td>
+    <td align="center" valign="middle"> HPO-CG </td>
+    <td align="center" valign="middle"> Hyperparameter optimization with approximate gradient </td>
+    <td align="center" valign="middle"> coming soon </a> </td>
+    </tr>
+    <tr> <!-- (start third block) -->
+    <td rowspan="1" align="center" valign="middle" halign="middle"> Basic </td>
+    <td align="center" valign="middle"> ResNet </td>
+    <td align="center" valign="middle"> Deep Learning-based Image Classification </td>
+    <td align="center" valign="middle"> <a href="https://github.com/D-X-Y/AutoDL-Projects/tree/main/docs/BASELINE.md">BASELINE.md</a> </a> </td>
+    </tr>
+ </tbody>
+</table>
+
+
+
+## Requirements and Preparation
+
+
+**First of all**, please use `pip install .` to install `xautodl` library.
+
+Please install `Python>=3.6` and `PyTorch>=1.5.0`. (You could use lower versions of Python and PyTorch, but may have bugs).
+Some visualization codes may require `opencv`.
+
+CIFAR and ImageNet should be downloaded and extracted into `$TORCH_HOME`.
+Some methods use knowledge distillation (KD), which require pre-trained models. Please download these models from [Google Drive](https://drive.google.com/open?id=1ANmiYEGX-IQZTfH8w0aSpj-Wypg-0DR-) (or train by yourself) and save into `.latent-data`.
+
+Please use
+```
+git clone --recurse-submodules https://github.com/D-X-Y/AutoDL-Projects.git XAutoDL
+```
+to download this repo with submodules.
+
+## Citation
+
+If you find that this project helps your research, please consider citing the related paper:
+```
+@inproceedings{dong2021autohas,
+  title     = {{AutoHAS}: Efficient Hyperparameter and Architecture Search},
+  author    = {Dong, Xuanyi and Tan, Mingxing and Yu, Adams Wei and Peng, Daiyi and Gabrys, Bogdan and Le, Quoc V},
+  booktitle = {2nd Workshop on Neural Architecture Search at International Conference on Learning Representations (ICLR)},
+  year      = {2021}
+}
+@article{dong2021nats,
+  title   = {{NATS-Bench}: Benchmarking NAS Algorithms for Architecture Topology and Size},
+  author  = {Dong, Xuanyi and Liu, Lu and Musial, Katarzyna and Gabrys, Bogdan},
+  doi     = {10.1109/TPAMI.2021.3054824},
+  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
+  year    = {2021},
+  note    = {\mbox{doi}:\url{10.1109/TPAMI.2021.3054824}}
+}
+@inproceedings{dong2020nasbench201,
+  title     = {{NAS-Bench-201}: Extending the Scope of Reproducible Neural Architecture Search},
+  author    = {Dong, Xuanyi and Yang, Yi},
+  booktitle = {International Conference on Learning Representations (ICLR)},
+  url       = {https://openreview.net/forum?id=HJxyZkBKDr},
+  year      = {2020}
+}
+@inproceedings{dong2019tas,
+  title     = {Network Pruning via Transformable Architecture Search},
+  author    = {Dong, Xuanyi and Yang, Yi},
+  booktitle = {Neural Information Processing Systems (NeurIPS)},
+  pages     = {760--771},
+  year      = {2019}
+}
+@inproceedings{dong2019one,
+  title     = {One-Shot Neural Architecture Search via Self-Evaluated Template Network},
+  author    = {Dong, Xuanyi and Yang, Yi},
+  booktitle = {Proceedings of the IEEE International Conference on Computer Vision (ICCV)},
+  pages     = {3681--3690},
+  year      = {2019}
+}
+@inproceedings{dong2019search,
+  title     = {Searching for A Robust Neural Architecture in Four GPU Hours},
+  author    = {Dong, Xuanyi and Yang, Yi},
+  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
+  pages     = {1761--1770},
+  year      = {2019}
+}
+```
+
+# Others
+
+If you want to contribute to this repo, please see [CONTRIBUTING.md](.github/CONTRIBUTING.md).
+Besides, please follow [CODE-OF-CONDUCT.md](.github/CODE-OF-CONDUCT.md).
+
+We use [`black`](https://github.com/psf/black) for Python code formatter.
+Please use `black . -l 88`.
+
+# License
+The entire codebase is under the [MIT license](LICENSE.md).

AutoDL-Projects/configs/NeurIPS-2019/C010-ResNet110.config

@@ -0,0 +1,13 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "110"],
+  "module"             : ["str"   , "ResNetBasicblock"],
+  "super_type"         : ["str"   , "infer-shape"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "10"],
+  "search_mode"        : ["str"   , "shape"],
+  "xchannels"          : ["int"   , ["3", "16", "14", "16", "14", "16", "16", "16", "16", "16", "14", "16", "16", "16", "12", "16", "16", "16", "9", "16", "8", "16", "4", "16", "4", "4", "4", "16", "4", "4", "4", "4", "6", "6", "4", "6", "11", "4", "32", "32", "32", "32", "32", "32", "32", "32", "28", "32", "32", "28", "22", "22", "22", "32", "32", "25", "28", "9", "9", "28", "12", "9", "12", "32", "9", "9", "22", "12", "16", "9", "12", "9", "9", "9", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "38", "64", "25", "19", "19", "19", "19", "19", "25", "32", "19", "19", "25", "25", "19", "19", "38", "38", "19", "19", "51"]],
+  "xblocks"            : ["int"   , ["11", "11", "9"]],
+  "estimated_FLOP"     : ["float" , "117.498238"]
+}

AutoDL-Projects/configs/NeurIPS-2019/C010-ResNet164.config

@@ -0,0 +1,13 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "164"],
+  "module"             : ["str"   , "ResNetBottleneck"],
+  "super_type"         : ["str"   , "infer-shape"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "10"],
+  "search_mode"        : ["str"   , "shape"],
+  "xchannels"          : ["int"   , ["3", "16", "8", "16", "64", "6", "16", "64", "14", "16", "64", "8", "16", "64", "4", "16", "64", "6", "16", "64", "6", "16", "64", "11", "11", "64", "4", "14", "64", "4", "4", "57", "4", "16", "64", "9", "12", "57", "4", "16", "64", "4", "8", "57", "6", "6", "51", "6", "4", "44", "6", "4", "57", "6", "6", "19", "32", "32", "128", "32", "32", "128", "32", "32", "128", "9", "32", "128", "32", "32", "128", "25", "28", "115", "12", "32", "128", "32", "32", "128", "32", "32", "128", "32", "32", "102", "28", "32", "128", "16", "32", "128", "28", "19", "128", "32", "9", "51", "16", "12", "102", "12", "22", "115", "9", "12", "51", "12", "16", "38", "64", "64", "256", "64", "64", "256", "64", "64", "256", "64", "64", "256", "64", "64", "256", "64", "64", "256", "64", "64", "256", "64", "64", "256", "64", "64", "256", "64", "64", "256", "64", "57", "204", "64", "25", "179", "19", "25", "204", "44", "19", "153", "38", "25", "76", "19", "32", "128", "19", "51", "76", "57", "32", "76"]],
+  "xblocks"            : ["int"   , ["13", "15", "13"]],
+  "estimated_FLOP"     : ["float" , "173.023672"]
+}

AutoDL-Projects/configs/NeurIPS-2019/C010-ResNet20.config

@@ -0,0 +1,13 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "20"],
+  "module"             : ["str"   , "ResNetBasicblock"],
+  "super_type"         : ["str"   , "infer-shape"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "10"],
+  "search_mode"        : ["str"   , "shape"],
+  "xchannels"          : ["int"   , ["3", "16", "6", "4", "4", "4", "4", "4", "32", "32", "12", "19", "32", "28", "64", "64", "64", "64", "64", "44"]],
+  "xblocks"            : ["int"   , ["3", "3", "3"]],
+  "estimated_FLOP"     : ["float" , "22.444472"]
+}

AutoDL-Projects/configs/NeurIPS-2019/C010-ResNet32.config

@@ -0,0 +1,13 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "32"],
+  "module"             : ["str"   , "ResNetBasicblock"],
+  "super_type"         : ["str"   , "infer-shape"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "10"],
+  "search_mode"        : ["str"   , "shape"],
+  "xchannels"          : ["int"   , ["3", "16", "6", "4", "12", "4", "4", "16", "9", "9", "6", "14", "32", "32", "9", "19", "28", "9", "32", "19", "32", "9", "64", "64", "64", "64", "64", "64", "64", "32", "38", "32"]],
+  "xblocks"            : ["int"   , ["5", "5", "5"]],
+  "estimated_FLOP"     : ["float" , "34.945344"]
+}

AutoDL-Projects/configs/NeurIPS-2019/C010-ResNet56.config

@@ -0,0 +1,13 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "56"],
+  "module"             : ["str"   , "ResNetBasicblock"],
+  "super_type"         : ["str"   , "infer-shape"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "10"],
+  "search_mode"        : ["str"   , "shape"],
+  "xchannels"          : ["int"   , ["3", "16", "16", "16", "14", "11", "9", "16", "12", "16", "6", "16", "4", "8", "4", "14", "6", "4", "4", "4", "32", "32", "32", "32", "32", "32", "22", "28", "32", "32", "19", "9", "19", "16", "9", "25", "16", "9", "64", "64", "64", "64", "64", "64", "64", "64", "64", "51", "19", "19", "32", "19", "19", "32", "19", "25"]],
+  "xblocks"            : ["int"   , ["5", "5", "5"]],
+  "estimated_FLOP"     : ["float" , "57.93305"]
+}

AutoDL-Projects/configs/NeurIPS-2019/C100-ResNet110.config

@@ -0,0 +1,13 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "110"],
+  "module"             : ["str"   , "ResNetBasicblock"],
+  "super_type"         : ["str"   , "infer-shape"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "100"],
+  "search_mode"        : ["str"   , "shape"],
+  "xchannels"          : ["int"   , ["3", "16", "14", "16", "11", "14", "16", "16", "11", "16", "9", "14", "12", "16", "16", "16", "8", "16", "14", "16", "12", "4", "11", "16", "4", "4", "4", "16", "12", "4", "8", "4", "9", "4", "6", "14", "4", "4", "32", "32", "32", "32", "28", "28", "32", "32", "32", "32", "32", "28", "32", "28", "25", "32", "32", "32", "9", "9", "32", "32", "9", "25", "28", "32", "28", "9", "9", "32", "12", "12", "9", "22", "12", "9", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "44", "64", "57", "19", "19", "19", "19", "25", "19", "25", "19", "25", "19", "19", "25", "19", "19", "19", "25", "25", "19"]],
+  "xblocks"            : ["int"   , ["13", "9", "11"]],
+  "estimated_FLOP"     : ["float" , "117.653164"]
+}

AutoDL-Projects/configs/NeurIPS-2019/C100-ResNet164.config

@@ -0,0 +1,13 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "164"],
+  "module"             : ["str"   , "ResNetBottleneck"],
+  "super_type"         : ["str"   , "infer-shape"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "100"],
+  "search_mode"        : ["str"   , "shape"],
+  "xchannels"          : ["int"   , ["3", "16", "16", "16", "57", "6", "11", "64", "4", "6", "51", "6", "9", "64", "4", "8", "64", "4", "14", "64", "4", "8", "64", "4", "8", "64", "6", "12", "64", "6", "16", "64", "8", "16", "64", "14", "12", "64", "4", "16", "64", "4", "14", "64", "11", "16", "64", "4", "14", "64", "11", "4", "64", "4", "4", "19", "25", "32", "128", "28", "32", "115", "28", "32", "128", "25", "32", "128", "32", "32", "128", "25", "32", "128", "12", "32", "128", "25", "32", "128", "28", "32", "128", "25", "28", "128", "32", "32", "128", "28", "19", "128", "32", "32", "128", "19", "28", "128", "9", "19", "128", "28", "9", "89", "28", "19", "128", "9", "16", "38", "64", "64", "256", "64", "64", "256", "64", "64", "256", "64", "64", "256", "64", "64", "256", "64", "64", "256", "64", "64", "256", "64", "64", "256", "64", "64", "256", "64", "64", "204", "64", "64", "179", "64", "64", "102", "64", "64", "102", "44", "19", "76", "19", "19", "76", "19", "38", "76", "25", "38", "153", "44", "25", "230"]],
+  "xblocks"            : ["int"   , ["15", "15", "15"]],
+  "estimated_FLOP"     : ["float" , "165.583512"]
+}

AutoDL-Projects/configs/NeurIPS-2019/C100-ResNet20.config

@@ -0,0 +1,13 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "20"],
+  "module"             : ["str"   , "ResNetBasicblock"],
+  "super_type"         : ["str"   , "infer-shape"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "100"],
+  "search_mode"        : ["str"   , "shape"],
+  "xchannels"          : ["int"   , ["3", "16", "4", "4", "4", "4", "6", "4", "32", "32", "9", "19", "32", "28", "64", "64", "64", "64", "64", "64"]],
+  "xblocks"            : ["int"   , ["3", "3", "3"]],
+  "estimated_FLOP"     : ["float" , "22.433792"]
+}

AutoDL-Projects/configs/NeurIPS-2019/C100-ResNet32.config

@@ -0,0 +1,12 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "32"],
+  "module"             : ["str"   , "ResNetBasicblock"],
+  "super_type"         : ["str"   , "infer-shape"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "100"],
+  "xchannels"          : ["int"   , ["3", "16", "4", "4", "6", "11", "6", "4", "8", "4", "4", "4", "32", "32", "9", "28", "28", "28", "28", "28", "32", "32", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64"]],
+  "xblocks"            : ["int"   , ["5", "5", "5"]],
+  "estimated_FLOP"     : ["float" , "42.47"]
+}

AutoDL-Projects/configs/NeurIPS-2019/C100-ResNet56.config

@@ -0,0 +1,13 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "56"],
+  "module"             : ["str"   , "ResNetBasicblock"],
+  "super_type"         : ["str"   , "infer-shape"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "100"],
+  "search_mode"        : ["str"   , "shape"],
+  "xchannels"          : ["int"   , ["3", "16", "16", "9", "14", "16", "14", "16", "8", "16", "8", "14", "4", "4", "4", "8", "4", "6", "4", "4", "32", "32", "32", "28", "32", "32", "32", "22", "32", "32", "32", "9", "25", "19", "25", "12", "9", "9", "64", "64", "64", "64", "64", "64", "64", "64", "64", "51", "19", "19", "19", "19", "25", "38", "19", "19"]],
+  "xblocks"            : ["int"   , ["5", "5", "7"]],
+  "estimated_FLOP"     : ["float" , "59.472556"]
+}

AutoDL-Projects/configs/NeurIPS-2019/ImageNet-ResNet18V1.config

@@ -0,0 +1,14 @@
+{
+  "dataset"            : ["str"   , "imagenet"],
+  "arch"               : ["str"   , "resnet"],
+  "block_name"         : ["str"   , "BasicBlock"],
+  "layers"             : ["int"   , ["2", "2", "2", "2"]],
+  "deep_stem"          : ["bool"  , "0"],
+  "zero_init_residual" : ["bool"  , "1"],
+  "class_num"          : ["int"   , "1000"],
+  "search_mode"        : ["str"   , "shape"],
+  "xchannels"          : ["int"   , ["3", "64", "25", "64", "38", "19", "128", "128", "38", "38", "256", "256", "256", "256", "512", "512", "512", "512"]],
+  "xblocks"            : ["int"   , ["1", "1", "2", "2"]],
+  "super_type"         : ["str"   , "infer-shape"],
+  "estimated_FLOP"     : ["float" , "1120.44032"]
+}

AutoDL-Projects/configs/NeurIPS-2019/ImageNet-ResNet50V1.config

@@ -0,0 +1,14 @@
+{
+  "dataset"            : ["str"   , "imagenet"],
+  "arch"               : ["str"   , "resnet"],
+  "block_name"         : ["str"   , "Bottleneck"],
+  "layers"             : ["int"   , ["3", "4", "6", "3"]],
+  "deep_stem"          : ["bool"  , "0"],
+  "zero_init_residual" : ["bool"  , "1"],
+  "class_num"          : ["int"   , "1000"],
+  "search_mode"        : ["str"   , "shape"],
+  "xchannels"          : ["int"   , ["3", "45", "45", "30", "102", "33", "60", "154", "68", "70", "180", "38", "38", "307", "38", "38", "410", "64", "128", "358", "38", "51", "256", "76", "76", "512", "76", "76", "512", "179", "256", "614", "100", "102", "307", "179", "230", "614", "204", "102", "307", "153", "153", "1228", "512", "512", "1434", "512", "512", "1844"]],
+  "xblocks"            : ["int"   , ["3", "4", "5", "3"]],
+  "super_type"         : ["str"   , "infer-shape"],
+  "estimated_FLOP"     : ["float" , "2291.316289"]
+}

AutoDL-Projects/configs/archs/CIFAR-ResNet08.config

@@ -0,0 +1,8 @@
+{
+  "dataset"   : ["str",  "cifar"],
+  "arch"      : ["str",  "resnet"],
+  "depth"     : ["int",   8],
+  "module"    : ["str",  "ResNetBasicblock"],
+  "super_type": ["str" , "basic"],
+  "zero_init_residual" : ["bool",  "0"]
+}

AutoDL-Projects/configs/archs/CIFAR-ResNet1001.config

@@ -0,0 +1,8 @@
+{
+  "dataset"   : ["str",  "cifar"],
+  "arch"      : ["str",  "resnet"],
+  "depth"     : ["int",  1001],
+  "module"    : ["str",  "ResNetBottleneck"],
+  "super_type": ["str" , "basic"],
+  "zero_init_residual" : ["bool",  "0"]
+}

AutoDL-Projects/configs/archs/CIFAR-ResNet110.config

@@ -0,0 +1,8 @@
+{
+  "dataset"   : ["str",  "cifar"],
+  "arch"      : ["str",  "resnet"],
+  "depth"     : ["int",  110],
+  "module"    : ["str",  "ResNetBasicblock"],
+  "super_type": ["str" , "basic"],
+  "zero_init_residual" : ["bool",  "0"]
+}

AutoDL-Projects/configs/archs/CIFAR-ResNet164.config

@@ -0,0 +1,8 @@
+{
+  "dataset"   : ["str",  "cifar"],
+  "arch"      : ["str",  "resnet"],
+  "depth"     : ["int",  164],
+  "module"    : ["str",  "ResNetBottleneck"],
+  "super_type": ["str" , "basic"],
+  "zero_init_residual" : ["bool",  "0"]
+}

AutoDL-Projects/configs/archs/CIFAR-ResNet20.config

@@ -0,0 +1,8 @@
+{
+  "dataset"   : ["str",  "cifar"],
+  "arch"      : ["str",  "resnet"],
+  "depth"     : ["int",  20],
+  "module"    : ["str",  "ResNetBasicblock"],
+  "super_type": ["str" , "basic"],
+  "zero_init_residual" : ["bool",  "0"]
+}

AutoDL-Projects/configs/archs/CIFAR-ResNet32.config

@@ -0,0 +1,8 @@
+{
+  "dataset"   : ["str",  "cifar"],
+  "arch"      : ["str",  "resnet"],
+  "depth"     : ["int",  32],
+  "module"    : ["str",  "ResNetBasicblock"],
+  "super_type": ["str" , "basic"],
+  "zero_init_residual" : ["bool",  "0"]
+}

AutoDL-Projects/configs/archs/CIFAR-ResNet56.config

@@ -0,0 +1,8 @@
+{
+  "dataset"   : ["str",  "cifar"],
+  "arch"      : ["str",  "resnet"],
+  "depth"     : ["int",  56],
+  "module"    : ["str",  "ResNetBasicblock"],
+  "super_type": ["str" , "basic"],
+  "zero_init_residual" : ["bool",  "0"]
+}

AutoDL-Projects/configs/archs/CIFAR-SIM05.config

@@ -0,0 +1,7 @@
+{
+  "dataset"   : ["str",  "cifar"],
+  "arch"      : ["str",  "simres"],
+  "depth"     : ["int",   5],
+  "super_type": ["str" , "basic"],
+  "zero_init_residual" : ["bool",  "0"]
+}

AutoDL-Projects/configs/archs/CIFAR-WRN28-10.config

@@ -0,0 +1,8 @@
+{
+  "dataset"   : ["str",  "cifar"],
+  "arch"      : ["str",  "wideresnet"],
+  "depth"     : ["int",  28],
+  "wide_factor":["int",  10],
+  "dropout"   : ["bool",  0],
+  "super_type": ["str" , "basic"]
+}

AutoDL-Projects/configs/archs/ImageNet-ResNet101V1.config

@@ -0,0 +1,12 @@
+{
+  "dataset"         : ["str",  "imagenet"],
+  "arch"            : ["str",  "resnet"],
+  "block_name"      : ["str",  "Bottleneck"],
+  "layers"          : ["int",  [3,4,23,3]],
+  "deep_stem"       : ["bool", 0],
+  "zero_init_residual" : ["bool",  "1"],
+  "groups"          : ["int",  1],
+  "width_per_group" : ["int", 64],
+  "norm_layer"      : ["none", "None"]
+
+}

AutoDL-Projects/configs/archs/ImageNet-ResNet101V2.config

@@ -0,0 +1,12 @@
+{
+  "dataset"         : ["str",  "imagenet"],
+  "arch"            : ["str",  "resnet"],
+  "block_name"      : ["str",  "Bottleneck"],
+  "layers"          : ["int",  [3,4,23,3]],
+  "deep_stem"       : ["bool", 1],
+  "zero_init_residual" : ["bool",  "1"],
+  "groups"          : ["int",  1],
+  "width_per_group" : ["int", 64],
+  "norm_layer"      : ["none", "None"]
+
+}

AutoDL-Projects/configs/archs/ImageNet-ResNet152V1.config

@@ -0,0 +1,11 @@
+{
+  "dataset"         : ["str",  "imagenet"],
+  "arch"            : ["str",  "resnet"],
+  "block_name"      : ["str",  "Bottleneck"],
+  "layers"          : ["int",  [3,8,36,3]],
+  "deep_stem"       : ["bool", 0],
+  "zero_init_residual" : ["bool",  "1"],
+  "groups"          : ["int",  1],
+  "width_per_group" : ["int", 64],
+  "norm_layer"      : ["none", "None"]
+}

AutoDL-Projects/configs/archs/ImageNet-ResNet152V2.config

@@ -0,0 +1,11 @@
+{
+  "dataset"         : ["str",  "imagenet"],
+  "arch"            : ["str",  "resnet"],
+  "block_name"      : ["str",  "Bottleneck"],
+  "layers"          : ["int",  [3,8,36,3]],
+  "deep_stem"       : ["bool", 1],
+  "zero_init_residual" : ["bool",  "1"],
+  "groups"          : ["int",  1],
+  "width_per_group" : ["int", 64],
+  "norm_layer"      : ["none", "None"]
+}

AutoDL-Projects/configs/archs/ImageNet-ResNet18V1.config

@@ -0,0 +1,11 @@
+{
+  "dataset"         : ["str",  "imagenet"],
+  "arch"            : ["str",  "resnet"],
+  "block_name"      : ["str",  "BasicBlock"],
+  "layers"          : ["int",  [2,2,2,2]],
+  "deep_stem"       : ["bool", 0],
+  "zero_init_residual" : ["bool",  "1"],
+  "groups"          : ["int",  1],
+  "width_per_group" : ["int", 64],
+  "norm_layer"      : ["none", "None"]
+}

AutoDL-Projects/configs/archs/ImageNet-ResNet18V2.config

@@ -0,0 +1,12 @@
+{
+  "dataset"         : ["str",  "imagenet"],
+  "arch"            : ["str",  "resnet"],
+  "block_name"      : ["str",  "BasicBlock"],
+  "layers"          : ["int",  [2,2,2,2]],
+  "deep_stem"       : ["bool", 1],
+  "zero_init_residual" : ["bool",  "1"],
+  "groups"          : ["int",  1],
+  "width_per_group" : ["int", 64],
+  "norm_layer"      : ["none", "None"]
+
+}

AutoDL-Projects/configs/archs/ImageNet-ResNet200V1.config

@@ -0,0 +1,11 @@
+{
+  "dataset"         : ["str",  "imagenet"],
+  "arch"            : ["str",  "resnet"],
+  "block_name"      : ["str",  "Bottleneck"],
+  "layers"          : ["int",  [3,24,36,3]],
+  "deep_stem"       : ["bool", 0],
+  "zero_init_residual" : ["bool",  "1"],
+  "groups"          : ["int",  1],
+  "width_per_group" : ["int", 64],
+  "norm_layer"      : ["none", "None"]
+}

AutoDL-Projects/configs/archs/ImageNet-ResNet200V2.config

@@ -0,0 +1,11 @@
+{
+  "dataset"         : ["str",  "imagenet"],
+  "arch"            : ["str",  "resnet"],
+  "block_name"      : ["str",  "Bottleneck"],
+  "layers"          : ["int",  [3,24,36,3]],
+  "deep_stem"       : ["bool", 1],
+  "zero_init_residual" : ["bool",  "1"],
+  "groups"          : ["int",  1],
+  "width_per_group" : ["int", 64],
+  "norm_layer"      : ["none", "None"]
+}

AutoDL-Projects/configs/archs/ImageNet-ResNet34V1.config

@@ -0,0 +1,12 @@
+{
+  "dataset"         : ["str",  "imagenet"],
+  "arch"            : ["str",  "resnet"],
+  "block_name"      : ["str",  "BasicBlock"],
+  "layers"          : ["int",  [3,4,6,3]],
+  "deep_stem"       : ["bool", 0],
+  "zero_init_residual" : ["bool",  "1"],
+  "groups"          : ["int",  1],
+  "width_per_group" : ["int", 64],
+  "norm_layer"      : ["none", "None"]
+
+}

AutoDL-Projects/configs/archs/ImageNet-ResNet34V2.config

@@ -0,0 +1,12 @@
+{
+  "dataset"         : ["str",  "imagenet"],
+  "arch"            : ["str",  "resnet"],
+  "block_name"      : ["str",  "BasicBlock"],
+  "layers"          : ["int",  [3,4,6,3]],
+  "deep_stem"       : ["bool", 1],
+  "zero_init_residual" : ["bool",  "1"],
+  "groups"          : ["int",  1],
+  "width_per_group" : ["int", 64],
+  "norm_layer"      : ["none", "None"]
+
+}

AutoDL-Projects/configs/archs/ImageNet-ResNet50V1.config

@@ -0,0 +1,12 @@
+{
+  "dataset"         : ["str",  "imagenet"],
+  "arch"            : ["str",  "resnet"],
+  "block_name"      : ["str",  "Bottleneck"],
+  "layers"          : ["int",  [3,4,6,3]],
+  "deep_stem"       : ["bool", 0],
+  "zero_init_residual" : ["bool",  "1"],
+  "groups"          : ["int",  1],
+  "width_per_group" : ["int", 64],
+  "norm_layer"      : ["none", "None"]
+
+}

AutoDL-Projects/configs/archs/ImageNet-ResNet50V2.config

@@ -0,0 +1,12 @@
+{
+  "dataset"         : ["str",  "imagenet"],
+  "arch"            : ["str",  "resnet"],
+  "block_name"      : ["str",  "Bottleneck"],
+  "layers"          : ["int",  [3,4,6,3]],
+  "deep_stem"       : ["bool", 1],
+  "zero_init_residual" : ["bool",  "1"],
+  "groups"          : ["int",  1],
+  "width_per_group" : ["int", 64],
+  "norm_layer"      : ["none", "None"]
+
+}

AutoDL-Projects/configs/archs/ImageNet-ResNext50-32x4dV2.config

@@ -0,0 +1,12 @@
+{
+  "dataset"         : ["str",  "imagenet"],
+  "arch"            : ["str",  "resnet"],
+  "block_name"      : ["str",  "Bottleneck"],
+  "layers"          : ["int",  [3,4,6,3]],
+  "deep_stem"       : ["bool", 1],
+  "zero_init_residual" : ["bool",  "1"],
+  "groups"          : ["int",  32],
+  "width_per_group" : ["int",  4],
+  "norm_layer"      : ["none", "None"]
+
+}

AutoDL-Projects/configs/archs/NAS-CIFAR-DARTS.config

@@ -0,0 +1,10 @@
+{
+  "arch"      : ["str",  "dxys"],
+  "genotype"  : ["str",  "DARTS"],
+  "dataset"   : ["str",  "cifar"],
+  "ichannel"  : ["int",   36],
+  "layers"    : ["int",    6],
+  "stem_multi": ["int",    3],
+  "auxiliary" : ["bool",   1],
+  "drop_path_prob": ["float", 0.2]
+}

AutoDL-Projects/configs/archs/NAS-CIFAR-GDAS_V1.config

@@ -0,0 +1,10 @@
+{
+  "arch"      : ["str",  "dxys"],
+  "genotype"  : ["str",  "GDAS_V1"],
+  "dataset"   : ["str",  "cifar"],
+  "ichannel"  : ["int",   36],
+  "layers"    : ["int",    6],
+  "stem_multi": ["int",    3],
+  "auxiliary" : ["bool",   1],
+  "drop_path_prob": ["float", 0.2]
+}

AutoDL-Projects/configs/archs/NAS-CIFAR-NAS.config

@@ -0,0 +1,10 @@
+{
+  "arch"      : ["str",  "dxys"],
+  "genotype"  : ["str",  "NASNet"],
+  "dataset"   : ["str",  "cifar"],
+  "ichannel"  : ["int",   33],
+  "layers"    : ["int",    6],
+  "stem_multi": ["int",    3],
+  "auxiliary" : ["bool",   1],
+  "drop_path_prob": ["float", 0.2]
+}

AutoDL-Projects/configs/archs/NAS-CIFAR-SETN.config

@@ -0,0 +1,10 @@
+{
+  "arch"      : ["str",  "dxys"],
+  "genotype"  : ["str",  "SETN"],
+  "dataset"   : ["str",  "cifar"],
+  "ichannel"  : ["int",   36],
+  "layers"    : ["int",    6],
+  "stem_multi": ["int",    3],
+  "auxiliary" : ["bool",   1],
+  "drop_path_prob": ["float", 0.2]
+}

AutoDL-Projects/configs/archs/NAS-CIFAR-none.config

@@ -0,0 +1,10 @@
+{
+  "super_type": ["str",  "infer-nasnet.cifar"],
+  "genotype"  : ["none", "none"],
+  "dataset"   : ["str",  "cifar"],
+  "ichannel"  : ["int",   33],
+  "layers"    : ["int",    6],
+  "stem_multi": ["int",    3],
+  "auxiliary" : ["bool",   1],
+  "drop_path_prob": ["float", 0.2]
+}

AutoDL-Projects/configs/archs/NAS-IMAGENET-DARTS_V2.config

@@ -0,0 +1,9 @@
+{
+  "arch"      : ["str",  "dxys"],
+  "genotype"  : ["str",  "DARTS_V2"],
+  "dataset"   : ["str",  "imagenet"],
+  "ichannel"  : ["int",   48],
+  "layers"    : ["int",    4],
+  "auxiliary" : ["bool",   1],
+  "drop_path_prob": ["float", 0]
+}

AutoDL-Projects/configs/archs/NAS-IMAGENET-GDAS_V1.config

@@ -0,0 +1,9 @@
+{
+  "arch"      : ["str",  "dxys"],
+  "genotype"  : ["str",  "GDAS_V1"],
+  "dataset"   : ["str",  "imagenet"],
+  "ichannel"  : ["int",   50],
+  "layers"    : ["int",    4],
+  "auxiliary" : ["bool",   1],
+  "drop_path_prob": ["float", 0]
+}

AutoDL-Projects/configs/archs/NAS-IMAGENET-SETN.config

@@ -0,0 +1,9 @@
+{
+  "arch"      : ["str",  "dxys"],
+  "genotype"  : ["str",  "SETN"],
+  "dataset"   : ["str",  "imagenet"],
+  "ichannel"  : ["int",   58],
+  "layers"    : ["int",    2],
+  "auxiliary" : ["bool",   1],
+  "drop_path_prob": ["float", 0]
+}

AutoDL-Projects/configs/archs/NAS-IMAGENET-SETN1.config

@@ -0,0 +1,9 @@
+{
+  "arch"      : ["str",  "dxys"],
+  "genotype"  : ["str",  "SETN"],
+  "dataset"   : ["str",  "imagenet"],
+  "ichannel"  : ["int",   73],
+  "layers"    : ["int",    1],
+  "auxiliary" : ["bool",   1],
+  "drop_path_prob": ["float", 0]
+}

AutoDL-Projects/configs/archs/NAS-IMAGENET-SETN2.config

@@ -0,0 +1,9 @@
+{
+  "arch"      : ["str",  "dxys"],
+  "genotype"  : ["str",  "SETN"],
+  "dataset"   : ["str",  "imagenet"],
+  "ichannel"  : ["int",   58],
+  "layers"    : ["int",    2],
+  "auxiliary" : ["bool",   1],
+  "drop_path_prob": ["float", 0]
+}

AutoDL-Projects/configs/archs/NAS-IMAGENET-SETN3.config

@@ -0,0 +1,9 @@
+{
+  "arch"      : ["str",  "dxys"],
+  "genotype"  : ["str",  "SETN"],
+  "dataset"   : ["str",  "imagenet"],
+  "ichannel"  : ["int",   49],
+  "layers"    : ["int",    3],
+  "auxiliary" : ["bool",   1],
+  "drop_path_prob": ["float", 0]
+}

AutoDL-Projects/configs/archs/NAS-IMAGENET-SETN4.config

@@ -0,0 +1,9 @@
+{
+  "arch"      : ["str",  "dxys"],
+  "genotype"  : ["str",  "SETN"],
+  "dataset"   : ["str",  "imagenet"],
+  "ichannel"  : ["int",   44],
+  "layers"    : ["int",    4],
+  "auxiliary" : ["bool",   1],
+  "drop_path_prob": ["float", 0]
+}

AutoDL-Projects/configs/archs/NAS-IMAGENET-none.config

@@ -0,0 +1,9 @@
+{
+  "super_type": ["str",  "infer-nasnet.imagenet"],
+  "genotype"  : ["none", "none"],
+  "dataset"   : ["str",  "imagenet"],
+  "ichannel"  : ["int",   50],
+  "layers"    : ["int",    4],
+  "auxiliary" : ["bool",   1],
+  "drop_path_prob": ["float", 0]
+}

AutoDL-Projects/configs/compares/CIFAR010-TAS-R110.config

@@ -0,0 +1,13 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "110"],
+  "module"             : ["str"   , "ResNetBasicblock"],
+  "super_type"         : ["str"   , "infer-shape"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "10"],
+  "search_mode"        : ["str"   , "shape"],
+  "xchannels"          : ["int"   , ["3", "16", "14", "16", "14", "16", "16", "16", "16", "16", "14", "16", "16", "16", "12", "16", "16", "16", "9", "16", "8", "16", "4", "16", "4", "4", "4", "16", "4", "4", "4", "4", "6", "6", "4", "6", "11", "4", "32", "32", "32", "32", "32", "32", "32", "32", "28", "32", "32", "28", "22", "22", "22", "32", "32", "25", "28", "9", "9", "28", "12", "9", "12", "32", "9", "9", "22", "12", "16", "9", "12", "9", "9", "9", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "38", "64", "25", "19", "19", "19", "19", "19", "25", "32", "19", "19", "25", "25", "19", "19", "38", "38", "19", "19", "51"]],
+  "xblocks"            : ["int"   , ["11", "11", "9"]],
+  "estimated_FLOP"     : ["float" , "117.498238"]
+}

AutoDL-Projects/configs/compares/CIFAR010-TAS-R56.config

@@ -0,0 +1,13 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "56"],
+  "module"             : ["str"   , "ResNetBasicblock"],
+  "super_type"         : ["str"   , "infer-shape"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "10"],
+  "search_mode"        : ["str"   , "shape"],
+  "xchannels"          : ["int"   , ["3", "16", "16", "16", "14", "14", "14", "16", "12", "16", "6", "16", "4", "4", "4", "14", "4", "4", "6", "4", "32", "32", "32", "32", "32", "32", "22", "25", "16", "32", "19", "9", "9", "16", "25", "12", "16", "9", "64", "64", "64", "64", "64", "64", "64", "64", "64", "51", "19", "19", "32", "51", "25", "32", "19", "19"]],
+  "xblocks"            : ["int"   , ["5", "5", "5"]],
+  "estimated_FLOP"     : ["float" , "57.52595"]
+}

AutoDL-Projects/configs/compares/CIFAR100-FIX-R32.config

@@ -0,0 +1,11 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "32"],
+  "module"             : ["str"   , "ResNetBasicblock"],
+  "super_type"         : ["str"   , "infer-width"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "100"],
+  "xchannels"          : ["int"   , ["3", "12", "12", "12", "12", "12", "12", "12", "12", "12", "12", "12", "25", "25", "25", "25", "25", "25", "25", "25", "25", "25", "50", "50", "50", "50", "50", "50", "50", "50", "50", "50"]],
+  "estimated_FLOP"     : ["float" , "41.095816"]
+}

AutoDL-Projects/configs/compares/CIFAR100-RAND-R32.config

@@ -0,0 +1,11 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "32"],
+  "module"             : ["str"   , "ResNetBasicblock"],
+  "super_type"         : ["str"   , "infer-width"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "100"],
+  "xchannels"          : ["int"   , ["3", "4", "11", "11", "11", "12", "14", "16", "8", "9", "6", "12", "28", "32", "28", "32", "12", "25", "28", "22", "28", "25", "57", "19", "38", "64", "64", "51", "57", "64", "64", "57"]],
+  "estimated_FLOP"     : ["float" , "42.908996"]
+}

AutoDL-Projects/configs/compares/CIFAR100-TAS-R110.config

@@ -0,0 +1,13 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "110"],
+  "module"             : ["str"   , "ResNetBasicblock"],
+  "super_type"         : ["str"   , "infer-shape"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "100"],
+  "search_mode"        : ["str"   , "shape"],
+  "xchannels"          : ["int"   , ["3", "16", "14", "16", "11", "14", "16", "16", "11", "16", "9", "14", "12", "16", "16", "16", "8", "16", "14", "16", "12", "4", "11", "16", "4", "4", "4", "16", "12", "4", "8", "4", "9", "4", "6", "14", "4", "4", "32", "32", "32", "32", "28", "28", "32", "32", "32", "32", "32", "28", "32", "28", "25", "32", "32", "32", "9", "9", "32", "32", "9", "25", "28", "32", "28", "9", "9", "32", "12", "12", "9", "22", "12", "9", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64", "44", "64", "57", "19", "19", "19", "19", "25", "19", "25", "19", "25", "19", "19", "25", "19", "19", "19", "25", "25", "19"]],
+  "xblocks"            : ["int"   , ["13", "9", "11"]],
+  "estimated_FLOP"     : ["float" , "117.653164"]
+}

AutoDL-Projects/configs/compares/CIFAR100-TAS-R32.config

@@ -0,0 +1,11 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "32"],
+  "module"             : ["str"   , "ResNetBasicblock"],
+  "super_type"         : ["str"   , "infer-width"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "100"],
+  "xchannels"          : ["int"   , ["3", "16", "4", "4", "4", "14", "6", "4", "8", "4", "4", "4", "32", "32", "9", "28", "28", "28", "28", "28", "32", "32", "64", "64", "64", "64", "64", "64", "64", "64", "64", "64"]],
+  "estimated_FLOP"     : ["float" , "42.493184"]
+}

AutoDL-Projects/configs/compares/CIFAR100-TAS-R56.config

@@ -0,0 +1,13 @@
+{
+  "dataset"            : ["str"   , "cifar"],
+  "arch"               : ["str"   , "resnet"],
+  "depth"              : ["int"   , "56"],
+  "module"             : ["str"   , "ResNetBasicblock"],
+  "super_type"         : ["str"   , "infer-shape"],
+  "zero_init_residual" : ["bool"  , "0"],
+  "class_num"          : ["int"   , "100"],
+  "search_mode"        : ["str"   , "shape"],
+  "xchannels"          : ["int"   , ["3", "16", "16", "9", "14", "16", "14", "16", "8", "16", "8", "14", "4", "4", "4", "8", "4", "6", "4", "4", "32", "32", "32", "28", "32", "32", "32", "22", "32", "32", "32", "9", "25", "19", "25", "12", "9", "9", "64", "64", "64", "64", "64", "64", "64", "64", "64", "51", "19", "19", "19", "19", "25", "38", "19", "19"]],
+  "xblocks"            : ["int"   , ["5", "5", "7"]],
+  "estimated_FLOP"     : ["float" , "59.472556"]
+}

AutoDL-Projects/configs/nas-benchmark/CIFAR.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "200"],
+  "warmup"   : ["int",   "0"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.1"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "batch_size": ["int", "256"]
+}

AutoDL-Projects/configs/nas-benchmark/ImageNet-16.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "200"],
+  "warmup"   : ["int",   "0"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.1"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "batch_size": ["int", "256"]
+}

AutoDL-Projects/configs/nas-benchmark/LESS.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "12"],
+  "warmup"   : ["int",   "0"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.1"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "batch_size": ["int", "256"]
+}

AutoDL-Projects/configs/nas-benchmark/algos/DARTS.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "LR"       : ["float", "0.025"],
+  "eta_min"  : ["float", "0.001"],
+  "epochs"   : ["int",   "50"],
+  "warmup"   : ["int",   "0"],
+  "optim"    : ["str",   "SGD"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "batch_size": ["int",  "64"]
+}

AutoDL-Projects/configs/nas-benchmark/algos/ENAS.config

@@ -0,0 +1,17 @@
+{
+  "scheduler": ["str",   "cos"],
+  "LR"       : ["float", "0.05"],
+  "eta_min"  : ["float", "0.0005"],
+  "epochs"   : ["int",   "250"],
+  "T_max"    : ["int",   "10"],
+  "warmup"   : ["int",   "0"],
+  "optim"    : ["str",   "SGD"],
+  "decay"    : ["float", "0.00025"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "controller_lr"   : ["float", "0.001"],
+  "controller_betas": ["float", [0, 0.999]],
+  "controller_eps"  : ["float", 0.001],
+  "criterion": ["str",   "Softmax"],
+  "batch_size": ["int",  "128"]
+}

AutoDL-Projects/configs/nas-benchmark/algos/GDAS.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "LR"       : ["float", "0.025"],
+  "eta_min"  : ["float", "0.001"],
+  "epochs"   : ["int",   "250"],
+  "warmup"   : ["int",   "0"],
+  "optim"    : ["str",   "SGD"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "batch_size": ["int",  "64"]
+}

AutoDL-Projects/configs/nas-benchmark/algos/R-EA.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.001"],
+  "epochs"   : ["int",   "25"],
+  "warmup"   : ["int",   "0"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.025"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "batch_size": ["int",  "64"]
+}

AutoDL-Projects/configs/nas-benchmark/algos/RANDOM.config

@@ -0,0 +1,14 @@
+{
+  "scheduler": ["str",   "cos"],
+  "LR"       : ["float", "0.025"],
+  "eta_min"  : ["float", "0.001"],
+  "epochs"   : ["int",   "250"],
+  "warmup"   : ["int",   "0"],
+  "optim"    : ["str",   "SGD"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "batch_size": ["int",  "64"],
+  "test_batch_size": ["int",  "512"]
+}

AutoDL-Projects/configs/nas-benchmark/algos/SETN.config

@@ -0,0 +1,14 @@
+{
+  "scheduler": ["str",   "cos"],
+  "LR"       : ["float", "0.025"],
+  "eta_min"  : ["float", "0.001"],
+  "epochs"   : ["int",   "250"],
+  "warmup"   : ["int",   "0"],
+  "optim"    : ["str",   "SGD"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "batch_size": ["int",  "64"],
+  "test_batch_size": ["int",  "512"]
+}

AutoDL-Projects/configs/nas-benchmark/algos/weight-sharing.config

@@ -0,0 +1,14 @@
+{
+  "scheduler": ["str",   "cos"],
+  "LR"       : ["float", "0.025"],
+  "eta_min"  : ["float", "0.001"],
+  "epochs"   : ["int",   "100"],
+  "warmup"   : ["int",   "0"],
+  "optim"    : ["str",   "SGD"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "batch_size": ["int",  "64"],
+  "test_batch_size": ["int",  "512"]
+}

AutoDL-Projects/configs/nas-benchmark/hyper-opts/01E.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "1"],
+  "warmup"   : ["int",   "0"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.1"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "batch_size": ["int", "256"]
+}

AutoDL-Projects/configs/nas-benchmark/hyper-opts/12E.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "12"],
+  "warmup"   : ["int",   "0"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.1"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "batch_size": ["int", "256"]
+}

AutoDL-Projects/configs/nas-benchmark/hyper-opts/200E.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "200"],
+  "warmup"   : ["int",   "0"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.1"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "batch_size": ["int", "256"]
+}

AutoDL-Projects/configs/nas-benchmark/hyper-opts/90E.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "90"],
+  "warmup"   : ["int",   "0"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.1"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "batch_size": ["int", "256"]
+}

AutoDL-Projects/configs/opts/CIFAR-E300-W5-L1-COS.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "300"],
+  "warmup"   : ["int",   "5"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.1"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "auxiliary": ["float", "-1"]
+}

AutoDL-Projects/configs/opts/CIFAR-E300-W5-L4-COS.config

@@ -0,0 +1,12 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "300"],
+  "warmup"   : ["int",   "5"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.4"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"]
+}

AutoDL-Projects/configs/opts/CIFAR-E600-W5-L1-COS.config

@@ -0,0 +1,12 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "600"],
+  "warmup"   : ["int",   "5"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.1"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"]
+}

AutoDL-Projects/configs/opts/CIFAR-Fast-Random.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "100"],
+  "warmup"   : ["int",   "5"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.1"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "auxiliary": ["float", "-1"]
+}

AutoDL-Projects/configs/opts/CIFAR-Slow-Random.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "300"],
+  "warmup"   : ["int",   "5"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.1"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "auxiliary": ["float", "-1"]
+}

AutoDL-Projects/configs/opts/Com-Paddle-NAS.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "595"],
+  "warmup"   : ["int",   "5"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.025"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "auxiliary": ["float", "0.4"]
+}

AutoDL-Projects/configs/opts/Com-Paddle-RES.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "300"],
+  "warmup"   : ["int",   "5"]  ,
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.1"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool" , "1"]  ,
+  "criterion": ["str"  , "Softmax"],
+  "auxiliary": ["float", "-1"]
+}

AutoDL-Projects/configs/opts/ImageNet-E120-Cos-Smooth.config

@@ -0,0 +1,14 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "120"],
+  "warmup"   : ["int",   "5"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.1"],
+  "decay"    : ["float", "0.0001"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "SmoothSoftmax"],
+  "label_smooth": ["float", 0.1],
+  "auxiliary": ["float", "-1"]
+}

AutoDL-Projects/configs/opts/ImageNet-E120-Cos-Soft.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "120"],
+  "warmup"   : ["int",   "5"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.1"],
+  "decay"    : ["float", "0.0001"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "auxiliary": ["float", "-1"]
+}

AutoDL-Projects/configs/opts/ImageNet-E120-Step-Soft.config

@@ -0,0 +1,14 @@
+{
+  "scheduler": ["str",   "multistep"],
+  "epochs"   : ["int",   "120"],
+  "warmup"   : ["int",   "5"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.1"],
+  "milestones":["int",   [30, 60, 90]],
+  "gammas"   : ["float", [0.1, 0.1, 0.1]],
+  "decay"    : ["float", "0.0001"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "auxiliary": ["float", "-1"]
+}

AutoDL-Projects/configs/opts/NAS-CIFAR-V2.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "595"],
+  "warmup"   : ["int",   "5"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.025"],
+  "decay"    : ["float", "0.0003"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "auxiliary": ["float", "0.4"]
+}

AutoDL-Projects/configs/opts/NAS-CIFAR.config

@@ -0,0 +1,13 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "295"],
+  "warmup"   : ["int",   "5"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.025"],
+  "decay"    : ["float", "0.0005"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "Softmax"],
+  "auxiliary": ["float", "0.4"]
+}

AutoDL-Projects/configs/opts/NAS-IMAGENET.config

@@ -0,0 +1,14 @@
+{
+  "scheduler": ["str",   "cos"],
+  "eta_min"  : ["float", "0.0"],
+  "epochs"   : ["int",   "245"],
+  "warmup"   : ["int",   "5"],
+  "optim"    : ["str",   "SGD"],
+  "LR"       : ["float", "0.1"],
+  "decay"    : ["float", "0.00003"],
+  "momentum" : ["float", "0.9"],
+  "nesterov" : ["bool",  "1"],
+  "criterion": ["str",   "SmoothSoftmax"],
+  "label_smooth": ["float", 0.1],
+  "auxiliary"   : ["float", "0.4"]
+}

AutoDL-Projects/configs/qlib/workflow_config_TabNet_Alpha360.yaml

@@ -0,0 +1,78 @@
+qlib_init:
+    provider_uri: "~/.qlib/qlib_data/cn_data"
+    region: cn
+market: &market all
+benchmark: &benchmark SH000300
+data_handler_config: &data_handler_config
+    start_time: 2008-01-01
+    end_time: 2020-08-01
+    fit_start_time: 2008-01-01
+    fit_end_time: 2014-12-31
+    instruments: *market
+    infer_processors:
+        - class: RobustZScoreNorm
+          kwargs:
+              fields_group: feature
+              clip_outlier: true
+        - class: Fillna
+          kwargs:
+              fields_group: feature
+    learn_processors:
+        - class: DropnaLabel
+        - class: CSRankNorm
+          kwargs:
+              fields_group: label
+    label: ["Ref($close, -2) / Ref($close, -1) - 1"]
+port_analysis_config: &port_analysis_config
+    strategy:
+        class: TopkDropoutStrategy
+        module_path: qlib.contrib.strategy.strategy
+        kwargs:
+            topk: 50
+            n_drop: 5
+    backtest:
+        verbose: False
+        limit_threshold: 0.095
+        account: 100000000
+        benchmark: *benchmark
+        deal_price: close
+        open_cost: 0.0005
+        close_cost: 0.0015
+        min_cost: 5
+task:
+    model:
+        class: TabnetModel
+        module_path: qlib.contrib.model.pytorch_tabnet
+        kwargs:
+            d_feat: 360
+            pretrain: True
+    dataset:
+        class: DatasetH
+        module_path: qlib.data.dataset
+        kwargs:
+            handler:
+                class: Alpha360
+                module_path: qlib.contrib.data.handler
+                kwargs: *data_handler_config
+            segments:
+                pretrain: [2008-01-01, 2014-12-31]
+                pretrain_validation: [2015-01-01, 2016-12-31]
+                train: [2008-01-01, 2014-12-31]
+                valid: [2015-01-01, 2016-12-31]
+                test: [2017-01-01, 2020-08-01]
+    record: 
+        - class: SignalRecord
+          module_path: qlib.workflow.record_temp
+          kwargs: {}
+        - class: SignalMseRecord
+          module_path: qlib.contrib.workflow.record_temp
+          kwargs: {}
+        - class: SigAnaRecord
+          module_path: qlib.workflow.record_temp
+          kwargs: 
+            ana_long_short: False
+            ann_scaler: 252
+        - class: PortAnaRecord
+          module_path: qlib.workflow.record_temp
+          kwargs: 
+            config: *port_analysis_config

AutoDL-Projects/configs/qlib/workflow_config_alstm_Alpha360.yaml

@@ -0,0 +1,86 @@
+qlib_init:
+    provider_uri: "~/.qlib/qlib_data/cn_data"
+    region: cn
+market: &market all
+benchmark: &benchmark SH000300
+data_handler_config: &data_handler_config
+    start_time: 2008-01-01
+    end_time: 2020-08-01
+    fit_start_time: 2008-01-01
+    fit_end_time: 2014-12-31
+    instruments: *market
+    infer_processors:
+        - class: RobustZScoreNorm
+          kwargs:
+              fields_group: feature
+              clip_outlier: true
+        - class: Fillna
+          kwargs:
+              fields_group: feature
+    learn_processors:
+        - class: DropnaLabel
+        - class: CSRankNorm
+          kwargs:
+              fields_group: label
+    label: ["Ref($close, -2) / Ref($close, -1) - 1"]
+port_analysis_config: &port_analysis_config
+    strategy:
+        class: TopkDropoutStrategy
+        module_path: qlib.contrib.strategy.strategy
+        kwargs:
+            topk: 50
+            n_drop: 5
+    backtest:
+        verbose: False
+        limit_threshold: 0.095
+        account: 100000000
+        benchmark: *benchmark
+        deal_price: close
+        open_cost: 0.0005
+        close_cost: 0.0015
+        min_cost: 5
+task:
+    model:
+        class: ALSTM
+        module_path: qlib.contrib.model.pytorch_alstm
+        kwargs:
+            d_feat: 6
+            hidden_size: 64
+            num_layers: 2
+            dropout: 0.0
+            n_epochs: 200
+            lr: 1e-3
+            early_stop: 20
+            batch_size: 800
+            metric: loss
+            loss: mse
+            GPU: 0
+            rnn_type: GRU
+    dataset:
+        class: DatasetH
+        module_path: qlib.data.dataset
+        kwargs:
+            handler:
+                class: Alpha360
+                module_path: qlib.contrib.data.handler
+                kwargs: *data_handler_config
+            segments:
+                train: [2008-01-01, 2014-12-31]
+                valid: [2015-01-01, 2016-12-31]
+                test: [2017-01-01, 2020-08-01]
+    record: 
+        - class: SignalRecord
+          module_path: qlib.workflow.record_temp
+          kwargs: {}
+        - class: SignalMseRecord
+          module_path: qlib.contrib.workflow.record_temp
+          kwargs: {}
+        - class: SigAnaRecord
+          module_path: qlib.workflow.record_temp
+          kwargs: 
+            ana_long_short: False
+            ann_scaler: 252
+        - class: PortAnaRecord
+          module_path: qlib.workflow.record_temp
+          kwargs: 
+            config: *port_analysis_config

AutoDL-Projects/configs/qlib/workflow_config_doubleensemble_Alpha360.yaml

@@ -0,0 +1,100 @@
+qlib_init:
+    provider_uri: "~/.qlib/qlib_data/cn_data"
+    region: cn
+market: &market all
+benchmark: &benchmark SH000300
+data_handler_config: &data_handler_config
+    start_time: 2008-01-01
+    end_time: 2020-08-01
+    fit_start_time: 2008-01-01
+    fit_end_time: 2014-12-31
+    instruments: *market
+    infer_processors: []
+    learn_processors:
+        - class: DropnaLabel
+        - class: CSRankNorm
+          kwargs:
+              fields_group: label
+    label: ["Ref($close, -2) / Ref($close, -1) - 1"]
+port_analysis_config: &port_analysis_config
+    strategy:
+        class: TopkDropoutStrategy
+        module_path: qlib.contrib.strategy.strategy
+        kwargs:
+            topk: 50
+            n_drop: 5
+    backtest:
+        verbose: False
+        limit_threshold: 0.095
+        account: 100000000
+        benchmark: *benchmark
+        deal_price: close
+        open_cost: 0.0005
+        close_cost: 0.0015
+        min_cost: 5
+task:
+    model:
+        class: DEnsembleModel
+        module_path: qlib.contrib.model.double_ensemble
+        kwargs:
+            base_model: "gbm"
+            loss: mse
+            num_models: 6
+            enable_sr: True
+            enable_fs: True
+            alpha1: 1
+            alpha2: 1
+            bins_sr: 10
+            bins_fs: 5
+            decay: 0.5
+            sample_ratios:
+                - 0.8
+                - 0.7
+                - 0.6
+                - 0.5
+                - 0.4
+            sub_weights:
+                - 1
+                - 0.2
+                - 0.2
+                - 0.2
+                - 0.2
+                - 0.2
+            epochs: 136
+            colsample_bytree: 0.8879
+            learning_rate: 0.0421
+            subsample: 0.8789
+            lambda_l1: 205.6999
+            lambda_l2: 580.9768
+            max_depth: 8
+            num_leaves: 210
+            num_threads: 20
+            verbosity: -1
+    dataset:
+        class: DatasetH
+        module_path: qlib.data.dataset
+        kwargs:
+            handler:
+                class: Alpha360
+                module_path: qlib.contrib.data.handler
+                kwargs: *data_handler_config
+            segments:
+                train: [2008-01-01, 2014-12-31]
+                valid: [2015-01-01, 2016-12-31]
+                test: [2017-01-01, 2020-08-01]
+    record:
+        - class: SignalRecord
+          module_path: qlib.workflow.record_temp
+          kwargs: {}
+        - class: SignalMseRecord
+          module_path: qlib.contrib.workflow.record_temp
+          kwargs: {}
+        - class: SigAnaRecord
+          module_path: qlib.workflow.record_temp
+          kwargs:
+            ana_long_short: False
+            ann_scaler: 252
+        - class: PortAnaRecord
+          module_path: qlib.workflow.record_temp
+          kwargs:
+            config: *port_analysis_config