MeCo: Zero-Cost Proxy for NAS Via Minimum Eigenvalue of Correlation on Feature Maps

Installation

Python >= 3.6
PyTorch >= 2.0.0
nas-bench-201

Preparation

1. Download three datasets (CIFAR-10, CIFAR-100, ImageNet16-120) from Google Drive, place them into the directory ./data
2. Download the data directory and save it to the root folder of this repo.
3. Download the benchmark files of NAS-Bench-201 from Google Drive , put them into the directory ./data
4. Download the NAS-Bench-101 dataset, put it into the directory ./data
5. Install zero-cost-nas

cd zero-cost-nas
pip install .
cd ..

Usage/Examples

Correlation Experiment

cd correlation
python NAS_Bench_101.py
python NAS_Bench_201.py

Experiments on NAS-Bench-201

1. Run Zero-Cost-PT with appointed zero-cost proxy:

cd exp_scripts
bash zerocostpt_nb201_pipline.sh --metric [metric] --batch_size [batch_size] --seed [seed]

You can choice metric from ['snip', 'fisher', 'synflow', 'grad_norm', 'grasp', 'jacob_cov','tenas', 'zico', 'meco']

Experiments on DARTS-CNN Space

1. DARTS CNN Space

cd exp_scripts
bash zerocostpt_darts_pipline.sh --metric [metric] --batch_size [batch_size] --seed [seed]

2. DARTS Subspaces S1-S4

cd exp_scripts
bash zerocostpt_darts_pipline.sh --metric [metric] --batch_size [batch_size] --seed [seed] --space [s1-s4]

You can run our Zero-Cost-PT with the following script:

bash zerocostpt_nb201_pipeline.sh --seed [SEED]

Reference

Our code is based on Zero-Cost-PT and Zero-Cost-NAS.