# Shiyi Scripts

Additional scripts for Vision-Language Model training, evaluation, and debugging.

## Directory Structure

```
shiyi_scripts/
├── README.md                      # This file
├── j63/                           # J63 dataset utilities
├── Training Scripts
│   ├── train_sharegpt4v_vlm.sh
│   ├── train_sharegpt4v_vlm_multinode.sh
│   ├── train_llava80k_vlm.sh
│   └── train_dvlm_qwen2_5_3B_llava.sbatch
├── Evaluation Scripts
│   ├── eval_qwen2_5_vl_3B_lm_eval.py
│   ├── eval_qwen2_5_vl_text_only.py
│   ├── eval_dvlm_batch.py
│   └── eval_qwen2_5_vl_3B_*.sh
├── Debug Scripts
│   ├── train_debug_gpu_1iter.sh
│   ├── train_debug_cpu_1iter.sh
│   └── debug_*.py
└── Utilities
    ├── test_vila_*.py
    ├── inspect_webdataset.py
    └── count_vila_mixture.py
```

---

## Training Scripts

### ShareGPT4V Training

**Single Node:**
```bash
bash shiyi_scripts/train_sharegpt4v_vlm.sh
```

**Multi-Node (Slurm):**
```bash
sbatch shiyi_scripts/train_sharegpt4v_vlm_multinode.sh
```

### LLaVA Training

```bash
bash shiyi_scripts/train_llava80k_vlm.sh
```

### Debug Training (1 iteration)

For quick testing of training pipeline:

**GPU:**
```bash
bash shiyi_scripts/train_debug_gpu_1iter.sh
```

**CPU:**
```bash
bash shiyi_scripts/train_debug_cpu_1iter.sh
```

---

## Evaluation Scripts

### Text-Only Evaluation

Evaluate model on text-only benchmarks:

```bash
python shiyi_scripts/eval_qwen2_5_vl_text_only.py \
  --model_path output_models/checkpoint \
  --tasks hellaswag,arc_easy,arc_challenge
```

### Batch Evaluation

```bash
python shiyi_scripts/eval_dvlm_batch.py \
  --model_path output_models/checkpoint \
  --output_dir eval_results
```

### 8-GPU Evaluation

```bash
bash shiyi_scripts/eval_qwen2_5_vl_3B_full_8gpu.sh
bash shiyi_scripts/eval_qwen2_5_vl_3B_text_only_8gpu.sh
```

---

## Dataset Utilities

### VILA Mixture Testing

```bash
# Test VILA dataset loading
python shiyi_scripts/test_vila_mixture.py

# Test VILA dataloader
python shiyi_scripts/test_vila_dataloader.py

# Count samples in VILA mixture
python shiyi_scripts/count_vila_mixture.py
```

### WebDataset Inspection

```bash
python shiyi_scripts/inspect_webdataset.py \
  --shard /path/to/shard.tar
```

---

## Debug Scripts

### CPU Debug

Test model loading and basic functionality on CPU:

```bash
python shiyi_scripts/debug_eval_cpu.py
```

### Dataset Debug

```bash
python shiyi_scripts/debug_j63_datasets.py
```

---

## Reports

| File | Description |
|------|-------------|
| `TRAINING_STATUS_REPORT.md` | Training progress and status |
| `EVAL_PIPELINE_FINAL_REPORT.md` | Evaluation pipeline documentation |
| `CPU_DEBUG_SUMMARY.md` | CPU debugging summary |
| `README_DVLM_EVAL.md` | dVLM evaluation guide |

---

## Interactive GPU Session

Request interactive GPU session on Slurm:

```bash
sbatch shiyi_scripts/interactive_gpu.slurm
```

---

## GPU Random Activator

Utility to keep GPU active during idle periods:

```bash
python shiyi_scripts/gpu_random_activator.py
```

---

## Relationship to chengyue_script

This directory contains experimental and development scripts. For production use, prefer the organized scripts in `chengyue_script/dvlm/`:

| Task | shiyi_scripts | chengyue_script/dvlm |
|------|---------------|----------------------|
| Training | Experimental | Production |
| Evaluation | Basic | Full pipeline |
| Documentation | Notes | Comprehensive |

---

## Environment

Requires the `dvlm-shiyi` conda environment (Python 3.10):

```bash
conda create -n dvlm-shiyi python=3.10 -y
conda activate dvlm-shiyi
pip install -r requirements.txt
pip install -e .
```

