YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

Indic Heritage Studio v2

A multimodal content creation tool that reimagines modern photos and prompts through the lens of Indian heritage art forms — running on SDXL + Stable Video Diffusion + ControlNet + per-style LoRAs across 8 × NVIDIA 80GB GPUs (dev) and AMD Radeon Cloud (final demo).

Submission for the AMD AI DevMaster Hackathon (July 15 – August 6, 2026) Track 1: Multimodal AI — Development of Multimodal Content Creation Tools Team: TeamIndicForge

🎯 What It Does (v2 — full rebuild)

Indic Heritage Studio v2 transforms everyday inputs into culturally-rooted visual art. Given a text prompt, a reference photo, or both, it produces:

Text → Heritage-styled image — SDXL 1.0 (DreamShaper-XL turbo) at 1024² with per-style LoRA overlay
Image → Stylized image — IP-Adapter XL on SDXL with optional multi-style blending (60% Madhubani + 40% Warli)
Image → 4-second video — Stable Video Diffusion XT 1.1 (25 frames @ 8 fps, 1024×576)
ControlNet composition — Canny / Depth / OpenPose conditioning for precise composition control
Inpainting — mask a region, restyle it in any heritage style (heritage restoration use case)
Batch processing — multi-GPU data parallelism across 4 worker GPUs (~4× speedup)

Each output is styled after one of five authentic Indian art traditions, each with its own fine-tuned LoRA trained on real heritage art samples:

🟢 Madhubani — Bihar folk art, geometric patterns, natural pigments
🟤 Warli — Maharashtra tribal art, white-on-earth, stick figures
🔵 Pattachitra — Odisha cloth painting, mythological narratives
🟡 Mughal Miniature — court painting, fine detail, gold leaf
🔴 Tanjore — Tamil Nadu, gold-gilded devotional icons

An optional AI Style Advisor (powered by AMD's free Qwen API) helps users pick a style and refine prompts — but the core generation runs 100% on AMD Radeon GPU in compliance with Track 1 rules.

🆕 What's New in v2

Axis	v1	v2
T2I model	SD 1.5 (DreamShaper)	SDXL 1.0 + DreamShaper-XL turbo
Resolution	512×512	1024×1024 (1536² for showcase)
Style conditioning	Prompt tags only	Per-style LoRA fine-tune + IP-Adapter XL
Image → video	AnimateDiff 16f	Stable Video Diffusion XT 1.1, 25 frames
Composition control	❌	ControlNet (Canny/Depth/OpenPose)
Image repair	❌	SDXL inpainting
Batch	Single GPU	Multi-GPU data parallel, 4 workers
Pipelines	Load/unload on demand	All 4 resident on dedicated GPUs

🏗️ System Architecture (Multi-GPU)

┌─────────────────────────────────────────────────────────────────┐
│                       USER INTERFACE (Gradio)                    │
│  6 tabs + Style Advisor widget + GPU monitor sidebar            │
└──────────────────────────────────┬──────────────────────────────┘
                                   ▼
┌─────────────────────────────────────────────────────────────────┐
│              AGENT LAYER (free AMD Qwen/DeepSeek API)            │
│  StyleAdvisor · PromptEngineer · Critic                         │
└──────────────────────────────────┬──────────────────────────────┘
                                   ▼
┌─────────────────────────────────────────────────────────────────┐
│        CORE GPU LAYER — MULTI-GPU (8 × 80 GB dev)                │
│  GPU 0: SDXL T2I + Inpainting    GPU 4: Batch worker 0          │
│  GPU 1: SDXL + IP-Adapter XL     GPU 5: Batch worker 1          │
│  GPU 2: SVD-XT 1.1 (I2V)         GPU 6: Batch worker 2          │
│  GPU 3: SDXL + ControlNet        GPU 7: Batch worker 3          │
│  Per-style LoRAs (~150 MB each) loaded on demand                │
└─────────────────────────────────────────────────────────────────┘

On AMD Radeon Cloud (single GPU), all pipelines share GPU 0 and load/unload on demand. The code auto-detects device count.

🛠️ Tech Stack

Layer	Tech
Dev hardware	8 × NVIDIA A100/H100 80GB (640 GB total VRAM)
Final demo hardware	AMD Radeon Cloud (Radeon GPU, ~16 GB VRAM)
Software stack	ROCm 6.2 (AMD) / CUDA 12.1 (NVIDIA dev)
Deep learning	PyTorch 2.4.1 + Diffusers 0.30 + Accelerate 0.34 + PEFT 0.12
T2I model	SDXL 1.0 + DreamShaper-XL turbo
Style transfer	IP-Adapter XL (`h94/IP-Adapter`)
Image-to-video	Stable Video Diffusion XT 1.1 (`stabilityai/stable-video-diffusion-img2vid-xt-1-1`)
Composition control	ControlNet (Canny/Depth/OpenPose SDXL variants)
Per-style LoRAs	Trained with PEFT, rank 32, ~800 steps each
Agent LLM	Free AMD Model APIs (Qwen3.6-35B-A3B / DeepSeek-V4-Flash)
UI	Gradio 4.x
Video post	FFmpeg + OpenCV + imageio
Image post	Pillow, imageio

📦 Installation

Step 1 — Clone

git clone https://huggingface.co/Dev2506/indic-heritage-studio
cd indic-heritage-studio

Setup

python -m venv .venv source .venv/bin/activate pip install torch==2.4.1 torchvision==0.19.1 --index-url https://download.pytorch.org/whl/cu121 pip install -r requirements.txt

Configure

cp .env.example .env

Edit .env to add AMD_MODEL_API_KEY (free at https://developer.amd.com.cn/radeon/modelapis)

Launch

python app.py

Open http://localhost:7860

```

🚀 Usage

CLI mode

# Text-to-image in Madhubani style (SDXL + LoRA)
python -m core.text_to_image \
    --prompt "a young woman reading under a banyan tree" \
    --style madhubani \
    --out outputs/t2i_madhubani.png \
    --steps 25 --size 1024 --seed 42

# Style transfer (IP-Adapter XL)
python -m core.style_transfer \
    --image inputs/photo.jpg \
    --style warli \
    --out outputs/styled_warli.png \
    --strength 0.7 --ip-scale 0.7

# Image-to-video (SVD)
python -m core.image_to_video \
    --image outputs/styled_warli.png \
    --out outputs/warli_animated.mp4 \
    --frames 25 --fps 8

# ControlNet (Canny → Mughal)
python -m core.controlnet \
    --condition-image inputs/sketch.png \
    --condition-type canny \
    --prompt "a courtly gathering" \
    --style mughal \
    --out outputs/controlnet_mughal.png

# Inpainting
python -m core.inpainting \
    --image inputs/damaged_painting.jpg \
    --mask inputs/damage_mask.png \
    --prompt "restore the floral border" \
    --style tanjore \
    --out outputs/restored.png

# Batch (multi-GPU, 4 workers)
python -m core.batch_processor \
    --mode style_transfer \
    --input-dir inputs/ \
    --output-dir outputs/batch/ \
    --workers 4

Web UI

python app.py

The UI exposes all 6 modes plus the Style Advisor widget and GPU monitor.

⚡ Multi-GPU Optimization

The codebase applies these optimizations:

Pipeline-to-GPU pinning — T2I on GPU 0, IP-Adapter on GPU 1, SVD on GPU 2, ControlNet on GPU 3. No thrash.
Batch data parallelism — Inputs sharded across GPUs 4-7 via multiprocessing.spawn.
Per-style LoRA hot-swap — unload_lora_weights → load_lora_weights in ~3 seconds (vs 30+ sec full pipeline reload).
FP16 inference — torch_dtype=torch.float16 everywhere.
SDPA attention — built into PyTorch 2.0+, replaces xformers (CUDA-only).
Attention + VAE slicing — reduces VRAM peak ~30%.
torch.inference_mode() — all hot paths.
Pipeline singleton reuse — load once, serve many requests.

Multi-GPU Benchmark

python scripts/benchmark.py --configs 1 2 4 8 --samples 4

Reports: latency per image, throughput (img/min), peak VRAM, scaling chart.

📊 Evaluation Rubric Coverage

Rubric item	Points	How v2 hits it
Complete input→processing→output workflow	40	6 pipelines: T2I, I2I-style, I2V, ControlNet, Inpaint, batch
Innovative creation scenarios	20	Indian heritage art × LoRA fine-tuning × ControlNet × SVD = unique vertical
Practical application & social value	20	Cultural preservation + heritage restoration + accessible creator tool
Clear, stable, diverse output on Radeon GPU	20	6 output modes × 5 art styles × LoRA = 30+ high-quality demo outputs

🗂️ Project Structure

indic-heritage-studio-v2/
├── README.md                       # This file
├── app.py                          # Gradio UI entry point
├── requirements.txt                # Python dependencies
├── .env.example                    # Environment template
├── .gitignore
├── config/
│   ├── settings.py                 # v2 global config (SDXL, multi-GPU, LoRA paths)
│   └── styles.py                   # 5 heritage styles + LoRA scales + motion tuning
├── agents/                         # LLM agent layer (free AMD API)
│   ├── base.py
│   ├── style_advisor.py
│   ├── prompt_engineer.py          # SDXL-aware prompt enrichment
│   └── critic.py                   # Heuristic + LLM critique
├── core/                           # GPU layer
│   ├── text_to_image.py            # SDXL + LoRA + optional refiner
│   ├── style_transfer.py           # IP-Adapter XL + multi-style blending
│   ├── image_to_video.py           # Stable Video Diffusion XT 1.1
│   ├── controlnet.py               # Canny / Depth / OpenPose
│   ├── inpainting.py               # SDXL inpainting + LoRA
│   └── batch_processor.py          # Multi-GPU data parallel
├── training/                       # NEW — LoRA training
│   ├── prepare_dataset.py          # Heritage art dataset prep
│   └── train_lora.py               # PEFT LoRA on SDXL UNet
├── ui/
│   └── gradio_app.py               # 6-tab web UI
├── utils/
│   ├── gpu_utils.py                # Multi-GPU device management
│   ├── image_utils.py
│   └── video_utils.py
├── scripts/
│   ├── benchmark.py                # Multi-GPU scaling benchmark
│   ├── download_models.py          # All v2 models (~35 GB)
│   ├── generate_demo_outputs.py    # Pre-bake demo gallery
│   ├── verify_rocm.py              # Env health check
│   ├── day1_setup.sh               # Day 1 environment setup
│   └── amd_demo_recording.sh       # AMD demo recording script
├── assets/
│   ├── styles/                     # IP-Adapter reference images (5)
│   ├── datasets/                   # LoRA training datasets
│   └── loras/                      # Trained LoRA weights (5 × ~150 MB)
├── docs/
│   ├── architecture.md             # Detailed v2 architecture
│   ├── 2_week_plan.md              # Replan for 8×80GB dev
│   ├── local_dev_setup.md          # NVIDIA CUDA setup
│   └── architecture_diagram.png
├── examples/                       # Demo gallery (pre-baked)
└── tests/                          # Smoke tests

📅 Build Schedule (2-week plan)

See docs/2_week_plan.md for the day-by-day plan. Summary:

Week 1 (Jul 16–22): Env setup, source heritage art datasets, train 5 LoRAs, smoke-test all 6 pipelines.
Week 2 (Jul 23–29): Pre-bake demo gallery, run multi-GPU benchmark, write PDF + PPT, code freeze.
Week 3 (Jul 30–Aug 5): AMD verification + demo recording + submit. Burns ~1.5 of 10 AMD credits.

📝 Submission Package

The 4 required deliverables (Track 1):

Project Profile Document (PDF) — generated via the pdf skill in Week 2
Source Code (this repo) — fork of AMD-DEV-CONTEST/Radeon-hackathon-2026-07
Demo Video (3–5 min) — rocm-smi → CLI → live GPU inference → UI walkthrough
PPT / Poster — generated via the pptx skill in Week 2

PR title format: Track 1, TeamIndicForge, Indic Heritage Studio

🆘 Troubleshooting

Issue	Fix
`torch.cuda.is_available() == False`	Reinstall PyTorch with `--index-url https://download.pytorch.org/whl/cu121`
LoRA training OOMs	Lower `--rank` from 32 → 16, or `--batch-size` from 1 → 1 with `--grad-accum 8`
SVD OOM on AMD	Use `--frames 14` (instead of 25)
ControlNet preprocessor download fails	`python -c "from controlnet_aux import CannyDetector; CannyDetector.from_pretrained('lllyasviel/Annotators')"`
Multi-GPU batch fails to spawn	Set `MULTI_GPU=0` in `.env`
HuggingFace download slow	`HF_HUB_ENABLE_HF_TRANSFER=1` (already in `.env`)
Qwen API 429 errors	Agent layer is optional; core tool still works

📧 Contact

Hackathon email: ai_dev_contests@amd.com
Discord: https://discord.gg/zt9caur5B3
Team: TeamIndicForg

📜 License

MIT for project code. Model checkpoints retain their original licenses (see HuggingFace model cards).

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support