AI & ML interests
AI models
Recent Activity
Aspaklarya - Qwen3-VL-8B for remote-sensing VQA + captioning
LoRA finetunes of Qwen3-VL-8B-Instruct, trained jointly on visual question answering and image captioning over overhead imagery. All model repos are private.
Two experiment rounds, run against two different answer styles:
| exp0XX | exp1XX | |
|---|---|---|
| supervision | VQA column, ~4-character answers (yes, 3, river) | VQA_1126 column, ~300-character grounded answers |
| benchmark | 1,000-image union draw, 2,000 graded items | four-slice seed-1105 draw, 1,991 graded items |
| best | Aspaklarya-020-step4500 | Aspaklarya-101-step4500 |
The two rounds are not comparable to each other - the benchmarks reward opposite answer styles. Within a round, every number is directly comparable.
Scoring is LLM-as-judge (gemma-4-31b-it-imiai), 0-100 per item. DM = the in-domain overhead-imagery subset; RSICD = the public remote-sensing benchmark subset. Two references appear in every table: an in-house reference model and the untuned base.
2. Experiment parameters
2.1 exp0XX
Constant everywhere: effective batch 32 (except 014 at 16, 015-017 at 64), warmup 0.03, grad clip 0.3 (1.0 for 011/012), seed 3407, 4 epochs. 4-bit base for 001-017, bf16 from 018 on.
| Model | r / alpha | Drop | LR | Sched | Vision | Per image | Items | What it tests |
|---|---|---|---|---|---|---|---|---|
| Aspaklarya-001 | 16/16 | 0.0 | 2e-4 | linear | trained | 1c+2q | 18,864 | reference recipe |
| Aspaklarya-001 (ep5) | 16/16 | 0.0 | 2e-4 | linear | trained | 1c+2q | 18,864 | one epoch past budget |
| Aspaklarya-002 | 32/64 | 0.05 | 1e-4 | cosine | trained | 1c+2q | 18,864 | higher rank, lower LR |
| Aspaklarya-003 | 16/16 | 0.0 | 2e-4 | linear | trained | 2c+4q | 37,728 | 2x resampled data |
| Aspaklarya-004 | 16/16 | 0.0 | 2e-4 | linear | frozen | 1c+2q | 18,864 | language-only |
| Aspaklarya-005 | 16/16 | 0.0 | 2e-4 | cosine | trained | 1c+6q | 43,864 | VQA-heavy |
| Aspaklarya-006 | 16/16 | 0.0 | 2e-4 | linear | trained | 1c+2q | 18,864 | batch 4 x accum 8 |
| Aspaklarya-007 | 8/8 | 0.0 | 2e-4 | linear | trained | 1c+2q | 18,864 | low rank |
| Aspaklarya-008 | 16/16 | 0.0 | 1e-4 | linear | trained | 1c+4q | 31,440 | more QA, lower LR |
| Aspaklarya-009 | 32/32 | 0.0 | 2e-4 | linear | trained | 1c+2q | 18,864 | high rank |
| Aspaklarya-010 | 16/16 | 0.0 | 2e-4 | linear | trained | 1c+2q | 18,864 | wider prompt pool |
| Aspaklarya-011 | 16/16 | 0.05 | 2e-4 | linear | trained | 1c+2q | 18,864 | dropout, wd 0.05, clip 1.0 |
| Aspaklarya-012 | 16/16 | 0.0 | 2e-4 | linear | trained | 1c+2q | 18,864 | as 011, no dropout |
| Aspaklarya-013 | 16/16 | 0.0 | 1e-4 | linear | trained | 1c+2q | 18,864 | low LR, long schedule |
| Aspaklarya-014 | 16/16 | 0.0 | 2e-4 | linear | trained | 1c+2q | 18,864 | effective batch 16 |
| Aspaklarya-015 | 16/16 | 0.0 | 2e-4 | linear | trained | 1c+2q | 18,864 | effective batch 64 |
| Aspaklarya-016 | 16/16 | 0.0 | 4e-4 | linear | trained | 1c+2q | 18,864 | batch 64 + linear LR scaling - stopped early, not scored |
| Aspaklarya-017 | 16/16 | 0.0 | 3e-4 | linear | trained | 1c+2q | 18,864 | batch 64 + sqrt LR scaling |
| Aspaklarya-018 | 16/16 | 0.0 | 2e-4 | linear | trained | 1c+2q | 18,864 | bf16 base |
| Aspaklarya-019 | 16/16 | 0.05 | 2e-4 | linear | trained | 1c+2q | 18,864 | bf16 base + 011 recipe |
| Aspaklarya-020 | 16/16 | 0.0 | 2e-4 | linear | trained | 1c+2q | 35,981 | bf16 + +91% images |
| Aspaklarya-021 | 16/16 | 0.0 | 2e-4 | linear | trained | 1c+2q | 35,981 | as 020, 8 epochs |
| in-house reference | - | - | - | - | - | - | - | reference |
| Qwen3-VL-8B-Instruct | - | - | - | - | - | - | - | untuned base |
2.2 exp1XX
Constant across the round: dropout 0, bf16 base, effective batch 32 (4x8), warmup 0.03, weight decay 0.001, grad clip 0.3, max side 1024, seq 2048, seed 3407 unless stated, LoRA on all linear layers of both towers, VQA_1126 answer column.
| Model | Phase | r / alpha | LR | Sched | Vision | Per image | Ep | Items | What it tests |
|---|---|---|---|---|---|---|---|---|---|
| Aspaklarya-101 | full | 16/16 | 2e-4 | linear | trained | 1c+2q | 4 | 35,979 | reference recipe, 1.9x data |
| Aspaklarya-102 | full | 16/16 | 2e-4 | linear | trained | 1c+2q | 4 | 18,862 | train split only, half the images |
| Aspaklarya-103 | full | 32/32 | 2e-4 | linear | trained | 1c+2q | 4 | 18,862 | rank 32 |
| Aspaklarya-104 | full | 16/32 | 2e-4 | linear | trained | 1c+2q | 4 | 18,862 | scale 1.0 -> 2.0 |
| Aspaklarya-105 | full | 16/16 | 2e-4 | linear | trained | 1c+2q | 4 | 18,862 | caption loss weight 0.4 |
| Aspaklarya-106 | full | 64/64 | - | - | - | - | - | - | LoRA soup of 103/104/105, no training |
| Aspaklarya-106b | full | 64/64 | - | - | - | - | - | - | soup, alternate mixing weights |
| Aspaklarya-107 | full | 16/16 | 1e-4 | cosine | trained | 1c+2q | 3 | 18,862 | cosine, warmup 0.10 |
| Aspaklarya-108 s1234 | full | 16/16 | 2e-4 | linear | trained | 1c+2q | 4 | 18,862 | seed 1234 of 102 |
| Aspaklarya-108 s2024 | full | 16/16 | 2e-4 | linear | trained | 1c+2q | 4 | 18,862 | seed 2024 of 102 |
| Aspaklarya-109 | full | 16/16 | 2e-4 | linear | trained | 1c+2q | 4 | 35,979 | seed 1234 of 101 |
| Aspaklarya-110 | full | 16/16 | 2e-4 | linear | trained | 1c+2q | 4 | 35,979 | seed 2024 of 101 |
| Aspaklarya-111 | full | 16/16 | 2e-4 | linear | trained | 2c+4q | 3 | 71,944 | QA depth 2 -> 4 |
| Aspaklarya-112 beam4 | probe | 16/16 | - | - | - | - | - | - | decoding only on the 101 adapter |
| Aspaklarya-112 t0.7 | probe | 16/16 | - | - | - | - | - | - | temperature 0.7 |
| Aspaklarya-112 rep | probe | 16/16 | - | - | - | - | - | - | repetition penalty 1.1 |
| Aspaklarya-113 | probe | 16/16 | 2e-4 | linear | frozen | 1c+2q | 1 | 35,979 | vision tower frozen |
| Aspaklarya-114 | probe | 16/16 | 2e-4 | linear | trained | 1c+2q | 1 | 35,979 | seq 3072 |
| Aspaklarya-114 1536 | probe | 16/16 | 2e-4 | linear | trained | 1c+2q | 1 | 35,979 | max side 1536 |
| Aspaklarya-115 | probe | 16/16 | 2e-4 | linear | trained | 1c+2q | 1 | 35,981 | paraphrased questions |
| Aspaklarya-116 | probe | 16/16 | 2e-4 | linear | trained | 1c+2q | 1 | 37,178 | +1,199 hard negatives |
| Aspaklarya-119 | probe | 16/16 | 2e-4 | linear | trained | 1c+2q | 1 | 35,980 | teacher-rewritten answers |
| Aspaklarya-120 B | probe | 16/16 | 2e-4 | linear | trained | 0c+2q | 1 | 23,986 | QA only, resumed from 120 A |
| Aspaklarya-121 vqa | routing | 16/16 | 2e-4 | linear | trained | 0c+2q | 4 | 23,986 | VQA expert alone |
| Aspaklarya-121 cap | routing | 16/16 | 2e-4 | linear | trained | 1c+0q | 4 | 11,993 | caption expert alone |
| Aspaklarya-121 routed | routing | 16/16 | 2e-4 | linear | trained | split | 4 | 35,979 | two task experts + router |
| Aspaklarya-122 s1234 | routing | 16/16 | 2e-4 | linear | trained | 0c+2q | 1 | 23,986 | replication of 120 B, seed 1234 |
| Aspaklarya-122 s2024 | routing | 16/16 | 2e-4 | linear | trained | 0c+2q | 1 | 23,986 | replication of 120 B, seed 2024 |
| Aspaklarya-123 | routing | 16/16 | 2e-4 | linear | trained | 1c+2q | 8 | 35,979 | double the epochs |
| control s3407 | probe | 16/16 | 2e-4 | linear | trained | 1c+2q | 1 | 35,979 | 1-epoch control |
| control s1234 | probe | 16/16 | 2e-4 | linear | trained | 1c+2q | 1 | 35,979 | 1-epoch control |
| control s2024 | probe | 16/16 | 2e-4 | linear | trained | 1c+2q | 1 | 35,979 | 1-epoch control |
| Aspaklarya-020 | exp0XX | 16/16 | 2e-4 | linear | trained | 1c+2q | 4 | 35,981 | cross-round reference |
| in-house reference | - | - | - | - | - | - | - | - | reference |
| Qwen3-VL-8B-Instruct | - | - | - | - | - | - | - | - | untuned base |
The numeric gap is real: 117 and 118 do not exist - both were designed, then cancelled when 114 proved the resolution axis a no-op.
3. Evaluation results
3.1 exp0XX - legacy benchmark
Best checkpoint per run on a fixed 1,000-image draw (2,000 graded items). Overall = mean of caption and VQA. Sorted by DM vqa.
| Model | DM cap | DM vqa | DM overall | RSICD cap | RSICD vqa | RSICD overall | All overall |
|---|---|---|---|---|---|---|---|
| in-house reference | 71.88 | 83.62 | 77.75 | 58.72 | 63.44 | 61.08 | 69.26 |
| Aspaklarya-011 | 81.85 | 83.51 | 82.68 | 78.06 | 68.33 | 73.20 | 77.85 |
| Aspaklarya-001 | 81.84 | 83.23 | 82.54 | 78.17 | 68.89 | 73.53 | 77.95 |
| Aspaklarya-006 | 81.40 | 82.95 | 82.18 | 78.26 | 68.88 | 73.57 | 77.80 |
| Aspaklarya-019 | 82.39 | 82.82 | 82.61 | 78.34 | 69.24 | 73.79 | 78.12 |
| Aspaklarya-015 | 82.14 | 82.76 | 82.45 | 78.07 | 66.98 | 72.52 | 77.40 |
| Aspaklarya-018 | 82.28 | 82.70 | 82.49 | 78.70 | 69.15 | 73.92 | 78.13 |
| Aspaklarya-005 | 81.37 | 82.37 | 81.87 | 77.79 | 69.35 | 73.57 | 77.64 |
| Aspaklarya-009 | 80.01 | 82.34 | 81.18 | 77.09 | 69.59 | 73.34 | 77.19 |
| Aspaklarya-001 (ep5) | 80.60 | 82.25 | 81.42 | 78.05 | 70.31 | 74.18 | 77.73 |
| Aspaklarya-013 | 80.58 | 82.20 | 81.39 | 77.70 | 68.74 | 73.22 | 77.23 |
| Aspaklarya-003 | 81.23 | 82.18 | 81.71 | 77.92 | 69.10 | 73.51 | 77.53 |
| Aspaklarya-020 | 82.74 | 82.10 | 82.42 | 79.93 | 70.28 | 75.10 | 78.70 |
| Aspaklarya-012 | 81.68 | 82.05 | 81.86 | 77.85 | 68.23 | 73.04 | 77.37 |
| Aspaklarya-021 | 82.14 | 81.93 | 82.03 | 79.76 | 69.12 | 74.44 | 78.17 |
| Aspaklarya-014 | 81.51 | 81.93 | 81.72 | 78.45 | 67.87 | 73.16 | 77.36 |
| Aspaklarya-017 | 81.49 | 81.86 | 81.67 | 77.59 | 69.83 | 73.71 | 77.62 |
| Aspaklarya-004 | 80.31 | 81.47 | 80.89 | 77.26 | 69.40 | 73.33 | 77.04 |
| Aspaklarya-008 | 80.62 | 81.42 | 81.02 | 79.05 | 69.60 | 74.33 | 77.61 |
| Aspaklarya-007 | 81.13 | 80.82 | 80.97 | 77.60 | 70.03 | 73.81 | 77.33 |
| Aspaklarya-010 | 80.50 | 80.66 | 80.58 | 77.32 | 68.12 | 72.72 | 76.58 |
| Aspaklarya-002 | 80.70 | 80.63 | 80.67 | 78.07 | 68.25 | 73.16 | 76.85 |
| Qwen3-VL-8B-Instruct | 68.37 | 58.94 | 63.66 | 63.07 | 53.18 | 58.13 | 60.84 |
Bootstrap standard error: DM cap 0.66, DM vqa 1.46, RSICD cap 0.69, RSICD vqa 1.64, all overall 0.63. Paired model-vs-model error is smaller because image difficulty cancels - 0.42 overall and 0.56 DM. Use the paired figure for any "A beats B" claim.
3.2 exp1XX - four-slice benchmark
Best checkpoint per experiment, 500 graded items per slice (491 for DM caption), one fixed draw. Sorted by DM vqa.
| Model | DM cap | DM vqa | DM overall | RSICD cap | RSICD vqa | RSICD overall | All overall |
|---|---|---|---|---|---|---|---|
| Aspaklarya-112 beam4 | 82.74 | 75.10 | 78.92 | 79.14 | 68.70 | 73.92 | 76.42 |
| Aspaklarya-122 s1234 | 70.08 | 75.00 | 72.54 | 33.33 | 68.70 | 51.02 | 61.78 |
| Aspaklarya-122 s2024 | 69.83 | 74.68 | 72.26 | 35.08 | 69.91 | 52.50 | 62.38 |
| Aspaklarya-106 soup | 82.38 | 74.57 | 78.48 | 79.10 | 69.38 | 74.24 | 76.36 |
| Aspaklarya-105 | 81.96 | 74.49 | 78.23 | 78.62 | 68.96 | 73.79 | 76.01 |
| Aspaklarya-120 B | 69.13 | 74.40 | 71.77 | 38.35 | 68.62 | 53.49 | 62.62 |
| Aspaklarya-121 routed | 81.51 | 74.22 | 77.87 | 80.04 | 69.47 | 74.76 | 76.31 |
| Aspaklarya-121 vqa | 69.01 | 74.22 | 71.62 | 34.11 | 69.47 | 51.79 | 61.70 |
| Aspaklarya-101 | 82.53 | 74.16 | 78.35 | 79.64 | 70.56 | 75.10 | 76.72 |
| Aspaklarya-103 | 81.34 | 74.13 | 77.74 | 79.30 | 69.82 | 74.56 | 76.15 |
| Aspaklarya-111 | 81.32 | 74.03 | 77.68 | 79.45 | 71.68 | 75.57 | 76.62 |
| Aspaklarya-123 | 82.56 | 73.92 | 78.24 | 79.67 | 68.56 | 74.12 | 76.18 |
| Aspaklarya-104 | 82.70 | 73.72 | 78.21 | 78.88 | 69.59 | 74.24 | 76.22 |
| Aspaklarya-110 s2024 | 80.99 | 73.68 | 77.34 | 79.32 | 71.01 | 75.17 | 76.25 |
| control s3407 | 80.80 | 73.59 | 77.20 | 78.64 | 69.82 | 74.23 | 75.71 |
| Aspaklarya-115 | 80.52 | 73.57 | 77.05 | 77.92 | 69.18 | 73.55 | 75.30 |
| Aspaklarya-102 | 81.56 | 73.45 | 77.51 | 78.60 | 68.93 | 73.77 | 75.64 |
| Aspaklarya-114 | 80.26 | 73.18 | 76.72 | 78.65 | 70.56 | 74.61 | 75.66 |
| Aspaklarya-114 1536 | 80.28 | 73.17 | 76.73 | 78.67 | 70.56 | 74.62 | 75.67 |
| control s1234 | 81.08 | 73.01 | 77.05 | 77.66 | 69.63 | 73.65 | 75.34 |
| Aspaklarya-109 s1234 | 82.49 | 72.84 | 77.67 | 79.62 | 71.31 | 75.47 | 76.56 |
| Aspaklarya-106b soup | 81.53 | 72.77 | 77.15 | 78.73 | 69.99 | 74.36 | 75.75 |
| Aspaklarya-108 s2024 | 82.10 | 72.66 | 77.38 | 78.58 | 70.83 | 74.71 | 76.04 |
| Aspaklarya-112 t0.7 | 81.71 | 72.48 | 77.10 | 79.81 | 68.31 | 74.06 | 75.58 |
| Aspaklarya-113 | 80.25 | 72.37 | 76.31 | 76.97 | 69.52 | 73.25 | 74.78 |
| Aspaklarya-108 s1234 | 81.06 | 72.24 | 76.65 | 78.25 | 69.38 | 73.82 | 75.23 |
| Aspaklarya-112 rep | 81.70 | 72.19 | 76.95 | 78.53 | 69.64 | 74.09 | 75.52 |
| Aspaklarya-107 | 80.98 | 72.16 | 76.57 | 77.06 | 69.90 | 73.48 | 75.03 |
| control s2024 | 81.29 | 71.87 | 76.58 | 77.87 | 70.99 | 74.43 | 75.50 |
| Aspaklarya-119 | 71.88 | 70.97 | 71.43 | 61.95 | 69.52 | 65.74 | 68.58 |
| Aspaklarya-116 | 80.50 | 70.85 | 75.68 | 78.44 | 69.64 | 74.04 | 74.86 |
| Aspaklarya-020 | 82.79 | 66.18 | 74.49 | 79.99 | 70.65 | 75.32 | 74.90 |
| in-house reference | 71.91 | 65.26 | 68.59 | 58.58 | 63.59 | 61.09 | 64.84 |
| Aspaklarya-121 cap | 81.51 | 63.52 | 72.52 | 80.04 | 59.43 | 69.74 | 71.12 |
| Qwen3-VL-8B-Instruct | 68.31 | 60.64 | 64.48 | 63.28 | 53.13 | 58.21 | 61.34 |
All overall is the unweighted mean of the four slices - previously called MEAN4 - so it equals the mean of the two domain-overall columns and a model cannot win it by specialising on one slice.
Noise floor, measured from three same-config replicates: all-overall sigma 0.31; per slice DM cap 0.25, DM vqa 0.88, RSICD cap 0.52, RSICD vqa 0.74. A gap under 0.9 in the all-overall column is not a result. The VQA slices are 3-4x noisier than the caption slices, which is why the aggregate is the gate.
One consequence of this layout is worth stating. DM overall re-ranks the table against its own sort order. Aspaklarya-112 beam4 leads both DM columns, but 101 and 123 beat it on DM overall despite a lower DM vqa - and 122 / 120 B, which hold three of the top six DM vqa scores, fall to the bottom third once captioning is folded in.
4. Insights
Hyperparameters are saturated. Data is the only lever that ever worked. Rank, alpha, dropout, learning rate, schedule shape, schedule length, weight decay, gradient clipping, batch geometry, effective batch size, LR scaling at large batch, base-weight precision, vision-tower freezing, decoding, input resolution, QA depth, paraphrase augmentation, hard negatives and teacher distillation have all been swept. None displaced the reference recipe by more than the noise floor. Adding 91% more images cleared it on the first attempt (+1.57 RSICD, p = 0.012) and is the only training change in roughly 45 experiments to do so.
It is images that matter, not items. Aspaklarya-111 raised QA depth from 2 to 4 and captions from 1 to 2 - doubling the item count from the same images
- and landed inside its own arm (76.62 against an arm mean of 76.37 +/- 0.23).
The 101 gain is therefore better explained by the 5,706 new images than by the extra rows they brought.
Data volume helped; data duration did not. Aspaklarya-021 (8 epochs on the enlarged corpus) and Aspaklarya-123 (8 epochs on the exp1XX recipe) both peaked mid-run and declined. 123's mechanism is the clearest diagnostic here: RSICD vqa peaks at epoch 2 and falls for six consecutive epochs (70.93 -> 65.41) while DM vqa and both caption slices plateau and hide the bleed inside the mean until epoch 6. Longer training memorises in-domain phrasing and loses out-of-domain generalisation. Out-of-domain VQA is the early-stopping signal - it turns over three epochs before the aggregate does.
The joint objective is load-bearing, and the interference is asymmetric. Training VQA alone destroys captioning - 121 vqa and 122 collapse RSICD cap to roughly 34 from roughly 79, a 45-point hole. Training captions alone merely dents VQA (74.22 -> 63.52). Caption supervision carries general instruction-following that VQA supervision does not. More surprisingly, separating the tasks helped neither side: the caption expert is worse at captioning (81.51 / 80.04) than 101 trained on the mixture (82.53 / 79.64), and worse than the untuned base (82.79 / 79.99). Task interference is not the ceiling - the mixture was helping captioning.
Warm-starting a VQA adapter from a caption adapter is the second confirmed lever. Aspaklarya-120 B produced the best DM vqa in the programme at n = 1; 122 re-ran it at two fresh seeds and the two-seed mean is 74.84 against a control of 72.82, above the pre-registered 74.58 bar. The effect is real but smaller than discovery suggested (75.48 -> 74.84) and clears the bar by only 0.26. It is a lever to fold into a joint recipe, not a model to ship - everything in that family is QA-only and has no captioning left.
Sorting by DM vqa and by all-overall give almost opposite answers. The top of table 3.2 is dominated by models that are catastrophic overall: 122 and 120 B hold three of the top six DM vqa scores and the three worst all-overall scores in the round. Conversely Aspaklarya-020, a round older, posts the worst DM vqa of any finetune and still wins three slices of four. Any single-slice claim on this page is a trap - that is what the aggregate gate exists to prevent.
Nothing has beaten 101-step4500 since it was trained, roughly 30 experiments later. The fair description of its recipe is 76.37 +/- 0.23 across three seeds, not 76.72; picking the 76.72 checkpoint is selecting on noise. 106-soup reaches 76.36 for zero GPU hours by stacking four rank-16 adapters into one rank-64 adapter, and is the better default when compute is the constraint - with the caveat that souping returns the parents' average, not a gain over it.
Two published results have been retracted after control errors. A reported +2.32 paraphrase gain was baseline mismatch almost end to end - re-measured against a matched control it is +1.00, which is 1.1 sigma on a slice whose sigma is 0.88. And sigma was wrongly attributed to judge drift: judge re-invocation reproduces to ~0.01, while same-seed re-runs still differ by 0.30, so the variance is irreducible CUDA nondeterminism, not the grader.
The judge is text-only. It compares an answer against a reference and never sees the image, so it measures agreement with the reference, not truth about the pixels. For a round built on longer answers this matters; human spot-checks are the only way to close that gap.