AI & ML interests

AI models

Recent Activity

Organization Card

Aspaklarya - Qwen3-VL-8B for remote-sensing VQA + captioning

LoRA finetunes of Qwen3-VL-8B-Instruct, trained jointly on visual question answering and image captioning over overhead imagery. All model repos are private.

Two experiment rounds, run against two different answer styles:

exp0XX exp1XX
supervisionVQA column, ~4-character answers (yes, 3, river)VQA_1126 column, ~300-character grounded answers
benchmark1,000-image union draw, 2,000 graded itemsfour-slice seed-1105 draw, 1,991 graded items
bestAspaklarya-020-step4500Aspaklarya-101-step4500

The two rounds are not comparable to each other - the benchmarks reward opposite answer styles. Within a round, every number is directly comparable.

Scoring is LLM-as-judge (gemma-4-31b-it-imiai), 0-100 per item. DM = the in-domain overhead-imagery subset; RSICD = the public remote-sensing benchmark subset. Two references appear in every table: an in-house reference model and the untuned base.

2. Experiment parameters

2.1 exp0XX

Constant everywhere: effective batch 32 (except 014 at 16, 015-017 at 64), warmup 0.03, grad clip 0.3 (1.0 for 011/012), seed 3407, 4 epochs. 4-bit base for 001-017, bf16 from 018 on.

Model r / alpha Drop LR Sched Vision Per image Items What it tests
Aspaklarya-00116/160.02e-4lineartrained1c+2q18,864reference recipe
Aspaklarya-001 (ep5)16/160.02e-4lineartrained1c+2q18,864one epoch past budget
Aspaklarya-00232/640.051e-4cosinetrained1c+2q18,864higher rank, lower LR
Aspaklarya-00316/160.02e-4lineartrained2c+4q37,7282x resampled data
Aspaklarya-00416/160.02e-4linearfrozen1c+2q18,864language-only
Aspaklarya-00516/160.02e-4cosinetrained1c+6q43,864VQA-heavy
Aspaklarya-00616/160.02e-4lineartrained1c+2q18,864batch 4 x accum 8
Aspaklarya-0078/80.02e-4lineartrained1c+2q18,864low rank
Aspaklarya-00816/160.01e-4lineartrained1c+4q31,440more QA, lower LR
Aspaklarya-00932/320.02e-4lineartrained1c+2q18,864high rank
Aspaklarya-01016/160.02e-4lineartrained1c+2q18,864wider prompt pool
Aspaklarya-01116/160.052e-4lineartrained1c+2q18,864dropout, wd 0.05, clip 1.0
Aspaklarya-01216/160.02e-4lineartrained1c+2q18,864as 011, no dropout
Aspaklarya-01316/160.01e-4lineartrained1c+2q18,864low LR, long schedule
Aspaklarya-01416/160.02e-4lineartrained1c+2q18,864effective batch 16
Aspaklarya-01516/160.02e-4lineartrained1c+2q18,864effective batch 64
Aspaklarya-01616/160.04e-4lineartrained1c+2q18,864batch 64 + linear LR scaling - stopped early, not scored
Aspaklarya-01716/160.03e-4lineartrained1c+2q18,864batch 64 + sqrt LR scaling
Aspaklarya-01816/160.02e-4lineartrained1c+2q18,864bf16 base
Aspaklarya-01916/160.052e-4lineartrained1c+2q18,864bf16 base + 011 recipe
Aspaklarya-02016/160.02e-4lineartrained1c+2q35,981bf16 + +91% images
Aspaklarya-02116/160.02e-4lineartrained1c+2q35,981as 020, 8 epochs
in-house reference-------reference
Qwen3-VL-8B-Instruct-------untuned base

2.2 exp1XX

Constant across the round: dropout 0, bf16 base, effective batch 32 (4x8), warmup 0.03, weight decay 0.001, grad clip 0.3, max side 1024, seq 2048, seed 3407 unless stated, LoRA on all linear layers of both towers, VQA_1126 answer column.

Model Phase r / alpha LR Sched Vision Per image Ep Items What it tests
Aspaklarya-101full16/162e-4lineartrained1c+2q435,979reference recipe, 1.9x data
Aspaklarya-102full16/162e-4lineartrained1c+2q418,862train split only, half the images
Aspaklarya-103full32/322e-4lineartrained1c+2q418,862rank 32
Aspaklarya-104full16/322e-4lineartrained1c+2q418,862scale 1.0 -> 2.0
Aspaklarya-105full16/162e-4lineartrained1c+2q418,862caption loss weight 0.4
Aspaklarya-106full64/64------LoRA soup of 103/104/105, no training
Aspaklarya-106bfull64/64------soup, alternate mixing weights
Aspaklarya-107full16/161e-4cosinetrained1c+2q318,862cosine, warmup 0.10
Aspaklarya-108 s1234full16/162e-4lineartrained1c+2q418,862seed 1234 of 102
Aspaklarya-108 s2024full16/162e-4lineartrained1c+2q418,862seed 2024 of 102
Aspaklarya-109full16/162e-4lineartrained1c+2q435,979seed 1234 of 101
Aspaklarya-110full16/162e-4lineartrained1c+2q435,979seed 2024 of 101
Aspaklarya-111full16/162e-4lineartrained2c+4q371,944QA depth 2 -> 4
Aspaklarya-112 beam4probe16/16------decoding only on the 101 adapter
Aspaklarya-112 t0.7probe16/16------temperature 0.7
Aspaklarya-112 repprobe16/16------repetition penalty 1.1
Aspaklarya-113probe16/162e-4linearfrozen1c+2q135,979vision tower frozen
Aspaklarya-114probe16/162e-4lineartrained1c+2q135,979seq 3072
Aspaklarya-114 1536probe16/162e-4lineartrained1c+2q135,979max side 1536
Aspaklarya-115probe16/162e-4lineartrained1c+2q135,981paraphrased questions
Aspaklarya-116probe16/162e-4lineartrained1c+2q137,178+1,199 hard negatives
Aspaklarya-119probe16/162e-4lineartrained1c+2q135,980teacher-rewritten answers
Aspaklarya-120 Bprobe16/162e-4lineartrained0c+2q123,986QA only, resumed from 120 A
Aspaklarya-121 vqarouting16/162e-4lineartrained0c+2q423,986VQA expert alone
Aspaklarya-121 caprouting16/162e-4lineartrained1c+0q411,993caption expert alone
Aspaklarya-121 routedrouting16/162e-4lineartrainedsplit435,979two task experts + router
Aspaklarya-122 s1234routing16/162e-4lineartrained0c+2q123,986replication of 120 B, seed 1234
Aspaklarya-122 s2024routing16/162e-4lineartrained0c+2q123,986replication of 120 B, seed 2024
Aspaklarya-123routing16/162e-4lineartrained1c+2q835,979double the epochs
control s3407probe16/162e-4lineartrained1c+2q135,9791-epoch control
control s1234probe16/162e-4lineartrained1c+2q135,9791-epoch control
control s2024probe16/162e-4lineartrained1c+2q135,9791-epoch control
Aspaklarya-020exp0XX16/162e-4lineartrained1c+2q435,981cross-round reference
in-house reference--------reference
Qwen3-VL-8B-Instruct--------untuned base

The numeric gap is real: 117 and 118 do not exist - both were designed, then cancelled when 114 proved the resolution axis a no-op.

3. Evaluation results

3.1 exp0XX - legacy benchmark

Best checkpoint per run on a fixed 1,000-image draw (2,000 graded items). Overall = mean of caption and VQA. Sorted by DM vqa.

Model DM cap DM vqa DM overall RSICD cap RSICD vqa RSICD overall All overall
in-house reference71.8883.6277.7558.7263.4461.0869.26
Aspaklarya-01181.8583.5182.6878.0668.3373.2077.85
Aspaklarya-00181.8483.2382.5478.1768.8973.5377.95
Aspaklarya-00681.4082.9582.1878.2668.8873.5777.80
Aspaklarya-01982.3982.8282.6178.3469.2473.7978.12
Aspaklarya-01582.1482.7682.4578.0766.9872.5277.40
Aspaklarya-01882.2882.7082.4978.7069.1573.9278.13
Aspaklarya-00581.3782.3781.8777.7969.3573.5777.64
Aspaklarya-00980.0182.3481.1877.0969.5973.3477.19
Aspaklarya-001 (ep5)80.6082.2581.4278.0570.3174.1877.73
Aspaklarya-01380.5882.2081.3977.7068.7473.2277.23
Aspaklarya-00381.2382.1881.7177.9269.1073.5177.53
Aspaklarya-02082.7482.1082.4279.9370.2875.1078.70
Aspaklarya-01281.6882.0581.8677.8568.2373.0477.37
Aspaklarya-02182.1481.9382.0379.7669.1274.4478.17
Aspaklarya-01481.5181.9381.7278.4567.8773.1677.36
Aspaklarya-01781.4981.8681.6777.5969.8373.7177.62
Aspaklarya-00480.3181.4780.8977.2669.4073.3377.04
Aspaklarya-00880.6281.4281.0279.0569.6074.3377.61
Aspaklarya-00781.1380.8280.9777.6070.0373.8177.33
Aspaklarya-01080.5080.6680.5877.3268.1272.7276.58
Aspaklarya-00280.7080.6380.6778.0768.2573.1676.85
Qwen3-VL-8B-Instruct68.3758.9463.6663.0753.1858.1360.84

Bootstrap standard error: DM cap 0.66, DM vqa 1.46, RSICD cap 0.69, RSICD vqa 1.64, all overall 0.63. Paired model-vs-model error is smaller because image difficulty cancels - 0.42 overall and 0.56 DM. Use the paired figure for any "A beats B" claim.

3.2 exp1XX - four-slice benchmark

Best checkpoint per experiment, 500 graded items per slice (491 for DM caption), one fixed draw. Sorted by DM vqa.

Model DM cap DM vqa DM overall RSICD cap RSICD vqa RSICD overall All overall
Aspaklarya-112 beam482.7475.1078.9279.1468.7073.9276.42
Aspaklarya-122 s123470.0875.0072.5433.3368.7051.0261.78
Aspaklarya-122 s202469.8374.6872.2635.0869.9152.5062.38
Aspaklarya-106 soup82.3874.5778.4879.1069.3874.2476.36
Aspaklarya-10581.9674.4978.2378.6268.9673.7976.01
Aspaklarya-120 B69.1374.4071.7738.3568.6253.4962.62
Aspaklarya-121 routed81.5174.2277.8780.0469.4774.7676.31
Aspaklarya-121 vqa69.0174.2271.6234.1169.4751.7961.70
Aspaklarya-10182.5374.1678.3579.6470.5675.1076.72
Aspaklarya-10381.3474.1377.7479.3069.8274.5676.15
Aspaklarya-11181.3274.0377.6879.4571.6875.5776.62
Aspaklarya-12382.5673.9278.2479.6768.5674.1276.18
Aspaklarya-10482.7073.7278.2178.8869.5974.2476.22
Aspaklarya-110 s202480.9973.6877.3479.3271.0175.1776.25
control s340780.8073.5977.2078.6469.8274.2375.71
Aspaklarya-11580.5273.5777.0577.9269.1873.5575.30
Aspaklarya-10281.5673.4577.5178.6068.9373.7775.64
Aspaklarya-11480.2673.1876.7278.6570.5674.6175.66
Aspaklarya-114 153680.2873.1776.7378.6770.5674.6275.67
control s123481.0873.0177.0577.6669.6373.6575.34
Aspaklarya-109 s123482.4972.8477.6779.6271.3175.4776.56
Aspaklarya-106b soup81.5372.7777.1578.7369.9974.3675.75
Aspaklarya-108 s202482.1072.6677.3878.5870.8374.7176.04
Aspaklarya-112 t0.781.7172.4877.1079.8168.3174.0675.58
Aspaklarya-11380.2572.3776.3176.9769.5273.2574.78
Aspaklarya-108 s123481.0672.2476.6578.2569.3873.8275.23
Aspaklarya-112 rep81.7072.1976.9578.5369.6474.0975.52
Aspaklarya-10780.9872.1676.5777.0669.9073.4875.03
control s202481.2971.8776.5877.8770.9974.4375.50
Aspaklarya-11971.8870.9771.4361.9569.5265.7468.58
Aspaklarya-11680.5070.8575.6878.4469.6474.0474.86
Aspaklarya-02082.7966.1874.4979.9970.6575.3274.90
in-house reference71.9165.2668.5958.5863.5961.0964.84
Aspaklarya-121 cap81.5163.5272.5280.0459.4369.7471.12
Qwen3-VL-8B-Instruct68.3160.6464.4863.2853.1358.2161.34

All overall is the unweighted mean of the four slices - previously called MEAN4 - so it equals the mean of the two domain-overall columns and a model cannot win it by specialising on one slice.

Noise floor, measured from three same-config replicates: all-overall sigma 0.31; per slice DM cap 0.25, DM vqa 0.88, RSICD cap 0.52, RSICD vqa 0.74. A gap under 0.9 in the all-overall column is not a result. The VQA slices are 3-4x noisier than the caption slices, which is why the aggregate is the gate.

One consequence of this layout is worth stating. DM overall re-ranks the table against its own sort order. Aspaklarya-112 beam4 leads both DM columns, but 101 and 123 beat it on DM overall despite a lower DM vqa - and 122 / 120 B, which hold three of the top six DM vqa scores, fall to the bottom third once captioning is folded in.

4. Insights

Hyperparameters are saturated. Data is the only lever that ever worked. Rank, alpha, dropout, learning rate, schedule shape, schedule length, weight decay, gradient clipping, batch geometry, effective batch size, LR scaling at large batch, base-weight precision, vision-tower freezing, decoding, input resolution, QA depth, paraphrase augmentation, hard negatives and teacher distillation have all been swept. None displaced the reference recipe by more than the noise floor. Adding 91% more images cleared it on the first attempt (+1.57 RSICD, p = 0.012) and is the only training change in roughly 45 experiments to do so.

It is images that matter, not items. Aspaklarya-111 raised QA depth from 2 to 4 and captions from 1 to 2 - doubling the item count from the same images

  • and landed inside its own arm (76.62 against an arm mean of 76.37 +/- 0.23).

The 101 gain is therefore better explained by the 5,706 new images than by the extra rows they brought.

Data volume helped; data duration did not. Aspaklarya-021 (8 epochs on the enlarged corpus) and Aspaklarya-123 (8 epochs on the exp1XX recipe) both peaked mid-run and declined. 123's mechanism is the clearest diagnostic here: RSICD vqa peaks at epoch 2 and falls for six consecutive epochs (70.93 -> 65.41) while DM vqa and both caption slices plateau and hide the bleed inside the mean until epoch 6. Longer training memorises in-domain phrasing and loses out-of-domain generalisation. Out-of-domain VQA is the early-stopping signal - it turns over three epochs before the aggregate does.

The joint objective is load-bearing, and the interference is asymmetric. Training VQA alone destroys captioning - 121 vqa and 122 collapse RSICD cap to roughly 34 from roughly 79, a 45-point hole. Training captions alone merely dents VQA (74.22 -> 63.52). Caption supervision carries general instruction-following that VQA supervision does not. More surprisingly, separating the tasks helped neither side: the caption expert is worse at captioning (81.51 / 80.04) than 101 trained on the mixture (82.53 / 79.64), and worse than the untuned base (82.79 / 79.99). Task interference is not the ceiling - the mixture was helping captioning.

Warm-starting a VQA adapter from a caption adapter is the second confirmed lever. Aspaklarya-120 B produced the best DM vqa in the programme at n = 1; 122 re-ran it at two fresh seeds and the two-seed mean is 74.84 against a control of 72.82, above the pre-registered 74.58 bar. The effect is real but smaller than discovery suggested (75.48 -> 74.84) and clears the bar by only 0.26. It is a lever to fold into a joint recipe, not a model to ship - everything in that family is QA-only and has no captioning left.

Sorting by DM vqa and by all-overall give almost opposite answers. The top of table 3.2 is dominated by models that are catastrophic overall: 122 and 120 B hold three of the top six DM vqa scores and the three worst all-overall scores in the round. Conversely Aspaklarya-020, a round older, posts the worst DM vqa of any finetune and still wins three slices of four. Any single-slice claim on this page is a trap - that is what the aggregate gate exists to prevent.

Nothing has beaten 101-step4500 since it was trained, roughly 30 experiments later. The fair description of its recipe is 76.37 +/- 0.23 across three seeds, not 76.72; picking the 76.72 checkpoint is selecting on noise. 106-soup reaches 76.36 for zero GPU hours by stacking four rank-16 adapters into one rank-64 adapter, and is the better default when compute is the constraint - with the caveat that souping returns the parents' average, not a gain over it.

Two published results have been retracted after control errors. A reported +2.32 paraphrase gain was baseline mismatch almost end to end - re-measured against a matched control it is +1.00, which is 1.1 sigma on a slice whose sigma is 0.88. And sigma was wrongly attributed to judge drift: judge re-invocation reproduces to ~0.01, while same-seed re-runs still differ by 0.30, so the variance is irreducible CUDA nondeterminism, not the grader.

The judge is text-only. It compares an answer against a reference and never sees the image, so it measures agreement with the reference, not truth about the pixels. For a round built on longer answers this matters; human spot-checks are the only way to close that gap.

datasets 0

None public yet