asadullahdogarr/gpt-oss-20b-financial-forensic-auditor

GPT-OSS-20B Financial Forensic Auditor

This repository contains the fine-tuned LoRA adapter layers for openai/gpt-oss-20b, optimized to perform strict, multi-step mathematical and logical reasoning chains over complex corporate financial statement disclosures, balance sheets, and SEC 10-K filings.

Developed exclusively as part of the AutoScientist Challenge 2026 under the Finance category.

Model Details

• Developed by: Asad Ullah Dogar
• Model Type: PEFT (Parameter-Efficient Fine-Tuning) using LoRA (Low-Rank Adaptation)
• Base Model: openai/gpt-oss-20b
• Language(s): English (100%)
• License: Apache 2.0
• Primary Domain: Corporate Finance, Forensic Auditing, Accounting Arithmetic

Intended Uses & Limitations

Direct Use

This model is specifically tailored to break down complex accounting queries into granular, programmatic scratchpad logic. It extracts relevant data figures from text fragments, structures step-by-step arithmetic operations, and executes verification processes to minimize financial value hallucinations.

Out-of-Scope Use

This adapter is not designed for real-time high-frequency trading applications, generalized creative writing, or non-financial domain tasks.

Training Data & Methodology

The model was trained using the adaption-financial-reasoning-steps dataset (27,172 total rows), engineered and optimized via the Adaption Labs Adaptive Data platform.

The fine-tuning dataset focuses heavily on converting basic financial QA formats into comprehensive reasoning traces, including:

• Year-over-year operational margin variances
• Structural lease commitments and multi-tier debt obligations
• Percentage-based asset allocation and revenue trajectory shifts

Training Hyperparameters

The model was fine-tuned with the following structural constraints:

• Training Method: Supervised Fine-Tuning (SFT)
• LoRA Rank ($r$): 8
• LoRA Alpha ($\alpha$): 8
• Trainable Targeted Modules: q_proj, v_proj
• Learning Rate: 0.0001 (with Cosine Decay scheduler)
• Warmup Ratio: 10% linear warmup
• Epochs: 1
• Train on Inputs: false (Loss computed strictly on target reasoning completions)

Evaluation Results

Through iterative optimization via the AutoScientist loop, the core instruction data was systematically evolved to close calculation drift gaps:

• Data Quality Score: 4.0 → 9.4 (+135.0% relative improvement)
• Dataset Grade Jump: Grade D Baseline → Grade A Elite Standard
• Task Percentile Placement: 6.8 → 33.0

The convergence profile demonstrates steady loss line alignment across both tracking steps and validation boundaries with zero gradient anomalies.

Technical Specifications

• Framework: PEFT 0.15.1
• Optimization Infrastructure: Adaption Labs Cloud Compute Cluster