# Base vs fine-tuned on the same 10 test inputs First 10 rows of the held-out test split (not cherry-picked). Both models get the identical prompt and identical output post-processing. ### 1. Tell me the ships in classs for kuha class ```sql -- schema CREATE TABLE table_name_52 (ships_in_class VARCHAR, class VARCHAR) ``` **Gold** ```sql SELECT ships_in_class FROM table_name_52 WHERE class = "kuha class" ``` **Base (raw output)** ``` ```sql SELECT DISTINCT ships_in_class FROM table_name_52 WHERE class = 'kuha' ``` ``` -> parsed: `SELECT DISTINCT ships_in_class FROM table_name_52 WHERE class = 'kuha'` | exact match: **no** **Fine-tuned (raw output)** ``` SELECT ships_in_class FROM table_name_52 WHERE class = "kuha" ``` -> parsed: `SELECT ships_in_class FROM table_name_52 WHERE class = "kuha"` | exact match: **no** --- ### 2. What is the Drawn has an Against smaller than 20 and Points of 15? ```sql -- schema CREATE TABLE table_name_46 (drawn VARCHAR, against VARCHAR, points VARCHAR) ``` **Gold** ```sql SELECT drawn FROM table_name_46 WHERE against < 20 AND points = 15 ``` **Base (raw output)** ``` ```sql SELECT drawn FROM table_name_46 WHERE against < 20 AND points = '15'; ``` ``` -> parsed: `SELECT drawn FROM table_name_46 WHERE against < 20 AND points = '15'` | exact match: **no** **Fine-tuned (raw output)** ``` SELECT drawn FROM table_name_46 WHERE against < 20 AND points = "15" ``` -> parsed: `SELECT drawn FROM table_name_46 WHERE against < 20 AND points = "15"` | exact match: **no** --- ### 3. How many picks for round 12? ```sql -- schema CREATE TABLE table_name_5 (pick__number VARCHAR, round VARCHAR) ``` **Gold** ```sql SELECT COUNT(pick__number) FROM table_name_5 WHERE round = 12 ``` **Base (raw output)** ``` SELECT COUNT(pick__number) FROM table_name_5 WHERE round = '12' ``` -> parsed: `SELECT COUNT(pick__number) FROM table_name_5 WHERE round = '12'` | exact match: **no** **Fine-tuned (raw output)** ``` SELECT COUNT(pick__number) FROM table_name_5 WHERE round = "round 12" ``` -> parsed: `SELECT COUNT(pick__number) FROM table_name_5 WHERE round = "round 12"` | exact match: **no** --- ### 4. wich were the events when bout 2 was zhang ( chn ) l 0-5? ```sql -- schema CREATE TABLE table_19398910_4 (event VARCHAR, bout_2 VARCHAR) ``` **Gold** ```sql SELECT event FROM table_19398910_4 WHERE bout_2 = "Zhang ( CHN ) L 0-5" ``` **Base (raw output)** ``` ```sql SELECT event FROM table_19398910_4 WHERE bout_2 = 'zhang (chn) l 0-5'; ``` ``` -> parsed: `SELECT event FROM table_19398910_4 WHERE bout_2 = 'zhang (chn) l 0-5'` | exact match: **YES** **Fine-tuned (raw output)** ``` SELECT event FROM table_19398910_4 WHERE bout_2 = "Zhang ( CHN ) L 0-5" ``` -> parsed: `SELECT event FROM table_19398910_4 WHERE bout_2 = "Zhang ( CHN ) L 0-5"` | exact match: **YES** --- ### 5. Find the name and hours of project that has the most number of scientists. ```sql -- schema CREATE TABLE assignedto (project VARCHAR); CREATE TABLE projects (name VARCHAR, hours VARCHAR, code VARCHAR) ``` **Gold** ```sql SELECT T1.name, T1.hours FROM projects AS T1 JOIN assignedto AS T2 ON T1.code = T2.project GROUP BY T2.project ORDER BY COUNT(*) DESC LIMIT 1 ``` **Base (raw output)** ``` ```sql SELECT p.name, p.hours FROM projects AS p JOIN assignedto AS at ON p.code = at.code GROUP BY p.name, p.hours ORDER BY COUNT(*) DESC LIMIT 1; ``` ``` -> parsed: `SELECT p.name, p.hours FROM projects AS p JOIN assignedto AS at ON p.code = at.code GROUP BY p.name, p.hours ORDER BY COUNT(*) DESC LIMIT 1` | exact match: **no** **Fine-tuned (raw output)** ``` SELECT T1.name, T1.hours FROM projects AS T1 JOIN assignedto AS T2 ON T1.code = T2.project GROUP BY T1.name ORDER BY COUNT(*) DESC LIMIT 1 ``` -> parsed: `SELECT T1.name, T1.hours FROM projects AS T1 JOIN assignedto AS T2 ON T1.code = T2.project GROUP BY T1.name ORDER BY COUNT(*) DESC LIMIT 1` | exact match: **no** --- ### 6. What is the total number of 3rd placed teams when the host is University of Manitoba, Winnipeg, Manitoba? ```sql -- schema CREATE TABLE table_1906920_1 (host VARCHAR) ``` **Gold** ```sql SELECT COUNT(3 AS rd_place) FROM table_1906920_1 WHERE host = "University of Manitoba, Winnipeg, Manitoba" ``` **Base (raw output)** ``` SELECT COUNT(*) FROM table_1906920_1 WHERE host = 'University of Manitoba' AND position = 3 ``` -> parsed: `SELECT COUNT(*) FROM table_1906920_1 WHERE host = 'University of Manitoba' AND position = 3` | exact match: **no** **Fine-tuned (raw output)** ``` SELECT COUNT(3 AS rd_placed) FROM table_1906920_1 WHERE host = "University of Manitoba, Winnipeg, Manitoba" ``` -> parsed: `SELECT COUNT(3 AS rd_placed) FROM table_1906920_1 WHERE host = "University of Manitoba, Winnipeg, Manitoba"` | exact match: **no** --- ### 7. What is the attendance for week 11? ```sql -- schema CREATE TABLE table_name_59 (attendance VARCHAR, week VARCHAR) ``` **Gold** ```sql SELECT attendance FROM table_name_59 WHERE week = 11 ``` **Base (raw output)** ``` SELECT attendance FROM table_name_59 WHERE week = '11' ``` -> parsed: `SELECT attendance FROM table_name_59 WHERE week = '11'` | exact match: **no** **Fine-tuned (raw output)** ``` SELECT attendance FROM table_name_59 WHERE week = "11" ``` -> parsed: `SELECT attendance FROM table_name_59 WHERE week = "11"` | exact match: **no** --- ### 8. Name the syrian christians for kollam ```sql -- schema CREATE TABLE table_23214055_2 (syrian_christians VARCHAR, district VARCHAR) ``` **Gold** ```sql SELECT syrian_christians FROM table_23214055_2 WHERE district = "Kollam" ``` **Base (raw output)** ``` ```sql SELECT syrian_christians FROM table_23214055_2 WHERE district = 'kollam' ``` ``` -> parsed: `SELECT syrian_christians FROM table_23214055_2 WHERE district = 'kollam'` | exact match: **YES** **Fine-tuned (raw output)** ``` SELECT syrian_christians FROM table_23214055_2 WHERE district = "Kollam" ``` -> parsed: `SELECT syrian_christians FROM table_23214055_2 WHERE district = "Kollam"` | exact match: **YES** --- ### 9. What is every entry for Saturday August 27 when the entry for Thursday August 25 is 23' 56.90 94.528mph? ```sql -- schema CREATE TABLE table_30058355_7 (sat_27_aug VARCHAR, thurs_25_aug VARCHAR) ``` **Gold** ```sql SELECT sat_27_aug FROM table_30058355_7 WHERE thurs_25_aug = "23' 56.90 94.528mph" ``` **Base (raw output)** ``` ```sql SELECT sat_27_aug FROM table_30058355_7 WHERE thurs_25_aug = '23\' 56.90 94.528mph'; ``` ``` -> parsed: `SELECT sat_27_aug FROM table_30058355_7 WHERE thurs_25_aug = '23\' 56.90 94.528mph'` | exact match: **no** **Fine-tuned (raw output)** ``` SELECT sat_27_aug FROM table_30058355_7 WHERE thurs_25_aug = "23' 56.90 94.528mph" ``` -> parsed: `SELECT sat_27_aug FROM table_30058355_7 WHERE thurs_25_aug = "23' 56.90 94.528mph"` | exact match: **YES** --- ### 10. Which Owner has a Name / Format of 105.3 kool fm - hot adult contemporary? ```sql -- schema CREATE TABLE table_name_18 (owner VARCHAR, name___format VARCHAR) ``` **Gold** ```sql SELECT owner FROM table_name_18 WHERE name___format = "105.3 kool fm - hot adult contemporary" ``` **Base (raw output)** ``` ```sql SELECT owner FROM table_name_18 WHERE name___format = '105.3 kool fm - hot adult contemporary'; ``` ``` -> parsed: `SELECT owner FROM table_name_18 WHERE name___format = '105.3 kool fm - hot adult contemporary'` | exact match: **YES** **Fine-tuned (raw output)** ``` SELECT owner FROM table_name_18 WHERE name___format = "105.3 kool fm - hot adult contemporary" ``` -> parsed: `SELECT owner FROM table_name_18 WHERE name___format = "105.3 kool fm - hot adult contemporary"` | exact match: **YES** ---