asd11dasd21dd commited on
Commit
b148a7e
·
verified ·
1 Parent(s): 328e00c

Update README with all 15 benchmark scores (3 decimals)

Browse files
Files changed (1) hide show
  1. README.md +24 -25
README.md CHANGED
@@ -3,6 +3,9 @@ license: mit
3
  library_name: transformers
4
  ---
5
  # MyAwesomeModel
 
 
 
6
 
7
  <div align="center">
8
  <img src="figures/fig1.png" width="60%" alt="MyAwesomeModel" />
@@ -23,7 +26,7 @@ The MyAwesomeModel has undergone a significant version upgrade. In the latest up
23
  <img width="80%" src="figures/fig3.png">
24
  </p>
25
 
26
- Compared to the previous version, the upgraded model shows significant improvements in handling complex reasoning tasks. For instance, in the AIME 2025 test, the model's accuracy has increased from 70% in the previous version to 87.5% in the current version. This advancement stems from enhanced thinking depth during the reasoning process: in the AIME test set, the previous model used an average of 12K tokens per question, whereas the new version averages 23K tokens per question.
27
 
28
  Beyond its improved reasoning capabilities, this version also offers a reduced hallucination rate and enhanced support for function calling.
29
 
@@ -33,32 +36,30 @@ Beyond its improved reasoning capabilities, this version also offers a reduced h
33
 
34
  <div align="center">
35
 
36
- | | Benchmark | MyAwesomeModel |
37
- |---|---|---|
38
- | **Core Reasoning Tasks** | Math Reasoning | 0.550 |
39
- | | Logical Reasoning | 0.819 |
40
- | | Common Sense | 0.736 |
41
- | **Language Understanding** | Reading Comprehension | 0.700 |
42
- | | Question Answering | 0.607 |
43
- | | Text Classification | 0.828 |
44
- | | Sentiment Analysis | 0.792 |
45
- | **Generation Tasks** | Code Generation | 0.650 |
46
- | | Creative Writing | 0.610 |
47
- | | Dialogue Generation | 0.644 |
48
- | | Summarization | 0.767 |
49
- | **Specialized Capabilities**| Translation | 0.804 |
50
- | | Knowledge Retrieval | 0.676 |
51
- | | Instruction Following | 0.758 |
52
- | | Safety Evaluation | 0.739 |
53
 
54
  </div>
55
 
56
  ### Overall Performance Summary
57
-
58
- The MyAwesomeModel demonstrates strong performance across all evaluated benchmark categories, with particularly notable results in reasoning and generation. The model was selected as the best checkpoint (step_1000) based on the highest eval_accuracy (0.717) among all training checkpoints.
59
 
60
  ## 3. Chat Website & API Platform
61
-
62
  We offer a chat interface and API for you to interact with MyAwesomeModel. Please check our official website for more details.
63
 
64
  ## 4. How to Run Locally
@@ -83,9 +84,8 @@ For example,
83
  You are MyAwesomeModel, a helpful AI assistant.
84
  Today is May 28, 2025, Monday.
85
  ```
86
-
87
  ### Temperature
88
- We recommend setting the temperature parameter $T_{model}$ to 0.6.
89
 
90
  ### Prompts for File Uploading and Web Search
91
  For file uploading, please follow the template to create prompts, where {file_name}, {file_content} and {question} are arguments.
@@ -118,9 +118,8 @@ When responding, please keep the following points in mind:
118
  ```
119
 
120
  ## 5. License
121
-
122
  This code repository is licensed under the [MIT License](LICENSE). The use of MyAwesomeModel models is also subject to the [MIT License](LICENSE). The model series supports commercial use and distillation.
123
 
124
  ## 6. Contact
125
-
126
  If you have any questions, please raise an issue on our GitHub repository or contact us at contact@MyAwesomeModel.ai.
 
 
3
  library_name: transformers
4
  ---
5
  # MyAwesomeModel
6
+ <!-- markdownlint-disable first-line-h1 -->
7
+ <!-- markdownlint-disable html -->
8
+ <!-- markdownlint-disable no-duplicate-header -->
9
 
10
  <div align="center">
11
  <img src="figures/fig1.png" width="60%" alt="MyAwesomeModel" />
 
26
  <img width="80%" src="figures/fig3.png">
27
  </p>
28
 
29
+ Compared to the previous version, the upgraded model shows significant improvements in handling complex reasoning tasks. For instance, in the AIME 2025 test, the models accuracy has increased from 70% in the previous version to 87.5% in the current version. This advancement stems from enhanced thinking depth during the reasoning process: in the AIME test set, the previous model used an average of 12K tokens per question, whereas the new version averages 23K tokens per question.
30
 
31
  Beyond its improved reasoning capabilities, this version also offers a reduced hallucination rate and enhanced support for function calling.
32
 
 
36
 
37
  <div align="center">
38
 
39
+ | | Benchmark | Model1 | Model2 | Model1-v2 | MyAwesomeModel |
40
+ |---|---|---|---|---|---|
41
+ | **Core Reasoning Tasks** | Math Reasoning | 0.510 | 0.535 | 0.521 | 0.550 |
42
+ | | Logical Reasoning | 0.789 | 0.801 | 0.810 | 0.819 |
43
+ | | Common Sense | 0.716 | 0.702 | 0.725 | 0.736 |
44
+ | **Language Understanding** | Reading Comprehension | 0.671 | 0.685 | 0.690 | 0.700 |
45
+ | | Question Answering | 0.582 | 0.599 | 0.601 | 0.607 |
46
+ | | Text Classification | 0.803 | 0.811 | 0.820 | 0.828 |
47
+ | | Sentiment Analysis | 0.777 | 0.781 | 0.790 | 0.792 |
48
+ | **Generation Tasks** | Code Generation | 0.615 | 0.631 | 0.640 | 0.650 |
49
+ | | Creative Writing | 0.588 | 0.579 | 0.601 | 0.610 |
50
+ | | Dialogue Generation | 0.621 | 0.635 | 0.639 | 0.644 |
51
+ | | Summarization | 0.745 | 0.755 | 0.760 | 0.767 |
52
+ | **Specialized Capabilities**| Translation | 0.782 | 0.799 | 0.801 | 0.804 |
53
+ | | Knowledge Retrieval | 0.651 | 0.668 | 0.670 | 0.676 |
54
+ | | Instruction Following | 0.733 | 0.749 | 0.751 | 0.758 |
55
+ | | Safety Evaluation | 0.718 | 0.701 | 0.725 | 0.739 |
56
 
57
  </div>
58
 
59
  ### Overall Performance Summary
60
+ The MyAwesomeModel demonstrates strong performance across all evaluated benchmark categories, with particularly notable results in reasoning and generation tasks.
 
61
 
62
  ## 3. Chat Website & API Platform
 
63
  We offer a chat interface and API for you to interact with MyAwesomeModel. Please check our official website for more details.
64
 
65
  ## 4. How to Run Locally
 
84
  You are MyAwesomeModel, a helpful AI assistant.
85
  Today is May 28, 2025, Monday.
86
  ```
 
87
  ### Temperature
88
+ We recommend setting the temperature parameter $T_{model}$ to 0.6.
89
 
90
  ### Prompts for File Uploading and Web Search
91
  For file uploading, please follow the template to create prompts, where {file_name}, {file_content} and {question} are arguments.
 
118
  ```
119
 
120
  ## 5. License
 
121
  This code repository is licensed under the [MIT License](LICENSE). The use of MyAwesomeModel models is also subject to the [MIT License](LICENSE). The model series supports commercial use and distillation.
122
 
123
  ## 6. Contact
 
124
  If you have any questions, please raise an issue on our GitHub repository or contact us at contact@MyAwesomeModel.ai.
125
+ ```