florianvoss commited on
Commit
ad2fc62
·
verified ·
1 Parent(s): 290baaf

adapt model card to neat

Browse files
Files changed (1) hide show
  1. README.md +24 -67
README.md CHANGED
@@ -34,33 +34,25 @@ The following performance metrics were measured with an image and a text prompt
34
  |---|---|---|---|---|
35
  | gemma3-siglip448-a16w4 | A16W8/A16W4 | Modalix | 23.2 tokens/sec | 0.45 sec |
36
 
37
-
38
  ## Prerequisites
39
 
40
  To run this model, you need:
41
 
42
  1. **SiMa.ai Modalix Device**
43
- 2. **SiMa.ai CLI**: [Installed](https://docs.sima.ai/pages/sima_cli/main.html#installation) on your Modalix device.
44
- 3. **Hugging Face CLI**: For downloading the model.
 
45
 
46
  ## Installation & Deployment
47
 
48
  Follow these steps to deploy the model to your Modalix device.
49
 
50
- ### 1. Install LLiMa Demo Application
51
- > **Note:** This is a **one-time setup**. If you have already installed the LLiMa demo application (e.g. for another model), you can skip this step and continue with model download.
52
 
53
- On your Modalix device, install the LLiMa demo application using the `sima-cli`:
54
 
55
- ```bash
56
- # Create a directory for LLiMa
57
- cd /media/nvme
58
- mkdir llima
59
- cd llima
60
- # Install the LLiMa runtime code
61
- sima-cli install -v 2.1.0 tools/llima -t selection
62
- ```
63
- > **Note:** To only download the LLiMa runtime code, select **Demo Web App** when prompted.
64
 
65
  ### 2. Download the Model
66
 
@@ -83,86 +75,51 @@ scp -r gemma3-siglip448-a16w4 sima@<modalix-ip>:/media/nvme/llima/models/
83
 
84
  ```text
85
  /media/nvme/llima/
86
- ├── run.sh
87
  └── models/
88
  └── gemma3-siglip448-a16w4/ # The compiled model
89
  ```
90
 
91
  ## Usage
92
 
93
- ### Run the Application
94
 
95
- Navigate to the demo directory and start the application:
96
 
97
  ```bash
98
- cd /media/nvme/llima/
99
- ./run.sh
100
  ```
101
 
102
- The script will detect the installed model(s) and prompt you to select one.
103
 
104
- Once the application is running, open a browser and navigate to:
105
- ```text
106
- https://<modalix-ip>:5000/
107
  ```
108
- *Replace \<modalix-ip\> with the IP address of your Modalix device.*
109
 
110
- ### API Usage
111
 
112
- To use OpenAI-compatible API, run the model in API mode:
113
- ```bash
114
- llima run gemma3-siglip448-a16w4 --mode web
115
- ```
116
 
117
- You can interact with it using `curl` or Python.
118
 
119
- **Example: Chat Completion**
120
 
121
- ```bash
122
- # Note: Replace <YOUR_BASE64_STRING_HERE> with an actual base64 encoded image string.
123
- curl -N -k -X POST "https://<modalix-ip>:9998/v1/chat/completions" \
124
- -H "Content-Type: application/json" \
125
- -d '{
126
- "model": "sima-vlm",
127
- "stream": true,
128
- "messages": [
129
- {
130
- "role": "system",
131
- "content": "You are a helpful assistant."
132
- },
133
- {
134
- "role": "user",
135
- "content": [
136
- {
137
- "type": "image_url",
138
- "image_url": {
139
- "url": "data:image/jpeg;base64,<YOUR_BASE64_STRING_HERE>"
140
- }
141
- },
142
- {
143
- "type": "text",
144
- "text": "Describe the image in two sentences."
145
- }
146
- ]
147
- }
148
- ]
149
- }'
150
- ```
151
- *Replace \<modalix-ip\> with the IP address of your Modalix device.*
152
 
153
  ## Limitations
154
 
155
  - **Quantization**: This model is quantized (A16W4/A16W8) for optimal performance on embedded devices. While this maintains high accuracy, minor deviations from the full-precision model may occur.
156
  - **Fixed Resolution**: This version has been specifically optimized and fixed to **448x448** resolution at compile time to achieve maximum throughput and efficiency on the SiMa.ai MLA.
157
 
158
-
159
  ## Troubleshooting
160
 
161
- - **`sima-cli` not found**: Ensure that sima-cli is installed on your Modalix device.
 
162
  - **Model can't be run**: Verify the model directory is exactly inside `/media/nvme/llima/models/` and not nested (e.g., `/media/nvme/llima/models/gemma3-siglip448-a16w4/gemma3-siglip448-a16w4`).
163
  - **Permission Denied**: Ensure you have read/write permissions for the `/media/nvme` directory.
164
 
165
  ## Resources
166
 
167
- - [SiMa.ai Documentation](https://docs.sima.ai)
168
- - [SiMa.ai Hugging Face Organization](https://huggingface.co/simaai)
 
 
 
34
  |---|---|---|---|---|
35
  | gemma3-siglip448-a16w4 | A16W8/A16W4 | Modalix | 23.2 tokens/sec | 0.45 sec |
36
 
 
37
  ## Prerequisites
38
 
39
  To run this model, you need:
40
 
41
  1. **SiMa.ai Modalix Device**
42
+ 2. **SiMa.ai CLI**: Installed on your Modalix device.
43
+ 3. **SiMa.ai Neat Runtime**: Install or update the Neat Library on Modalix. The LLiMa runtime is installed as part of the Neat runtime.
44
+ 4. **Hugging Face CLI**: Optional, for downloading the model on a host before copying it to Modalix.
45
 
46
  ## Installation & Deployment
47
 
48
  Follow these steps to deploy the model to your Modalix device.
49
 
50
+ ### 1. Install or Update Neat Runtime
51
+ > **Note:** This is a **one-time setup**. If the Neat Library is already installed on your Modalix device, you can skip this step and continue with model download.
52
 
53
+ Follow the [SiMa.ai Neat getting started guide](https://developer.sima.ai/software/getting-started/) to install or update the Neat Library on your Modalix device.
54
 
55
+ The `llima` CLI is available on Modalix after the Neat runtime is installed. It manages precompiled GenAI models under `/media/nvme/llima/models` by default. Set `LLIMA_MODELS_PATH` to use a different model directory.
 
 
 
 
 
 
 
 
56
 
57
  ### 2. Download the Model
58
 
 
75
 
76
  ```text
77
  /media/nvme/llima/
 
78
  └── models/
79
  └── gemma3-siglip448-a16w4/ # The compiled model
80
  ```
81
 
82
  ## Usage
83
 
84
+ ### Validate with LLiMa CLI
85
 
86
+ Run the model directly on Modalix:
87
 
88
  ```bash
89
+ llima run gemma3-siglip448-a16w4
 
90
  ```
91
 
92
+ For all runtime options, run:
93
 
94
+ ```bash
95
+ llima run -h
 
96
  ```
 
97
 
98
+ ### GenAI Demo Application
99
 
100
+ The GenAI demo application is separate from LLiMa installation. Use the [GenAI Multimodal Assistant](https://developer.sima.ai/examples/app/genai%2Fmultimodal-assistant) page to install and run the demo app. Once installed, the demo app can use precompiled models such as this one.
 
 
 
101
 
102
+ ### API Usage
103
 
104
+ To serve this model with OpenAI- or Ollama-compatible APIs and send requests to it, use the GenAI server workflow in [Serve GenAI Models](https://developer.sima.ai/software/tutorials/serve-genai-models).
105
 
106
+ For direct VLM calls without setting up a server, see [Run a VLM](https://developer.sima.ai/software/tutorials/run-a-vlm).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
107
 
108
  ## Limitations
109
 
110
  - **Quantization**: This model is quantized (A16W4/A16W8) for optimal performance on embedded devices. While this maintains high accuracy, minor deviations from the full-precision model may occur.
111
  - **Fixed Resolution**: This version has been specifically optimized and fixed to **448x448** resolution at compile time to achieve maximum throughput and efficiency on the SiMa.ai MLA.
112
 
 
113
  ## Troubleshooting
114
 
115
+ - **`sima-cli` not found**: Ensure that `sima-cli` is installed on your Modalix device.
116
+ - **`llima` not found**: Install or update the Neat Library. See [Getting Started](https://developer.sima.ai/software/getting-started/).
117
  - **Model can't be run**: Verify the model directory is exactly inside `/media/nvme/llima/models/` and not nested (e.g., `/media/nvme/llima/models/gemma3-siglip448-a16w4/gemma3-siglip448-a16w4`).
118
  - **Permission Denied**: Ensure you have read/write permissions for the `/media/nvme` directory.
119
 
120
  ## Resources
121
 
122
+ - [GenAI with LLiMa](https://developer.sima.ai/software/genai-llima/)
123
+ - [Serve GenAI Models](https://developer.sima.ai/software/tutorials/serve-genai-models)
124
+ - [Run a VLM](https://developer.sima.ai/software/tutorials/run-a-vlm)
125
+ - [GenAI Multimodal Assistant](https://developer.sima.ai/examples/app/genai%2Fmultimodal-assistant)