florianvoss commited on
Commit
d803c4b
·
verified ·
1 Parent(s): 1dac84c

adapt model card to neat

Browse files
Files changed (1) hide show
  1. README.md +24 -68
README.md CHANGED
@@ -36,33 +36,25 @@ The following performance metrics were measured with an image and a text prompt
36
  |---|---|---|---|---|
37
  | LFM2-VL-450M-a16w4 | A16W8/A16W4 | Modalix | 221.35 tokens/sec | 0.11 sec |
38
 
39
-
40
  ## Prerequisites
41
 
42
  To run this model, you need:
43
 
44
  1. **SiMa.ai Modalix Device**
45
- 2. **SiMa.ai CLI**: [Installed](https://docs.sima.ai/pages/sima_cli/main.html#installation) on your Modalix device.
46
- 3. **Hugging Face CLI**: For downloading the model.
 
47
 
48
  ## Installation & Deployment
49
 
50
  Follow these steps to deploy the model to your Modalix device.
51
 
52
- ### 1. Install LLiMa Demo Application
53
- > **Note:** This is a **one-time setup**. If you have already installed the LLiMa demo application (e.g. for another model), you can skip this step and continue with model download.
54
 
55
- On your Modalix device, install the LLiMa demo application using the `sima-cli`:
56
 
57
- ```bash
58
- # Create a directory for LLiMa
59
- cd /media/nvme
60
- mkdir llima
61
- cd llima
62
- # Install the LLiMa runtime code
63
- sima-cli install -v 2.1.0 tools/llima -t selection
64
- ```
65
- > **Note:** To only download the LLiMa runtime code, select **Demo Web App** when prompted.
66
 
67
  ### 2. Download the Model
68
 
@@ -85,87 +77,51 @@ scp -r LFM2-VL-450M-a16w4 sima@<modalix-ip>:/media/nvme/llima/models/
85
 
86
  ```text
87
  /media/nvme/llima/
88
- ├── run.sh
89
  └── models/
90
  └── LFM2-VL-450M-a16w4/ # The compiled model
91
  ```
92
 
93
  ## Usage
94
 
95
- ### Run the Application
96
 
97
- Navigate to the demo directory and start the application:
98
 
99
  ```bash
100
- cd /media/nvme/llima/
101
- ./run.sh
102
  ```
103
 
104
- The script will detect the installed model(s) and prompt you to select one.
105
 
106
- Once the application is running, open a browser and navigate to:
107
- ```text
108
- https://<modalix-ip>:5000/
109
  ```
110
- *Replace \<modalix-ip\> with the IP address of your Modalix device.*
111
 
112
- ### API Usage
113
 
114
- To use OpenAI-compatible API, run the model in API mode:
115
- ```bash
116
- llima run LFM2-VL-450M-a16w4 --mode web
117
- ```
118
 
119
- You can interact with it using `curl` or Python.
120
 
121
- **Example: Chat Completion**
122
 
123
- ```bash
124
- # Note: Replace <YOUR_BASE64_STRING_HERE> with an actual base64 encoded image string.
125
- curl -N -k -X POST "https://<modalix-ip>:9998/v1/chat/completions" \
126
- -H "Content-Type: application/json" \
127
- -d '{
128
- "model": "sima-vlm",
129
- "stream": true,
130
- "messages": [
131
- {
132
- "role": "system",
133
- "content": "You are a helpful assistant."
134
- },
135
- {
136
- "role": "user",
137
- "content": [
138
- {
139
- "type": "image_url",
140
- "image_url": {
141
- "url": "data:image/jpeg;base64,<YOUR_BASE64_STRING_HERE>"
142
- }
143
- },
144
- {
145
- "type": "text",
146
- "text": "Describe the image in two sentences."
147
- }
148
- ]
149
- }
150
- ]
151
- }'
152
- ```
153
- *Replace \<modalix-ip\> with the IP address of your Modalix device.*
154
 
155
  ## Limitations
156
 
157
  - **Quantization**: This model is quantized (A16W4/A16W8) for optimal performance on embedded devices. While this maintains high accuracy, minor deviations from the full-precision model may occur.
158
  - **Fixed Resolution**: While the standard LFM2-VL architecture supports dynamic input resolutions, this version has been specifically optimized and fixed to **512x512** resolution at compile time to achieve maximum throughput and efficiency on the SiMa.ai MLA.
159
 
160
-
161
  ## Troubleshooting
162
 
163
- - **`sima-cli` not found**: Ensure that sima-cli is installed on your Modalix device.
 
164
  - **Model can't be run**: Verify the model directory is exactly inside `/media/nvme/llima/models/` and not nested (e.g., `/media/nvme/llima/models/LFM2-VL-450M-a16w4/LFM2-VL-450M-a16w4`).
165
  - **Permission Denied**: Ensure you have read/write permissions for the `/media/nvme` directory.
166
 
167
  ## Resources
168
 
169
- - [SiMa.ai Documentation](https://docs.sima.ai)
170
- - [SiMa.ai Hugging Face Organization](https://huggingface.co/simaai)
171
- - [Liquid AI Website](https://liquid.ai/)
 
 
36
  |---|---|---|---|---|
37
  | LFM2-VL-450M-a16w4 | A16W8/A16W4 | Modalix | 221.35 tokens/sec | 0.11 sec |
38
 
 
39
  ## Prerequisites
40
 
41
  To run this model, you need:
42
 
43
  1. **SiMa.ai Modalix Device**
44
+ 2. **SiMa.ai CLI**: Installed on your Modalix device.
45
+ 3. **SiMa.ai Neat Runtime**: Install or update the Neat Library on Modalix. The LLiMa runtime is installed as part of the Neat runtime.
46
+ 4. **Hugging Face CLI**: Optional, for downloading the model on a host before copying it to Modalix.
47
 
48
  ## Installation & Deployment
49
 
50
  Follow these steps to deploy the model to your Modalix device.
51
 
52
+ ### 1. Install or Update Neat Runtime
53
+ > **Note:** This is a **one-time setup**. If the Neat Library is already installed on your Modalix device, you can skip this step and continue with model download.
54
 
55
+ Follow the [SiMa.ai Neat getting started guide](https://developer.sima.ai/software/getting-started/) to install or update the Neat Library on your Modalix device.
56
 
57
+ The `llima` CLI is available on Modalix after the Neat runtime is installed. It manages precompiled GenAI models under `/media/nvme/llima/models` by default. Set `LLIMA_MODELS_PATH` to use a different model directory.
 
 
 
 
 
 
 
 
58
 
59
  ### 2. Download the Model
60
 
 
77
 
78
  ```text
79
  /media/nvme/llima/
 
80
  └── models/
81
  └── LFM2-VL-450M-a16w4/ # The compiled model
82
  ```
83
 
84
  ## Usage
85
 
86
+ ### Validate with LLiMa CLI
87
 
88
+ Run the model directly on Modalix:
89
 
90
  ```bash
91
+ llima run LFM2-VL-450M-a16w4
 
92
  ```
93
 
94
+ For all runtime options, run:
95
 
96
+ ```bash
97
+ llima run -h
 
98
  ```
 
99
 
100
+ ### GenAI Demo Application
101
 
102
+ The GenAI demo application is separate from LLiMa installation. Use the [GenAI Multimodal Assistant](https://developer.sima.ai/examples/app/genai%2Fmultimodal-assistant) page to install and run the demo app. Once installed, the demo app can use precompiled models such as this one.
 
 
 
103
 
104
+ ### API Usage
105
 
106
+ To serve this model with OpenAI- or Ollama-compatible APIs and send requests to it, use the GenAI server workflow in [Serve GenAI Models](https://developer.sima.ai/software/tutorials/serve-genai-models).
107
 
108
+ For direct VLM calls without setting up a server, see [Run a VLM](https://developer.sima.ai/software/tutorials/run-a-vlm).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
109
 
110
  ## Limitations
111
 
112
  - **Quantization**: This model is quantized (A16W4/A16W8) for optimal performance on embedded devices. While this maintains high accuracy, minor deviations from the full-precision model may occur.
113
  - **Fixed Resolution**: While the standard LFM2-VL architecture supports dynamic input resolutions, this version has been specifically optimized and fixed to **512x512** resolution at compile time to achieve maximum throughput and efficiency on the SiMa.ai MLA.
114
 
 
115
  ## Troubleshooting
116
 
117
+ - **`sima-cli` not found**: Ensure that `sima-cli` is installed on your Modalix device.
118
+ - **`llima` not found**: Install or update the Neat Library. See [Getting Started](https://developer.sima.ai/software/getting-started/).
119
  - **Model can't be run**: Verify the model directory is exactly inside `/media/nvme/llima/models/` and not nested (e.g., `/media/nvme/llima/models/LFM2-VL-450M-a16w4/LFM2-VL-450M-a16w4`).
120
  - **Permission Denied**: Ensure you have read/write permissions for the `/media/nvme` directory.
121
 
122
  ## Resources
123
 
124
+ - [GenAI with LLiMa](https://developer.sima.ai/software/genai-llima/)
125
+ - [Serve GenAI Models](https://developer.sima.ai/software/tutorials/serve-genai-models)
126
+ - [Run a VLM](https://developer.sima.ai/software/tutorials/run-a-vlm)
127
+ - [GenAI Multimodal Assistant](https://developer.sima.ai/examples/app/genai%2Fmultimodal-assistant)