mirror of
https://github.com/DrHo1y/ezrknn-llm.git
synced 2026-03-23 09:06:47 +07:00
Initial README
This commit is contained in:
31
README.md
31
README.md
@@ -1,3 +1,32 @@
|
||||
# ezrknn-toolkit2
|
||||
This repo tries to make RKNN LLM usage easier
|
||||
|
||||
## Requirements
|
||||
Keep in mind this repo is focused for:
|
||||
- High-end Rockchip SoCs, mainly the RK3588
|
||||
- Linux, not Android
|
||||
- Linux kernels from Rockchip (as of writing 5.10 and 6.1 from Rockchip should work, if your board has one of these it will very likely be Rockchip's kernel)
|
||||
|
||||
## Quick Install
|
||||
Run:
|
||||
|
||||
```bash
|
||||
# TODO
|
||||
```
|
||||
|
||||
## Test
|
||||
Run (cd is required):
|
||||
|
||||
```bash
|
||||
# TODO
|
||||
```
|
||||
|
||||
# Original README starts below
|
||||
|
||||
<hr>
|
||||
<hr>
|
||||
<hr>
|
||||
|
||||
# Description
|
||||
RKLLM software stack can help users to quickly deploy AI models to Rockchip chips. The overall framework is as follows:
|
||||
<center class="half">
|
||||
@@ -36,4 +65,4 @@ Please ensure to download a version of the [Phi2](https://hf-mirror.com/microsof
|
||||
- Supports the conversion and deployment of LLM models on RK3588/RK3576 platforms
|
||||
- Compatible with Hugging Face model architectures
|
||||
- Currently supports the models LLaMA, Qwen, Qwen2, and Phi-2
|
||||
- Supports quantization with w8a8 and w4a16 precision
|
||||
- Supports quantization with w8a8 and w4a16 precision
|
||||
|
||||
Reference in New Issue
Block a user