mirror of
https://github.com/DrHo1y/ezrknn-llm.git
synced 2026-03-24 01:26:44 +07:00
Initial README
This commit is contained in:
31
README.md
31
README.md
@@ -1,3 +1,32 @@
|
|||||||
|
# ezrknn-toolkit2
|
||||||
|
This repo tries to make RKNN LLM usage easier
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
Keep in mind this repo is focused for:
|
||||||
|
- High-end Rockchip SoCs, mainly the RK3588
|
||||||
|
- Linux, not Android
|
||||||
|
- Linux kernels from Rockchip (as of writing 5.10 and 6.1 from Rockchip should work, if your board has one of these it will very likely be Rockchip's kernel)
|
||||||
|
|
||||||
|
## Quick Install
|
||||||
|
Run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# TODO
|
||||||
|
```
|
||||||
|
|
||||||
|
## Test
|
||||||
|
Run (cd is required):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# TODO
|
||||||
|
```
|
||||||
|
|
||||||
|
# Original README starts below
|
||||||
|
|
||||||
|
<hr>
|
||||||
|
<hr>
|
||||||
|
<hr>
|
||||||
|
|
||||||
# Description
|
# Description
|
||||||
RKLLM software stack can help users to quickly deploy AI models to Rockchip chips. The overall framework is as follows:
|
RKLLM software stack can help users to quickly deploy AI models to Rockchip chips. The overall framework is as follows:
|
||||||
<center class="half">
|
<center class="half">
|
||||||
@@ -36,4 +65,4 @@ Please ensure to download a version of the [Phi2](https://hf-mirror.com/microsof
|
|||||||
- Supports the conversion and deployment of LLM models on RK3588/RK3576 platforms
|
- Supports the conversion and deployment of LLM models on RK3588/RK3576 platforms
|
||||||
- Compatible with Hugging Face model architectures
|
- Compatible with Hugging Face model architectures
|
||||||
- Currently supports the models LLaMA, Qwen, Qwen2, and Phi-2
|
- Currently supports the models LLaMA, Qwen, Qwen2, and Phi-2
|
||||||
- Supports quantization with w8a8 and w4a16 precision
|
- Supports quantization with w8a8 and w4a16 precision
|
||||||
|
|||||||
Reference in New Issue
Block a user