Initial README

This commit is contained in:
Angel-HF
2024-04-03 20:15:06 +02:00
committed by GitHub
parent dfa1592b2c
commit 15311c3a13

View File

@@ -1,3 +1,32 @@
# ezrknn-toolkit2
This repo tries to make RKNN LLM usage easier
## Requirements
Keep in mind this repo is focused for:
- High-end Rockchip SoCs, mainly the RK3588
- Linux, not Android
- Linux kernels from Rockchip (as of writing 5.10 and 6.1 from Rockchip should work, if your board has one of these it will very likely be Rockchip's kernel)
## Quick Install
Run:
```bash
# TODO
```
## Test
Run (cd is required):
```bash
# TODO
```
# Original README starts below
<hr>
<hr>
<hr>
# Description # Description
RKLLM software stack can help users to quickly deploy AI models to Rockchip chips. The overall framework is as follows: RKLLM software stack can help users to quickly deploy AI models to Rockchip chips. The overall framework is as follows:
<center class="half"> <center class="half">
@@ -36,4 +65,4 @@ Please ensure to download a version of the [Phi2](https://hf-mirror.com/microsof
- Supports the conversion and deployment of LLM models on RK3588/RK3576 platforms - Supports the conversion and deployment of LLM models on RK3588/RK3576 platforms
- Compatible with Hugging Face model architectures - Compatible with Hugging Face model architectures
- Currently supports the models LLaMA, Qwen, Qwen2, and Phi-2 - Currently supports the models LLaMA, Qwen, Qwen2, and Phi-2
- Supports quantization with w8a8 and w4a16 precision - Supports quantization with w8a8 and w4a16 precision