I try all the things to get Vicuna-13B-v1.5 deployed on an NVIDIA T4 so you don't have to.
Yeah, tgi took a bit of figuring out and you kind of have to get deep in the code. This vid I made may help a little: https://youtu.be/Ror2xOOA-VE?si=7u09EwZ0xYbShRQb
Curious why Vicuna and not Llama 2?
Yeah, tgi took a bit of figuring out and you kind of have to get deep in the code. This vid I made may help a little: https://youtu.be/Ror2xOOA-VE?si=7u09EwZ0xYbShRQb
Curious why Vicuna and not Llama 2?