Optimizing Response Speed for Efficient Text Generation

·

Matt Mireles I wanted to raise a question about response speed - what's the most effective way to optimize how quickly responses are generated and then pasted into whatever program I'm using at the time?

1

4 comments

Yousif Astarabadi
·
·
Hi Michael Britt, we don’t currently have a way to do this on the user’s end. I’m happy to set you up with a smaller, faster model. I will dm you.
Niket Desai
·
·
BTW — Highly recommend using DeepStructure to help out here for both model management and investigating the agent path.
Matt Mireles
·
·
Niket Desai link?
Matt Mireles
·
·
Michael Britt This is really on us, tbh. It's the #1 thing I think about.