Home
Creators
Search Recent Random
Posts
Search DMs Popular Hash Lookup Tags Random
Importer
Import FAQ
Filehaus
Recent
Register Favorites Keys Review DMs Logout
Home Creators Posts Import Register Favorites Logout

What do you use for offline text generation? (Patreon)

Published:
2024-06-06 20:55:59
Imported:
2026-06
Tags:

Poll

What do you use for offline text generation?

  • LM Studio 11
  • Oobabooga 22
  • Kobold 16
  • NovelAI 51
  • Other (specify below) 11
  • 2024-06-06
  • 111 votes
{'title': 'What do you use for offline text generation?', 'choices': [{'text': 'LM Studio', 'votes': 11}, {'text': 'Oobabooga', 'votes': 22}, {'text': 'Kobold', 'votes': 16}, {'text': 'NovelAI', 'votes': 51}, {'text': 'Other (specify below)', 'votes': 11}], 'closes_at': None, 'created_at': datetime.datetime(2024, 6, 6, 20, 46, 59, tzinfo=datetime.timezone.utc), 'description': None, 'allow_multiple': False, 'total_votes': 111}

Content

I am gearing up to provide offline local LLM support and i'd like to support the current most popular tools for: text generation and hopefully voice generation.

Comments

DatGai

Only muddled around for a bit with NovelAI but seen many results from others using it that I forgot that other models exists out there. Whoops lol. Curious to see what you'll go with and how well it will integrate with the game.

sgthale
>>140138310

Yeah the challenging part is just the voice gen. Really considering adding a chirp or Undertale style talking.

VR AnimeTed

lm studio is pretty good for story stuff with an easy install process, ollama has become insanely easy to use with a lot of people using it as a base

Benjamin Allen

I use all of the ones you mention, but NovelAI allows for API use for my own dev projects so it's the one I've been using mostly, though not local per se, they encrypt the data and give you that API use bonus, so I'd be using it most likely with MyRobot or other companions you might be making in the future. The version 2 voice generation NAI API has is pretty great too, but no settings. You give it a seed and hope it's the right voice, though you can mix seeds and separate into 3 voice qualities in the payload and build a dictionary of seeds to get the right gender and quality. Also, you may want to have a different text for what they say vs printed because inflection is controlled in a weird way with special characters, punctuation, and capitalization, so I have sometimes write the text said as a printable version and the version I use to control inflection, but even breath, mumbling, and stuttering is supported. Just an example of my case use, I can get somewhat decent [conversation/read time] streamed results with Mini Orca and Phi-3 Mini Instruct on my RTX 2060 and have "meh" to fair, limited 3D performance, pretty good in between generations but it bogs things down during gen. Maybe it would be doable on a more modern GPU on my part, but I don't think mini Orca and phi-3 mini would be good unless you had a specifically trained version of them as they are tiny compared to LLM that can make immersive, steerable(?) results, while giving decent 3D. I might be missing something though. Sorry, I know no one asked but this particular thing is both interesting and important to me in products I have interest in and my own current project, LULZ. Pardon my rambling.

Ege

can I have the discord link?

Marc

I've used Convai.com - which allows you to upload almost unlimited text documents to train a character on. It is made for game creators so I was wondering if that's what you're using? I think it works really well (at least by itself) and has an emotion engine built in. I haven't attached it to any sort of game character, but eventually I'd like to. Or at least some kind of nice interface. The character I made is for a stop motion series where the character resembles an AI but she's animated by hand. The Convai.com character is trained on the back story as well as lots of real world stop motion how-to information. If you're interested this is a link to the chatbot online. It's something you might want to experiment with if you don't already know about it: https://convai.com/shared-character?id=98f1a66c-2372-11ee-b429-42010a400002