Conversation Tool / facial animation

#1 You could do facial animation by using bones placed in the models head to deform th emesh and do jaw movements. Then in Torque you would just need to write code that would move the jaw bone based on the waveform of the audio file.

That's a basic way of doing things, you certainly could make it more elaborate from there.

Logan

#2 Thanks for the reply, guess that'll work with a little tweaking...
But the first and major step would be to build such a dialogue system I've mentioned... the characters actually need something to talk...
A comfortable editor would be needed here, where you can type in/save/edit all your dialogues, tie them to all your NPCs, trigger everything synchronized with the scripted actions in your game (assuming these are already scripted and setup).
Anybody working on such a tool for Torque? I guess I will be sooner or later, cause we will definitely need it for our singleplayer game... any volunteers out there joining me? ;-)

#3 Hm.... doesn't anybody else need dialogues for their game? Everybody working on Multiplayer/Bot only stuff then? What about the RPG guys out there?

#4 I'm interested in such a project, but I don't think I know enough about Torque to help you out. Sorry...

#5 the guy who made expression toolkit says on his site that hed cooperate with any attempts to port it to another platform. not that this solves your problem, but i'd like to see it ported to milkshape or whatever myself.

#6 Just a little update: the UNREAL engine now has the LIPSync SDK integrated directly into their UnrealED - man, it would be SO cool to have such a tool in the TORQUE editor... just imagine, you simply import a WAV file and it automatically generates facial animations from the data:

Quote:
...analyzes the speech and produces very realistic facial animation data that includes not only the mouth, jaw and lip movements but also eye blinks, eyebrow raises, and head nods. You can then preview it, right there in the animation browser, combined with whatever other character animation you want to play. For example you can have a character talking while doing a ladder-climbing animation and it looks very realistic.

Read the backgounds here...
But I don't have any clue of audio programming to extend the editor for this kind of stuff, so I guess I'll stick with the Expression Toolkit for now... :-)
But still, there is nothing like ConEd for Torque yet... maybe I can get something going in that direction - but that's an addon for the future I guess ... by the way, there really should be a team working on this, I think! Anybody interested?

#7 If you're so persistant that this is important, why don't YOU do it?

#8 Well, I said I'm interested in doing it, but I was hoping not to be the only one... ;-) will be a quite huge project, I guess... so any help would be great!
But it's not on top of my huge to-do list at the moment... ;-) (although it may seem so, hehe...)

#9 Logan said how to do it, and frankly it doesn't sound like that major of an undertaking.

Learn how to load and analyse the wavelengths of sounds, and how to manipulate the bones based on some actions taken by the sound processing code.

Simple, yet elegant!

If you really want to do this, you'll figure it out. Don't doubt yourself, and don't listen to anyone who doubts you (unless they backup their statements).

You can do it kid!

/me cues Rocky music

#10 Hey folks, I started working on a very basic concept for a Torque Conversation Editor (TGEConEd) - pretty much similar to DeusEx' ConEdit... just some ideas and possible "data structures" we could use (well, simple mission file entries how it might look) - it would be great if someone who is interested in the topic could check it out and provide some feedback/ideas/alternatives/nasty comments or whatever ;-)
Check it out here...
It's a zipped file containing an RTF document (and a txt file if you can't read RTFs), a pic of the DeusEx editor as well as a first screenshot of a first and very rough possible Torque GUI ...
Thanks!! :-)

#11 I think its a good plan so far ..
with the only real thing bugging me is the add to mission all this stuff.
that is no good I think, I would rather see a script for each level or something.
Lets keep the mission managable :)

and keep in mind how important it will be to have Anything related all tied into one editor.

its got a way to go in design but, tis a good plan :)

your best bet is to work towards getting a basic gui going with all the entry stuff and the actual system running .
stay away from the animation and sound for now :)

#12 Hey badguy,
thanks for reading this stuff :-) !
Well, you're right with *not* writing this to the mission files, maybe we should just put in something like:

new TConversation()
{
   TConFile = "~/data/conversations/mission01.tcon";
   preloadSounds = "true";
   foo = "bar";
   anything = "else";
};

and keep it seperate this way... could be pretty ugly otherwise, hehe... ;-)

So you think it would be a good way trying to set it up this way? I can't think of a better alternative yet (although there surely *is* one out there...).
Well, I guess I'll stick with this idea for now and, as you said, start off with getting the actual GUI to run, load, edit and save my custom files and maybe activate some text which is then printed on the screen.
The *next* step would be to add multiple choice dialogues which you can select with the mouse or keyboard and jump to the appropriate text marker/target...
AFTER that, I will take care for playing the sounds, setting camera perspectives, etc.
But I guess the first step is enough work for now :-)
Thanks again for your feedback!

#13 np, good game plan :)
P.S.
heh dont preload the sounds for this one :)

#14 Hehe, okay... but do you think it's okay to set "foo" to "bar"? ;-)
Also, I think there is no need to introduce a new file extension, is there? Guess I'll just stick with *.cs ...
Hope to finish the GUI / file editing stuff next week... I'll keep you informed! Cheers!

#15 Hey there, just a small update... work has begun on the TGEConEd, currently I've made some basic C++ classes providing the datablock functionality I need, I've finished a basic GUI (with switchable panes for different dialogue types, one for DefaultDialouges and one for MultipleChoiceDialogues - don't know if the structures/datablocks I'm thinking of at the moment make much sense, but that will evolve hopefully... the GUI actually already generates *.cs files with the appropriate datablocks and fills these with the values you put in the fields...
So the very basics are done, now comes the hard part -
finishing the GUI (load the scripts to edit them), processing the datablocks, set up the different triggers, render the dialogues in a cool GUI interface, make the multiple choice sections selectable, make the answers jump to the right targets,... you get the idea... so lots of work ahead... let's run away! ... just kiddin' ;-)
Btw., any comments, suggestions, ideas, etc. extremely welcome!
Later!

#16 Another quick update, just to proof that I'm working on this from time to time, hehe... ;-)
So, I've made *some* progress, I've extended my tgeConEd classes to provide the core methods to instantiate conversation sequence blocks (which are generated/saved by the TGEConEd GUI) and to get/set the basic fields (participant, text, label, etc.).
Then I've integrated David Myers' ObjectSelection code into the engine, which allowed me to test a first trigger event for the conversations: clicking on a bot with the mouse, which then (if it is really a bot you've clicked on and if his "nameBase" property matches the "Participant" entry in the TGEConEdSequence I've created for testing) pops up a dialog screen and for now only displays the first sequence text.

Up next:
- store all the dialogs/sequences in a global hashtable or something
- implement some "Conversation Manager" with an event-driven, message based system, which lets the bots/players/NPCs register themselves and maps everyone to his/her sequences, triggers/dispatches everything on request, and then keeps track of the conversations taking place, which would also involve some "save game" feature, which is yet to be implemented... ah well ;-) (btw. is anybody working on such a thing???)
- make the answers selectable, implement multiple choice dialogs, and finally adjust gameplay depending on the answers you give....

OMG, this will keep me busy for some months.... :-(

Anyhow, here's a screenshot of a bot talking to you after I clicked on him ;-) (the text is already coming from a sequence - *.cs file loaded at startup and previously generated with the GUI and "connected" to this "Bot0" via the "Participant" field) ... So you better listen now, hehe...

#17 Interesting stuff. You might want to take a look at Neverwinter Nights. They have a robust conversation editor that struck me as much more user friendly than ConEd.

Vic-D

#18 This looks like an awesome tool! I'm impressed with your progress. (This could also be a nifty thing to have powered by a database)

Perhaps you could use a tree view to organize the conversation?

#19 Hey Ben! Thanks a lot! :) I was planning to put it in a tree view, but I couldn't get it to work ... :/ but I sure will try that again when I find the time...
Combining it with a database is an awesome idea, never thought about that! Would be easier than this file stuff actually... I think I will do that, I've got mySQL support in the engine anyways!
Thanks for the tip! :D