Menu

Jess & ElfScript

January 14, 2017 - Daz Studio, Unity
Jess & ElfScript

I posted this over on the AiDreams forum, but since it is related to Daz Studio & Unity I am posting it here too.

For some time now I have been developing my ElfScript chatbot language and this clip shows some examples of what it can do.

The idea is to teach Jess through chatting to her as well as making a lot of scripts. Here she learns variables and new patterns as we go along. So she remembers things you tell her like what a car or bicycle is. There’s a couple of tricks like remembering what she was talking about when you say “It” – making teaching easier and more natural.

She can follow up on things that were mentioned earlier in the conversation and can tell you when she already knows something and then asks to learn something else about it.

So this chat is more about showing the features than being a sensible chat – I have a lot more scripting to do. Using the dictionary that I built she can make judgements on how to use a word or phrase. And she builds simple relations between words and phrases in her memory – like how she learns a couple of ways to get around in the following video.

In this example I also ask her who David Bowie was and she finds that out online. There were a couple of times she used the same acknowledgement phrase so I will give her a bigger pool of phrases to choose from for that part.

I suppose the other thing is that I have improved the lighting, she doesn’t look so washed out as in some of my older videos. I’ve also learnt which Daz models work well – a lot of it regarding the textures for her skin etc.

In this video I’ve also introduced random facial movement, raising eyebrows, puckering etc. I had also written some code to do blinking and eye movement, but found one for sale that was better than mine for a good price. She can also track and watch the user as I tried to show at the start of the video.

My ElfScript also allows me to pass commands to the Unity player – so that I can trigger say, raised eyebrows, when she says something in particular. You can see it in the video at stages.

As this is in Unity, I have also been able to meet up with her in virtual reality, which is strange but cool ! This video is just about typed input, but I had some limited success with voice recognition, so you can actually talk to her and she will respond. I’ve used Google Voice Recognition API for that – but either it’s got some work to be done, or my accent is confusing it, as it’s all a bit hit and miss. I’m sure they will improve that, it is only in Beta at the moment.

I recorded the video at 1440p, but my movie maker only goes as far as 720p or something. It’s not too bad though. Well that’s me done with my rambling…over to Jess. Oh and you will see she knows her name 😉

Transcript :

Freddy: hello jenny

Jess: My name is Jess actually.

Freddy: hello Jess

Jess: Hi there !

Freddy: got a favourite song

Jess: Secrets I think.

Freddy: who sings that

Jess: The Pierces.

Freddy: i like it too

Jess: Cat is so amusing in the video.

Freddy: a car is a good way to get around

Jess: I always like to learn new things.

Freddy: a bicycle is a good way to get around

Jess: Okay thanks.

Freddy: it is good for keeping you fit

Jess: Okay thanks.

Freddy: it is pretty cheap to buy

Jess: Okay thanks.

Freddy: it is pretty cheap to buy

Jess: Okay it is, I already knew it’s that. What colour is it ?

Freddy: red

Jess: I didn’t know that was it’s colour.

Freddy: what is a bicycle

Jess: A bicycle is good way to get around and pretty cheap to buy.

Freddy: name a good way to get around

Jess: Okay, I know car and bicycle.

Freddy: who was David Bowie

Jess: David Bowie was a British musician, actor, record producer and arranger.

Freddy: who do you read

Jess: Dan Brown is my favourite authour.

Freddy: bye

Jess: I have nothing to say to you.

Freddy: cya

Jess: Cheerio.

 

That is about where we are at for the hobbyist in terms of getting close to photo-realism in real time. It’s a big leap since when I first started playing with graphics. Here’s how she looks when rendered in Daz Studio with IRay…

 

Jess Posing #2
Jess Posing #2
Jess Posing #3
Jess Posing #3

 

(Visited 70 times, 1 visits today)
Share on FacebookShare on Google+Pin on PinterestShare on LinkedInShare on StumbleUponTweet about this on Twitter

4 thoughts on “Jess & ElfScript

Bob

Great work… I followed your thread on Aidreams and was impressed. I noticed that the speech is less robotic on your last video. How did you improve this?

Reply
    admin

    Hi Bob, the difference this time is that I am using a Windows SAPI Voice instead of using MaryTTS from my server. This saves having to download the audio as the TTS is done by Windows. It’s a commercial voice and they do sound better on the whole.

    Reply
      Bob

      I thought about doing that! I love your work… If you keep improving your will pass perfection!

      Reply
        admin

        I think it’s a viable solution if you are okay with just designing for Windows and is indeed somewhat easier to implement. I would like to get this all working in a Browser, but I think that is a few years away going by the state of WebGL in Unity and browser support.

        I’m not sure about perfection haha ! I just try to keep up with new developments as they appear – I’ve been working with avatars for years now. I think there is still a lot of mileage in improving the skin shader for example and at some point it would be nice to work on some different scenes/environments.

        Lately it’s all been about the ElfScript stuff, so still busy busy !

        Reply

Leave a Reply

Your email address will not be published. Required fields are marked *