SilverStraw

There are so many features to add, so I am going to implement them one at a time.

Misaki, have you played with the drop down box below the Reset Render Camera button on the Model Settings? I might have not made it clear that drop down is where you change certain properties about the model(s).
SilverStraw
  • Messaggi: 79

Misaki

There are so many features to add, so I am going to implement them one at a time.
Sure, take your time :)
Misaki, have you played with the drop down box below the Reset Render Camera button on the Model Settings? I might have not made it clear that drop down is where you change certain properties about the model(s).
I have tried some of them! The position and scale of the model can be changed on the canvas with the mouse (this is really comfortable!), so I have not had to change them using the settings box, though.
Avatar utente
Misaki

Misaki
  • Messaggi: 858

SilverStraw

The scale and position really comes to play when you add more than one Spine model. With my recent unreleased testing, I was able to have 3 faces tracked on the same camera. There is a way to assign each tracked face to each Spine model. All the models start at 0, 0 position. You would not want overlapping models when you are using multi-face tracking. The scaling function helps when the models are different sizes. While over the canvas, your mouse only changes the viewing port. The model's scaling is still at 1 scale for x and y.

I wonder what you think about skins and animation properties if you tried them.
SilverStraw
  • Messaggi: 79

Misaki

While over the canvas, your mouse only changes the viewing port. The model's scaling is still at 1 scale for x and y.
Ah, I see. I'm looking forward to seeing what fun looks like when multiple models are placed.

Regarding animation property, what is the middle box set for? I can see that changing the animation of the pull down changes the animation, but is it played on track 0?
Screen Shot 2022-07-13 at 9.20.29.png


Regarding skin property, I can confirm that it works correctly, but is it currently only possible to apply one skin?
Non hai i permessi necessari per visualizzare i file allegati in questo messaggio.
Avatar utente
Misaki

Misaki
  • Messaggi: 858

SilverStraw

The middle box is for inputting values and is not used for the animation property. That box is for the other properties. Certain properties would use either the value input or the drop-drown list. Since properties like skin and animation have a finite number of choices, a drop-down list would be more appropriate. If value input box is too confusing, I could hide it for properties that do not need it.

I forgot to update the track layer for the animation property so it is not working on the intended track layer. It is in the middle of the track stack so you have half the tracks overriding it. :D

So far you can only apply a single skin. I only worked out populating the drop-down list of all skins. I never had the intent of multiple skins in early development but it seems like something I could expand upon.
SilverStraw
  • Messaggi: 79

Misaki

Thank you for elaborating on those! :)
If value input box is too confusing, I could hide it for properties that do not need it.
Yes, I think the value input box should be hidden as it would mislead people into thinking it is something that is meant to set up the animation.
I forgot to update the track layer for the animation property so it is not working on the intended track layer. It is in the middle of the track stack so you have half the tracks overriding it. :D
Ah, that makes sense. I had a feeling that something was wrong. It would be useful if it could be used to switch from the default breathing animation to another, or to switch facial expressions.
So far you can only apply a single skin. I only worked out populating the drop-down list of all skins. I never had the intent of multiple skins in early development but it seems like something I could expand upon.
Yes, it would be useful to be able to use more than one skin, such as one for costumes and one for expressions or postures, since skins can be used to change facial expressions and postures using constraints.
Avatar utente
Misaki

Misaki
  • Messaggi: 858

SilverStraw

Expanding on more animation tracks and skins add another dimension of complexity. I will need to figure how to expand on those features.
It would be useful if it could be used to switch from the default breathing animation to another, or to switch facial expressions.
Yes, it would be useful to be able to use more than one skin, such as one for costumes and one for expressions or postures, since skins can be used to change facial expressions and postures using constraints.
It will probably a good idea to get animation and skin properties their own sections in the future. They are getting too complicated to be used as a single input setting. If this keeps up, I will end up with a Spine visual programming web application. :scared:

I was hoping to soft cap the amount of animation tracks and skin layers. If I follow your suggestion, everything would need to be uncapped ( or until you web browser crashes :p ). So far I am using 20 something animation tracks just for face tracking.

On a side note, this topic got so many views within a day. I checked my itch.io stats and there hasn't been that many downloads on the test model. So many lurkers among us or it is just you, Misaki. :o
SilverStraw
  • Messaggi: 79

Misaki

I give you my suggestions on many things, but of course this is your tool, so make it what you want it to be :)
I was hoping to soft cap the amount of animation tracks and skin layers. If I follow your suggestion, everything would need to be uncapped ( or until you web browser crashes :p ). So far I am using 20 something animation tracks just for face tracking.
I'm not familiar with the cost of having a lot of tracks, so it would be better to have Nate or Mario comment on this.
On a side note, this topic got so many views within a day. I checked my itch.io stats and there hasn't been that many downloads on the test model. So many lurkers among us or it is just you, Misaki. :o
I’m sure I'm not the only one checking this topic! The order of topics will go up when there are responses in this forum, and a lot of people see the posts at the top. That’s why there are a lot of views.
Avatar utente
Misaki

Misaki
  • Messaggi: 858

SilverStraw

I give you my suggestions on many things, but of course this is your tool, so make it what you want it to be :)
Your suggestions are really important to me because it is the feedback that I have gotten. I am going to slowly implement them because I have no idea what I am doing :lol:. It is uncharted territory.
I'm not familiar with the cost of having a lot of tracks, so it would be better to have Nate or Mario comment on this.
Soft capping is really just less work for me :grinteeth:. For example, with only one animation or skin I do not have implement a more complex system. I will get to it eventually.
I’m sure I'm not the only one checking this topic! The order of topics will go up when there are responses in this forum, and a lot of people see the posts at the top. That’s why there are a lot of views.
I wish the feedback was that popular.

For all those lurking and have Discord, I am Aestos on unofficial Spine Discord server. :think:
SilverStraw
  • Messaggi: 79

Misaki

It has been a while, but I have made various minor modifications to my skeleton. The latest version of the recording is here:

As for the old videos, I had set them to limited public access, but I have set the latest video to public access. For people who are new to this tool, I have recorded the process from uploading the skeleton data so that it can be a simplified tutorial.

My skeleton still has some issues as the clipping attachments for the eyes sometimes go wrong, but I have attached the latest file here for your reference:
face-for-Spine-Vtuber-Prototype.zip

(Also, I deleted old files in this thread.)

I will come back to this thread when I have time. Cheers! :beer:
Non hai i permessi necessari per visualizzare i file allegati in questo messaggio.
Avatar utente
Misaki

Misaki
  • Messaggi: 858

SilverStraw



https://silverstraw.itch.io/spine-vtuber-prototype
https://silverstraw.itch.io/spine-vtube-test-model

1.0.3
  • Rearrange model property drop-down list.
  • Add "maximum number faces" option to model property drop-down list. This property will allow more than one face to be tracked using a single web camera.
  • Add "Single Value Properties" label to the left of model property drop-down list. It should be clearer about the drop-down list if it was not before.
  • Update the track layer for the model property "animation" option".
  • Hide value input box for animation and skin properties.
  • Add setting for "face pitch strength", "face yaw strength", "face roll strength", "mouth height strength", "mouth width strength", "left brow strength", "right brow strength", "left eye strength", "right eye strength", "left pupil pitch strength", "left pupil yaw strength", "right pupil pitch strength", "right pupil yaw strength" next to "Single Value Properties".

---

Misaki ha scritto:My skeleton still has some issues as the clipping attachments for the eyes sometimes go wrong, but I have attached the latest file here for your reference:
Hello. What is wrong with the clipping attachments for the eyes?
Non hai i permessi necessari per visualizzare i file allegati in questo messaggio.
SilverStraw
  • Messaggi: 79

Misaki

That sounds like a great update! However, I could not find where the strength setting was. I don't see the drop down menu in your video, is it visible on your end?
Hello. What is wrong with the clipping attachments for the eyes?
When the eyes were closed, the clipping masks sometimes crossed over, and this caused the eyes that should have been hidden to be visible. I fixed the problem today and I replaced the attached file on my previous reply.
Avatar utente
Misaki

Misaki
  • Messaggi: 858

SilverStraw

Misaki ha scritto:That sounds great update! However, I could not find where Add strength setting was. I don't see the drop down menu in your video, is it visible on your end?
I meant that I added the setting for the user to change "face pitch", "face yaw", "face roll", "mouth height, "mouth width", "left brow", "right brow", "left eye", "right eye", "left pupil pitch", "left pupil roll", "right pupil pitch", "right pupil roll" next to property label. I apologize that Open Broadcast Studio did not capture any pop-up menus when I recorded the video. The drop-down does appear at my end but I did not full screen capture the process.
SilverStraw
  • Messaggi: 79

Misaki

Hmm, somehow I can't find that setting on my end....
Screen Shot 2022-07-22 at 9.43.48.png

Also, I can't find "maximum number faces", so the update seems not to be reflected properly. Is there anything I need to do to use the updated version?
Non hai i permessi necessari per visualizzare i file allegati in questo messaggio.
Avatar utente
Misaki

Misaki
  • Messaggi: 858

SilverStraw

Also, I can't find "maximum number faces", so the update seems not to be reflected properly. Is there anything I need to do to use the updated version?
I apologize; mistake is made at my end :tear: . I forgot to rename the new web file as "index". In effect, itch.io read the old web file and ignored the new. I have corrected the name and should reflect the updated interface. Good thing you mentioned it :handshake: .
SilverStraw
  • Messaggi: 79

Misaki

Sorry for the late response, thank you for reflecting the updated interface! I have tried to do the multiple face tracking, and the following video is the result (Sorry for the video looks a bit confusing because the position of the model and the face being tracked are reversed):

It worked at first, but it stopped after about 20 seconds :think:
The tracking itself seemed accurate while it was working, so I’m very impressed when I saw it for the first time! I don't know the cause why it would stop, but it would be a very interesting feature if it could be made to work consistently.

I haven't tried the other settings yet, so I'll try them later!
Avatar utente
Misaki

Misaki
  • Messaggi: 858

SilverStraw

I have not managed to break the multiple face tracking and replicate the bug in your video. This is something that would require further investigating. I would like to know if you could replicate this bug repeatedly. It could be either hardware or software that is causing the bug. For one, the camera only allow one application to read the web camera stream data from what I have tried on Windows. Since the web application does not write any log files onto your computer or communicate to a server, it will be harder to pinpoint the cause of the bug.
SilverStraw
  • Messaggi: 79

Misaki

I would like to know if you could replicate this bug repeatedly.
Yes, I was able to replicate this bug again. I think it's possible that I have the wrong settings, so I recorded the process from when I accessed the tool:

One thing I noticed is that the model that was initially tracking my movement will follow the second face’s movement when the second face tracking starts. Is this something wrong with my settings?
Avatar utente
Misaki

Misaki
  • Messaggi: 858

SilverStraw

Can you open up the web inspector ( should be F12 shortcut )? See if there are any errors in the console section.
Misaki ha scritto:One thing I noticed is that the model that was initially tracking my movement will follow the second face’s movement when the second face tracking starts. Is this something wrong with my settings?
It is not your settings. The face tracking system is probably scanning the camera image from the top to down and left to right direction. The second face that appeared after is identified as face 1 because it is top left of the first face.I am going to add an numeral indicator near face meshes later to help identify the face index.
SilverStraw
  • Messaggi: 79

Misaki

The face tracking system is probably scanning the camera image from the top to down and left to right direction. The second face that appeared after is identified as face 1 because it is top left of the first face.I am going to add an numeral indicator near face meshes later to help identify the face index.
Ah, I see. I am relieved to hear that my settings were not wrong.

I tried again and found an error in the console tab.
Uncaught (in promise) TypeError: Cannot read properties spine_vtuber.js:1 of undefined (reading 'x')
at V (spine_vtuber.js:1:13566)
at l (spine_vtuber.js:1:13819)
at face_mesh.js:73:321
Here is a video recorded with the console tab open.
Avatar utente
Misaki

Misaki
  • Messaggi: 858

SilverStraw

I think I have located the problem from the error message and fixed it. Try again and see if you still get the error. For some reason your web browser was generating incomplete data. I made the program check if the data is missing beforehand so it would not stop working :wounded: .
SilverStraw
  • Messaggi: 79

Misaki

Thank you for your quick response! Unfortunately, I still got the error again. I didn't open the error log details last time so I showed that at the end of this video:

Let me know if I need to try anything else!
Avatar utente
Misaki

Misaki
  • Messaggi: 858

SilverStraw

The good news is that the previous fix worked but the same problem occurred on another section. I did the same check up on that section as well. I hope I squashed all those bugs :wounded: .

I am surprised that those errors has not appeared on my computer with multiple face tracking :think: .
SilverStraw
  • Messaggi: 79

Misaki

Now it's stable and working! Awesome :D


However, as you can see in the video, only the model placed on the right side sometimes momentarily appears to be in a strange look where each facial part seems to pop outward.
Screen Shot 2022-07-27 at 11.33.06.png

I checked the recorded video frame-by-frame and it seems that sometimes the face goes off the tracking and when it comes back on, the face parts seem to be pulled as shown above.
I don't know why the tracking is off, especially since the face is not moving significantly, but the face closer to the camera appears stable, so it may not be stable if it is farther away.
Non hai i permessi necessari per visualizzare i file allegati in questo messaggio.
Avatar utente
Misaki

Misaki
  • Messaggi: 858

SilverStraw

Now it's stable and working! Awesome :D
Great to hear.
However, as you can see in the video, only the model placed on the right side sometimes momentarily appears to be in a strange look where each facial part seems to pop outward.
I had the animation track scaling factor too high for when the AI stopped recognizing a face. The initial value of the vector has been adjusted by a million fold. Hopefully that will not produce an image that is as jarring as your image. I added another circumvention to stop moving the Spine model in that situation as well.
I don't know why the tracking is off, especially since the face is not moving significantly, but the face closer to the camera appears stable, so it may not be stable if it is farther away.
I lowered the face recognition threshold from 80% to 50%. When the AI is at least 50% confident, the computer will start producing facial coordinates for the Spine model. Hopefully that will help with finding faces further away from the camera. I am not sure if I want to expose the ability to change the threshold in the future.
SilverStraw
  • Messaggi: 79


Torna a Showcase