Technical question about physics (it's broken in local spaces, right?)

[Shifty Geezer]

I've been struggling to get some simple stuff going in Dreams, and the more I look at it, the more I think there are limitations in the engine that require workarounds and one cannot simply build what one would imagine as the obvious solution. However, I don't know what's a bug that needs reporting and what's just a limitation of the engine.

It would appear that physics only works correctly in the global space, and trying to get physics working in a transformed local space will fail. That is, it all works great until you put some mechanisms inside an Imp Tracker, say, and then physics goes to pot.

For my specific case, I had mind to create a puppet (not a Dreams puppet). Using a real world analogue, imagine a cylinder of wood with a polystyrene ball for a hand and a thing, springy wire between them. As you move the wood body, the hand would wave around. I tried to create this in Dreams with a springy joint on a body and arm inside an Imp Tracker but it didn't work properly, so I started experimenting and found the simplest case just doesn't work:

1. Create a cylinder.
2. Create a second cylinder floating above and connect them with a piston - the top cylinder moves up/down when playing.
3. Group those objects into an Imp Tracker and the bottom object follows the imp but the piston flaps around.
4. Make the piston rigid and it moves up/down regardless of the orientation of the Imp Tracker. The bottom cylinder is aligned with the motion controller but the top one stoically stays upright as the piston remains vertical.

I would have expected the piston to align to the Imp Tracker and the Move controller that it's following, but that's not the case, and it appears there's no way to make that the case. It occurred to me perhaps there was a setting for the piston to use local or global space as there are on other widgets, but that's not the case either.

So does physics not play nice with motion tracked controllers? And as such, something as simple as a ball stuck on a wire stuck in a puppet as a wobbly arm needs some complex rigging and keyframing? Is this a bug and not the intended behaviour of Dreams, or is it a limitation of the engine? Or am I missing something fundamental and it actually IS possible to embed physical connections inside motion-tracked objects and have them work?

Cheers!

[TAPgiles]

There's no such thing as simulating physics in a "local space"; it's being simulated with the rest of the scene. So there are some situations where the joints can't keep up properly because the anchoring parent object is moving too quickly or being used in a particular way.

One thing that helps with joints in a head or hand tracker is to give it 1% springyness. That can fix it freaking out.

Something to make sure of is that the entire rig is inside the tracker, not just the anchoring object. It can be easy to grab just 1 object and scope it in but that will leave the rest outside for example.

Workarounds you can try is adding a tag in the tracker, grouping up the rig, and teleporting the rig to the hand. A teleported object sort of has its physics turned off and the entire chain of joints come along too. So that may fix it up for you.

If you just want something to be springy, you could actually use the joint but then keyframe its position or rotation, and add springyness to the keyframe's setting. That kind of springyness isn't using physics per se; just the change in position the object should be. So it can work a lot better for secondary motion.

[Shifty Geezer]

Glad you're still around, TAPgiles! Whatever happened to the old feedback forum? Seems most discussion has moved to Discord which is really inefficient for threaded conversations and QA.

Anyhows, I've got it working revisiting this video from a Molecule: https://www.youtube.com/watch?v=99un_NjKTRw

By putting objects in groups, I have a piston that tracks where I'm pointing. Similarly string flexes inside a hand-tracked group, so I can create a string puppet.

However, I still don't get it. I'm a game dev with a good understanding of how physics engines typically work and the situation where the piston stays pointing upwards when the object its connected to changes orientation is a little mind boggling. I *really* want to know how Dreams works underneath, but we don't get the same technical documentation and insights we do with other game engines.

[TAPgiles]

Yeah I visit from time to time ;P You should try the subreddit: https://www.reddit.com/r/PS4Dreams/

The feedback forum was taken down when they changed to a new system internally--which is what indreams.me/support taps into I think? The forum will come back eventually, but I guess it's taking time.

Ah so grouping things differently sorted it?

Yeah... I have written the closest thing we have at tapgiles.com/docs. I make things as clear and accurate as I (and anyone from the community) knows, and sometimes we get help from the odd person at Mm answering a question. But that's all for now :D

[Shifty Geezer]

Sadly grouping things 'properly' hasn't solved all the issues and I think there's a fundamental weirdness with how Dreams does things. A piston works, but the more you increases tightness, the more it sticks to its global orientation rather than local one. Likewise, attach a bolt between two objects and have one mapped to the motion controllers and the hinge kinda works though is wobbly. Add 1% springiness and the attached object teleports to the grabbed object. Even weirder, dangle a couple of objects on elastic as an independent object in the scene and they hang...until an imp tracked object gets within a few games metres and then they start drooping! Move the near object away and they stop.

After another hour investigating, my current hypothesis is that Dreams uses keyframing to fake physics. The odd behaviour of the elastic makes me think physics is being evaluated based on proximity of changes or something. I don't see how it can be a conventional solver given the behaviour and, seeing how MM had to invent a way to get physics into a novel SDS renderer, maybe they had to go down some unusual route?

I'll float my next questions on reddit. Do you happen to know of anyone creating Dreams with motion controlled puppets? I couldn't find any searching the Dreamiverse. My interest in Dreams comes from the showcase MM did of their puppets with the dancing and music:

https://www.youtube.com/watch?v=TYY-J3ckeSI&t=140s

I actually had an LBP animation included at the Cannes Film Festival and I was very excited about the prospects Dreams offered. It should be possible for me to create a body, a cylinder say, and attach two arms via bolts, and as I waggle the body around with the Move Controller, have the arms follow and bounce about. This was very intuitive in LBP but Dreams keeps resisting me!

[TAPgiles]

Ah that's annoying. Well, I'll just blurt everything I've got on this subject and hopefully some of it will make sense ;P

Physics is done using a unique system, as you guessed. Non-movable and collidable sculpts use their SDF for collision. For movable and collidable sculpts spheres are procedurally placed to fill the sculpt, more densely or sparsely depending on the "physics cost" setting. Each sphere is its own physically simulated object, effectively. So more spheres = more processing.

In some cases, a lower physics cost will fit the object better. Best example is a sphere made on-grid. Lower physics uses a single sphere so it fits perfectly--as well as being a lot cheaper for the sim.

Objects go to "sleep" if they don't move enough and nothing is near them they can collide with (or will be affected by). When something approaches that they could be affected by they "wake." This is similar to other physics sims I believe? So you can get cases where an object seems to come to rest but when another object gets close it starts to move by itself again because it continues to simulate that object.

You can use test mode > heatmaps > physics, to see the state of sculpts. Black = non-movable, non-collidable (ignored by the physics sim). White = non-movable, collidable (sdf collision only, no sim). Blue through red, spheres visible = movable, collidable. Colour based on how many spheres are simulated, indicating how processing-expensive that object is. In this mode you can tweak any sculpt even if it's inside groups, and adjust the physics cost. https://youtu.be/XeiD3Ce9poc?t=503

Objects go green when "sleeping."

When you change the physics cost in the normal view, you see a flash of the spheres.

Physics is simulated in fixed time steps (making it deterministic with identical starting conditions) at 30fps (I think?) and rendering is interpolated between sim frames. If framerate slows, physics seems to get slower and slower because of the interpolation.

[TAPgiles]

Regarding that "tech demo" thing... it's actually very easy to set that up. Add a puppet, tweak the controller to "follow imp" mode. The procedural animations are what drives the legs and whatnot. So then you can just move your controller around to move the puppet as if it were dancing.

If you're still having trouble with the hand tracker, have you tried the "follow imp" mode on a controller sensor instead? They're similar but work differently under the hood, so it may work better for you with the controller sensor as shown in that video you linked.

[Shifty Geezer]

The puppet demo was just inspiration. Watching it, seems mostly key-framing and logic. Everything I can find on Dreams regarding 'puppets' is using the 'puppet' base - I think I'm in a field on my own here!

And yeah, I mix and match trying Hand Trackers and Controller Sensors but they both seem to have the same issues. Interestingly, physics on an object that's grabbed by the imp while editing seems more robust, so that's another area of investigation.

[TAPgiles]

If you're grabbing stuff while editing it works completely differently than in play. Just something to be careful of there.

I've definitely made a ragdoll type puppet before that hangs from the imp tracker. I think I had just that cross shape at the top in the imp tracker, and the rest was outside? It was just hard for me to move slow enough that the whole thing didn't fly about violently XD