[UE4] Loading skin atlas textures dynamically?

NRSEKiander

Right now if an artist sets up multiple skins, once you import this in to UE4 it will create a USpineAtlasAsset which has references to every single texture used by both skins. So when I'm trying to use just skin A, the textures for Skin-A and Skin-B will be loaded. I'm guessing "skinning" data for both will also be loaded but I'm assuming that is fairly small compared to texture sizes.

I there a way to just load the textures for one skin and not the other. I was thinking of making

TArray<UTexture2D*> atlasPages;

to a

TArray<TSoftObjectPtr<UTexture2D>> atlasPages;

but USpineAtlasAsset loops through all the texture assets anyways to make the spine atlas rather then using just the ones it needs depending on the current skin.

Mario

Yeah, this is sadly not as straight forward. I'm not sure we can come up with a code-side solution for this, given UE4's APIs and the workflow people would expect in the UE4 editor. Apart from that, the texture atlas does not know which images in it belong to which skin, so your proposed solution does not quite help.

You can however export separate atlases for each skin. At runtime, you can simply switch between the atlases when you switch skin.

NRSEKiander

Well I was tinkering around with it and I got something working but as you said it's not particularly straight forward. The problem as you stated is the atlas doesn't know which textures the skin needs and to get at the skin you need the atlas. So basically what I did was added some functionality to create the atlas but with atlasPages that have null textures/render objects. But when the game goes through the old path it loads all the textures


UCLASS(BlueprintType, ClassGroup=(Spine))
class SPINEPLUGIN_API USpineAtlasAsset: public UObject {
.
.
spine::Atlas* GetAtlasWithoutLoading();
void LoadPages(const TArray<spine::AtlasPage*>& PagesToLoad);

UPROPERTY(EditAnywhere, BlueprintReadWrite)
TArray<TSoftObjectPtr<UTexture2D>> atlasPages;

UPROPERTY(Transient)
TArray<UTexture2D*> cachedAtlasPages;
.
.
}

Atlas* USpineAtlasAsset::GetAtlas () {
   if (!atlas) {
      if (atlas) {
         delete atlas;
         atlas = nullptr;
      }
      std::string t = TCHAR_TO_UTF8(*rawData);

  atlas = new (__FILE__, __LINE__) Atlas(t.c_str(), strlen(t.c_str()), "", nullptr);
  cachedAtlasPages.SetNumZeroed(atlasPages.Num());
  Vector<AtlasPage*> &pages = atlas->getPages();
  for (size_t i = 0, n = pages.size(), j = 0; i < n; i++) {
     AtlasPage* page = pages[i];
     
     if (atlasPages.Num() > 0 && atlasPages.Num() > (int32)i)
     {
        cachedAtlasPages[j] = atlasPages[j].LoadSynchronous();
        page->setRendererObject(cachedAtlasPages[j]);
        ++j;
     }
  }
   }
   return this->atlas;
}
Atlas* USpineAtlasAsset::GetAtlasWithoutLoading() {
   if (!atlas) {
      if (atlas) {
         delete atlas;
         atlas = nullptr;
      }
      std::string t = TCHAR_TO_UTF8(*rawData);

  atlas = new (__FILE__, __LINE__) Atlas(t.c_str(), strlen(t.c_str()), "", nullptr);
   }
   return this->atlas;
}

void USpineAtlasAsset::LoadPages(const TArray<spine::AtlasPage*>& PagesToLoad)
{
   Vector<spine::AtlasPage*> &pages = atlas->getPages();
   for (auto& pageToLoad : PagesToLoad)
   {
      const int32 pageIndex = pages.indexOf(pageToLoad);
      if (pageIndex >= 0)
      {
         cachedAtlasPages[pageIndex] = atlasPages[pageIndex].LoadSynchronous();
         pageToLoad->setRendererObject(cachedAtlasPages[pageIndex]);

  }
   }
}

void USpineAtlasAsset::PostLoad()
{
   Super::PostLoad();

   cachedAtlasPages.SetNumZeroed(atlasPages.Num());
}

Set up the skeleton with that that atlas. Set the skin to the appropriate skin. Then get the required pages that the skin uses and load up the texture for those.

TArray<AtlasPage*> USpineSkeletonAnimationComponent::GetRequiredAtlasPages()
{
   TArray<AtlasPage*> requiredAtlasPages;
   

   CheckState();
   if (skeleton)
   {
      Skin* skin = skeleton->getSkin();

  if (skin)
  {
     Skin::AttachmentMap::Entries entries = skin->getAttachments();
     while (entries.hasNext())
     {
        Skin::AttachmentMap::Entry entry = entries.next();
        if (entry._attachment->getRTTI().isExactly(RegionAttachment::rtti))
        {
           RegionAttachment* regionAttachment = static_cast<RegionAttachment*>(entry._attachment);
           AtlasRegion* attachmentAtlasRegion = static_cast<AtlasRegion*>(regionAttachment->getRendererObject());
           requiredAtlasPages.AddUnique(attachmentAtlasRegion->page);
        }
        else if(entry._attachment->getRTTI().isExactly(MeshAttachment::rtti))
        {
           MeshAttachment* regionAttachment = static_cast<MeshAttachment*>(entry._attachment);
           AtlasRegion* attachmentAtlasRegion = static_cast<AtlasRegion*>(regionAttachment->getRendererObject());
           requiredAtlasPages.AddUnique(attachmentAtlasRegion->page);
        }
     }
  }
   }

   return requiredAtlasPages;
}

We have our own actor class for spine actors and I just go through the new path before spine has a chance to go through the old code path.

USpineAtlasAsset& spineAtlas = mustDeref(streamableManager.LoadSynchronous<USpineAtlasAsset>(characterData.CharacterVisualTemplate.SpineAtlas));
spineAtlas.GetAtlasWithoutLoading();
SpineSkeletonAnimComp->Atlas = &spineAtlas;

USpineSkeletonDataAsset& spineSkeletonData = mustDeref(streamableManager.LoadSynchronous<USpineSkeletonDataAsset>(characterData.CharacterVisualTemplate.SpineSkeletonData));
SpineSkeletonAnimComp->SkeletonData = &spineSkeletonData;
SpineSkeletonAnimComp->SetSkin(characterData.CharacterVisualTemplate.SkinName);

TArray<spine::AtlasPage*> requiredAtlasPages = SpineSkeletonAnimComp->GetRequiredAtlasPages();
spineAtlas.LoadPages(requiredAtlasPages);

There were a few other changes I had to make as some things access the atlasPages directly which I had to make use the cached version.

Mario

That's much less terrible than I expected! Glad you found a solution that works for your use case. Thank you for taking the time to detail your solution here for others that might run into the same need.

Antidamage

I found a slightly different way to do this: make the material accessible in the plugin. Open SpineSkeletonRenderComponent.h and and change all of the material parameters from BlueprintReadOnly to BlueprintReadWrite. After that you can assign dynamic material instances at runtime and therefore change the texture it uses. I use it to control additional material features like water ripples and adding a mask for them to some spine actors. IMO this should be the default setting for materials.

Mario

Wait, that is possible? I remember playing with this leading to all kinds of rendering artifacts. I've created an issue to change that default here:

Mario

Thanks for the suggestion. The materials on the skeleton renderer component are now blueprint read and writeable. This change is live in both the 3.8 and 3.9-beta branch.

Antidamage

Nice work!

While you're at it, it'd be super, super cool if you could add mesh sockets to the UE spine runtime that can be defined in the editor somehow. I know there's a Spine-native way to do attachments, but this is the first thing I think of when I think about attaching something to a spine actor in a UE implementation. We can also do things like fetch the world position of the socket, attach particle emitters, etc.

UE sockets typically rely on bones to position sockets and would I be right in guessing that Spine just does baked vertex animation? Maybe the way to do it would be to add an extra data track to Spine that tracks where the socket should be and the runtime updates the socket location.

Mario

Spine doesn't do baked vertex animation, but actually generates a mesh each frame, based on calculations like IK, paths, or free form deformations. The underlying rendering is done through a ProceduralMeshComponent, provided by UE4 itself.

Have you looked into the BoneFollower component we ship with spine-ue4? It essentially drives the location of an actor based on a bone in a Spine skeleton.

The problem with sockets is two-fold:

I couldn't find any docs on how to create or expose sockets programmatically from spine-ue4. If you have any pointers I'd be much obliged.
Attaching arbitrary things to a socket, like a 3D mesh or particle system, might be problematic in terms of rendering order. While the skeleton rendering writes out z-coordinates for depth tests, results might not necessarily what you expect depending on what you attached, e.g. a translucent object.

Antidamage

I have not, that's exactly what I'm after. I'll check it out, cheers!

When it comes to Z-sorting lately I've just been using Translucent Sort Priority and setting it explicitly on everything. Works well in cases where the camera is always in the same position relative to the planes. and it's not a problem limited to Spine either, it's just UE being slack at translucent sorting.