Arthmoor

[Skyrim SE] Things to know when converting standard mods to SSE

414 posts in this topic

With Skyrim Special Edition on the horizon, and with the go ahead given to be able to discuss modding technicalities finally given, it's time to throw up a quick list of things I've noted need to be done in order to get an existing mod from 32 bit Skyrim to work on Special Edition.
 
BSA Files
 
If your mod uses a BSA file (and pretty much any decent one ought to be if it has assets to provide), you can't simply use that file with SSE. The game will either hang or crash at the opening menu. Fortunately all you need to do in this case is unpack your existing BSA file somewhere and then use the new archive tool that comes with the new CK to repackage it. The reason for this is that they've changed the header structures and compression methods.
 
Animations
 
Animation files for 32 bit will not work out of the box. Depending on what they are, you may end up with NPCs or the player stuck in a default pose unable to move, or perhaps even being unable to move the camera at all. Fortunately Bethesda has supplied a tool with the new CK that will allow you to update your .hkx files to the new format. This is presumed necessary due to Havok being upgraded to 64 bits.
 
Water and Flow Data
 
There are numerous instances throughout the game that have been updated to take advantage of the new water flow mechanics that were introduced. So far as I can see, every river has had the flow type changed and new drop in icons have been placed to indicate the direction the water should go. In some cases the type of water has also been changed. Mods which edit cells with water in them will need to update this data or they're going to cause noticeable patches of rivers, lakes, and even the ocean that don't look right or flow properly with the surrounding cells. Yes, Bethesda themselves forgot to account for this in their DLCs too so guess who's going to have to fix that now :P
 
New Ground Clutter
 
If you've been watching any of the new footage that's all over Youtube you're either already aware of this or are wondering if it might be true. It is. There's TONS of new ground clutter objects all over the place. Mostly plants and shrubs, but some occasional rocks and other small things as well. Mods which edit populated areas will need to be mindful of this when updating so that the new plants can be accounted for. All of this was added to Update.esm so if you're not already using it as a master, you need to be, so that you can move these new objects out of the way of your stuff to avoid ugly clipping. It also appears some landscaping may have been done too.
 
Scripts
 
Nothing new here, but I'm mentioning it because there are rumors afoot that they will break. They won't. Your scripts can be copied over as-is with no modification. Absolutely no changes to any of Skyrim's 13000+ scripts were made beyond many of the DLC ones getting recompiled with their debug statements commented out. Which the source code had already but the .pex files did not.

When the CK first starts though, all of the vanilla scripts will be incorrectly unpacked to Data\Source\Scripts instead of Data\Scripts\Source. The CK should be able to find them, but if you'd prefer to have them where they belong then you need to make a change to CreationKit.ini and add the following to its Papyrus section:

sScriptSourceFolder = ".\Data\Scripts\Source"

As with the 32 bit CK, the DLC scripts need to be moved directly into the same folder. The CK will be unable to find them at all if you don't. You should move them into the proper folder in the same order the DLCs were released. It would be strongly advised to do the work for this in a seaprate folder, then make an installable package for the base scripts so you can put it in as the first thing in the Wrye Bash installers tab (or NMM too I suppose) so that it will have proper conflict tracking for anything you may edit later on.

Meshes

Meshes for SSE may need to be updated. There have been some changes which the Nifskope guys are better equipped to explain but the updated format for them lies somewhere between what 32 bit Skyrim offers and what Fallout 4 offers. So far as I've been able to tell, all of the ones my mods have work, and USLEEP's fixed ones aside from one troublesome Horker work too. There's some possibility that old meshes may be a slight performance drag but it may not be enough to matter in the end.

Skinned Meshes (Armor, Clothing, Body, etc)

If you have these and the NiSkinPartitions blocks are using Strips instead of Triangles (I'm going to presume those of you doing these know what this means) then you need to grab an updated copy of Nifskope to fix this. When you verify that it needs fixing, do the following in Nifskope:

Spells -> Batch -> Make All Skin Partitions. On the pop-up menu that appears, set "Number of Bones per Partition" to 60 and click OK.

Once that's done, go to:

Spells -> Batch -> Update All Tangent Spaces.

You can then save the mesh and it should work. I've personally tried this on the old Friendly Guard Helmets mod and it corrected the issues with that without any apparent in-game problems. If I can figure this out, you gurus who make this stuff can too :P

All credit to ousnius for this information. According to him, you could also fix this by importing the mesh into Outfit Studio and then immediately exporting it.

Decal Flags

Static meshes with the SLSF1_Decal or SLSF1_Dynamic_Decal need to have those flags removed if they are not using a moss or other overlay type of layer (ie: Riften city walls) on them. If these flags are not removed, the portions of your meshes where they are used will appear semi-transparent in SSE. Classic Skyrim generally did not manifest problems with this but SSE apparently has no tolerance for it.

Textures

.tga files are no longer supported. If you have assets using these, they need to be converted into proper DDS files.

Textures originally saved in either the A1R5G5B5 format or the B5G6R5 format must be resaved in A8R8G8B8, R8G8B8, or an equivalent compressed format like DXT5 or DXT1. If this is not done, some installations of DirectX 11 will not be able to process it and SSE will crash. This seems to be most common for folks using Windows 7. NifScan will detect this when run.

ESP Files + NPC Facegen

ESP files need to be loaded into the new CK and saved out to be sure they're updated correctly. If your mod includes edits to NPCs or new NPCs entirely, you absolutely MUST do this. Failure to do so will result in your NPCs having heads which are entirely black - not just the usual issues with the familiar grey-face. The CK will automatically update any needed facegen data in your mod, and it may generate some for you that you didn't know you needed. When it does, don't include the .tga files it still generated because the game will ignore them entirely.

Records Updated by the new Update.esm

All of the game's grass records (GRAS) have been updated to push higher grass density.

Most of the game's Image Space Adapters (IMAD) have been updated with new alpha and blur values.

Most of the game's Image Spaces (IMGS) relating to weather have been updated with new HDR values.

A few Landscape Textures (LTEX) picked up an unknown INAM value, along with a couple having specular values modified.

Many snow and ice related statics (STAT) have been updated with new texture sets and material shader settings.

5 new landscape texture sets were added. Referenced by many of the other edits.

The entirely new Volumetric Lighting group (VOLI) was added. These are used in weather records.

Pretty much every typ of water (WATR) was updated for the new water shaders.

The vast majority of the game's weather types (WTHR) have been updated to include the Volumetric Lighting information.

Record changes in Skyrim.esm

Yep, there are apparently also some stealth patches embedded in Skyrim.esm too.

Several book records have been corrected and will now show up as ITMs if checked against USLEEP, which hasn't been exposed to the new CK. USSEP, which has, had several additional records improperly reverted back to vanilla despite those being wrong. It appears as though the hidden Note/Scroll flag is no longer supported and the CK will revert these out, resulting in numerous ITMs if the only change you made was to that flag via xEdit.

Share this post


Link to post
Share on other sites

​Thanks for compiling the list Arthmoor. You mostly talk about forward compatibility. But what about backwards compatibility? Will, say, a bsa packed for SE work for the 32bit version? I.e. will we, once we do properly migrate as you outlined, be able to serve both versions with the new bsa, or would we have to maintain two separate mods?

Share this post


Link to post
Share on other sites

No. There is no backward compatibility at all because the formats on these things are different. Also I need to add some more stuff to the list cause I forgot about them :P

Share this post


Link to post
Share on other sites

Lol you've probably, one week before release, already forgotten more things than i will ever know about SE modding :D

 

Thanks for the answer. Guess it's time to give a last hard look at my vanilla Skyrim mods to see if they are in a state they can be abandoned, moving forward to the SE.

Share this post


Link to post
Share on other sites

The water edits are the big thing right now. There's a lot, cause any incidental cell edit will include that data and LOTS of stuff was near water, and some of it was directly addressing old water flow that was bad. It's not really feasible to throw USLEEP into the SSE CK and start hunting by hand either. So we're waiting on xEdit support for that.

 

This will in turn delay anything that depends on USLEEP, like Cutting Room Floor.

Share this post


Link to post
Share on other sites

I hope this SSE is worth the trouble to get mods up and going with it. It sounds like a lot of work so far that I don't want to do myself :P

 

I think I'm going to wait and see for it to mature before I dive in myself.

Share this post


Link to post
Share on other sites

Very useful.  Thanks for posting this Arthmoor. :)

 

Although, for the moment only the last section applies for my mod and if I got it right I only need to load my mod into the new CK then re-save it.

 

Of course I need to clean it in TES5Edit a *final* time, unless I'm updating my mod, before I've completely converted my mod into a SSE mod.  Btw...  Can TES5Edit handle SSE mods properly from the day the new CK is available?

Share this post


Link to post
Share on other sites

xEdit hasn't bee updated yet. I'm not sure what their schedule is for doing so, but I am hoping it's ready to roll before the 28th so we can use it to get USLEEP updated.

Share this post


Link to post
Share on other sites

I think that is a little too optimistic to expect an updated TES5Edit to work with SSE mods the 28th.  I would be more than satisfied if zilav or who ever updated TES5Edit for SSE mods the 29th or a week later perhaps two weeks.

Share this post


Link to post
Share on other sites

 

Water and Flow Data

There are numerous instances throughout the game that have been updated to take advantage of the new water flow mechanics that were introduced. So far as I can see, every river has had the flow type changed and new drop in icons have been placed to indicate the direction the water should go. In some cases the type of water has also been changed. Mods which edit cells with water in them will need to update this data or they're going to cause noticeable patches of rivers, lakes, and even the ocean that don't look right or flow properly with the surrounding cells. Yes, Bethesda themselves forgot to account for this in their DLCs too so guess who's going to have to fix that now :P

 

 

:facepalm: .. though coming from Bethesda its not surprising

 

Very pleased the water upgrade was not just a case of shader / effects, but does actually address the flow direction aswell, that was my biggest immersion killer in the original. Not so good for the USLEEP update, but by the nine its going to be nice not looking at water going into the side of river banks

 

---------------------

 

One question reference Textures - Did they not update the way they pack them into BA2s' ? ( same as they did for FO4 in that the highest resolution top layer mipmap is packed as a separate file, then the game only loads the big one on the rare occasions it is needed = more optimal loading / caching )

 

I know I will need to redo my Vanilla Reduced Textures BSAs and dummy plugins anyway ( well, that is if there is a need for VRT for SE, I need to give that some thought really ), but I would be surprised if the textures were not part of the optimisations made to SE

 

 

Edit : Oooh and, Arthmoor is there any need to update your guide on using the official Archiver, or is it pretty much the same still ?

Share this post


Link to post
Share on other sites

They're not using BA2 for SSE. It's still BSA but with LZ4 compression instead, and something else about the headers for non-compressed files that the existing tools don't know how to handle. Textures are still standard dds files. They aren't using the new BC7 compression in SSE.

 

The official archiver works exactly the same as the old one. You just need to make sure you're using the right one because the program doesn't even tell you.

Share this post


Link to post
Share on other sites

I think that is a little too optimistic to expect an updated TES5Edit to work with SSE mods the 28th.  I would be more than satisfied if zilav or who ever updated TES5Edit for SSE mods the 29th or a week later perhaps two weeks.

Uploaded new xEdit build with SSE support.

Elgar, smokex, Leonardo and 3 others like this

Share this post


Link to post
Share on other sites

No. There is no backward compatibility at all because the formats on these things are different.

 

Oh god you just saved me from a very big mistake. :D

 

 

Uploaded new xEdit build with SSE support.

That's what they call dropping the mic. :)

Share this post


Link to post
Share on other sites

one question regarding updated meshes:

Did bethesda update all their vanilla meshes themselves? or does the engine still handle old meshes?

Since you mentioned most meshes of USLEEP work, but only a handful needed upgrading, is there something to look out for, to spot which meshes need an upgrade (aside from going in-game and see if i encounter crashes)?

Share this post


Link to post
Share on other sites

Jon could answer that better than I can. I'm not actually sure what all they did to their meshes, but the only one in USLEEP to complain so far was a horker mesh we removed a null bone from. We won't be including it with the SSE archive.

Share this post


Link to post
Share on other sites

Uploaded new xEdit build with SSE support.

That's great.  Now, can I finish my 1.2 update and only wait for the new CK to come out the 28th. :D

EnaiSiaion likes this

Share this post


Link to post
Share on other sites

Question: there's a rumour on Reddit that cell data has changed so cell modifications are not compatible and will need to be redone. What is this about, and is it something one could just fix in the CK/xEdit or does it mean that existing cell modifications are broken/gone and need to be redone from scratch?

Share this post


Link to post
Share on other sites

One question which isn't answered here is.. how are we supposed to port a mod that requires USLEEP as a master? Will the switch masters xEdit script work?

Share this post


Link to post
Share on other sites

one question regarding updated meshes:

Did bethesda update all their vanilla meshes themselves? or does the engine still handle old meshes?

 

It handles old meshes, but seems to be a bit more picky about them being well-formed.  Also certain texture formats are no longer supported like uncompressed grayscale, which the game reads as if it were RGB with only the red filled out.

 

With that said every single NIF file has been run through their internal converter program to have a better vertex data layout, which enables the engine to more optimally read all the data into DX11. Not to mention it's significantly compressed, where a typical vertex (position, UVs, normals, tangents, bitangents, and vertex colors) goes from 60 bytes to 32 bytes.**

 

The vertex format is the exact same as FO4's except SSE meshes are never half-precision (using 16-bit half floats for all of the vertices). Most all blocks are completely identical,  but no existing tools will even be able to open the NIFs without an update even if they can read FO4 NIFs, since SSE NIFs are a bit of a mixture.  The NIF file versions go 83 -> 100 -> 130 for Skyrim -> SSE -> FO4, showing that SSE came between the two developmentally (which we already knew).

 

 

** Which, if they're doing it correctly, the engine is having to take a bit of extra time to load the old 60 byte vertex, but it's getting converted to the new format before being sent to DX11.  So if there is any kind of overhead with old meshes it should only affect loading time.

Share this post


Link to post
Share on other sites

Thanks for the explanation. So that means i should expect my old meshes to work, if they adhere to Skyrim's standard.

That internal converter program you mentioned, i suppose that means it will not be publicly available, right?

Is a conversion functionality planned from the community (e.g. nifscope?)

Is there a tool to check meshes for errors, that could cause issues? I have hundreds of meshes, mostly old armors and weapons from unsupported mods in my load order, and have invested countless hours keeping the associated merged esp error free and up to date. I see no way of doing the same manually for all meshes, and would hate to let that work all go. I'd be fine dropping parts of it, if there was an automated way that told me where the black sheeps are hiding. Just a bit of background, why such a tool would help me massively.

 

Since team USLEEP managed to find that rotten horker mesh so quickly, i was hoping there is in fact an automation tool...

Share this post


Link to post
Share on other sites

Question: there's a rumour on Reddit that cell data has changed so cell modifications are not compatible and will need to be redone. What is this about, and is it something one could just fix in the CK/xEdit or does it mean that existing cell modifications are broken/gone and need to be redone from scratch?

It's just a rumor. An unfounded one. Old ESP files will work, but you should still resave them so that it triggers whatever internal updates they've done to take place, such as updating NPC facegen. The only thing you're going to have to check for manually afterward are cells with water data in them as explained in the OP.

 

One question which isn't answered here is.. how are we supposed to port a mod that requires USLEEP as a master? Will the switch masters xEdit script work?

You can use SSEEdit to change the master file in the header. Normally that's something we all strongly recommend against, but in USLEEP's case since the filename is all that's changing for SSE, swapping the master won't damage the form IDs.

 

That new file name will be "Unofficial Skyrim Special Edition Patch.esp" btw.

 

 

Since team USLEEP managed to find that rotten horker mesh so quickly, i was hoping there is in fact an automation tool...

That Horker literally announced itself as a missing mesh icon, it wasn't hard to spot while out wandering around :P

Share this post


Link to post
Share on other sites

They didn't bring over the precombined visuals system so that won't be an issue.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now