Jump to content

Photo

Unofficial Patch ESM Conversion

unofficial esm patch

  • Please log in to reply
34 replies to this topic

#1
Arthmoor

Arthmoor

    Horkerborn

  • Administrators
  • 4999 posts
  • LocationUnited Socialist States of America

Now that this kalpa for the USKP is complete (for you non-lorehound types, kalpa = cycle) it's probably time we make public our plans for the next one.

Due to an increasing number of odd engine related bugs it's been decided among the team that we will need to convert the USKP (and therefore all the DLC patches too) into what's called a "False ESM". That is, and ESP file which is flagged as an ESM by way of changing the flag in TES5Edit.

So why would we want this? There are a few classes of bugs this has already been shown to help with in internal testing:
 

  • Seemingly random crashes in and around major cities, and other areas where a lot of data is edited. Especially navmeshes.
  • The "landscape foliage" bug in which a child worldspace begins displaying landscape foliage from its parent worldspace because an object in the parent worldspace was edited. This is readily apparent in Darkwater Pass.
  • The "LOD trees show up at close range" bug. This is where you will sometimes see trees that are clearly intended to be displayed as LOD sitting within the currently loaded cells, sometimes even merged into the real trees. As described here.


There may be others, but these 3 classes of bugs are confirmed to be correctable by converting to a false ESM.

So you might be asking then. Why not a proper ESM file? Two reasons mainly:

1. Steam cannot support ESM files, we'd have to cease distribution there all together and not just for the USKP.
2. Other mods are relying on the filenames being ESP files and not all of those mods are currently being maintained anymore.

It's already been verified that Wrye Bash is OK with changing the flag. It will not complain when the files get bumped up into the ESM section.

With proper CRC checks, BOSS won't complain either. Though I'm hoping some other method can be used to detect it ongoing because having a growing mountain of CRC checks won't be helpful. Perhaps version number parsing from the description field.

I cannot vouch for how NMM will react since I don't use it, and I'm not sure anyone else on the team has tried it either.

Who knows what Mod Organizer will think.

We will bump all the version numbers on all the patches up to 2.0 when this happens. We don't have a time frame yet, because there's also a giant mountain of vanilla bugs piled up in the tracker to address too. But when it happens, all of the patches will be updated at the same time.

Our proposed changes in the load order for the game will look like this:

Skyrim.esm
Update.esm
Unofficial Skyrim Patch.esp
Dawnguard.esm
Unofficial Dawnguard Patch.esp
Hearthfires.esm
Unofficial Hearthfire Patch.esp
Dragonborn.esm
Unofficial Dragonborn.esp
[The rest of your stuff]

The main reason for this should be obvious. It will generate much less interference with things following on in the ESM group, and at the same time allow us to undo some of the changes in each DLC patch that are only there because the USKP canceled them out due to its own edits.

Best of all, thus far in testing, it poses no risk for converting them in the middle of an active playthrough, even with a lot of other mods loaded.

So anyway, that's where we're headed now. We've already kind of leaked hints at it, so now it can be considered confirmed rather than rumor :)

 

The ESM Conversion Checklist

 

Wrye Bash Support - Supported as of release 304.2.

NMM Support - Works as expected, NMM flags the files with a warning but allows them to move into the ESM group.

Mod Organizer Support - v1.0.0rc1 and up has been confirmed to work.

TES5Edit Support - Supported as of release 3.0.31.

BOSS Support - Working in 2.1.1. (was some confusion here, but it *IS* working)

 

Checklist items need to be completed before we move forward on this.

 

BETA Scheduling

 

We're planning to go into beta on all 4 patches Saturday (October 5). I expect we'll have a minimum of 2 weeks testing, perhaps longer, to see how all this affects things and to validate that none of the new fixes coming in with this are broken.

 

With that in mind, I figured this is as good a time as any to say so. Once this round of beta testing is complete, the USKP will no longer be distributed on Steam. Leaving the 1.3.2c version up there is already causing us problems and we can't in good conscience leave it up there after 2.0 has gone live. So anyone still using Steam to get the USKP needs to transition to one of the other available download sites.

 

We're also going to have to check to see if Steam will accept a falsely flagged ESP file and deliver it back to the user intact as such. I mention this now because if it doesn't, then ALL of the unofficial patches will have to be taken down from Steam and everyone using those too will need to make provisions to change to another download site.

 

So keep that in mind for those of you using Steam.


My Oblivion Mods My Skyrim Mods Donations Welcome
 
"Now, I am become Death, the Destroyer of Bugs!"


DreamtimeSigBar.jpg


#2
Nico coiN

Nico coiN

    Brother

  • Members
  • 169 posts
  • LocationWiwiland, France

I last used Wrye Bash a very long time ago... Does it process proper ONAMs for Skyrim when converting to esm ?



#3
Arthmoor

Arthmoor

    Horkerborn

  • Administrators
  • 4999 posts
  • LocationUnited Socialist States of America

No, Wrye Bash does not do that. TES5Edit is the only tool capable of generating the necessary ONAM subrecords.


My Oblivion Mods My Skyrim Mods Donations Welcome
 
"Now, I am become Death, the Destroyer of Bugs!"


DreamtimeSigBar.jpg


#4
Gruftlord

Gruftlord

    Initiate

  • Members
  • 254 posts
  • LocationGermany

does this also mean the Gildergreen LOD will be available once more?



#5
Arthmoor

Arthmoor

    Horkerborn

  • Administrators
  • 4999 posts
  • LocationUnited Socialist States of America

I don't think that was caused by the same sort of issue, but we can certainly test it to find out.


My Oblivion Mods My Skyrim Mods Donations Welcome
 
"Now, I am become Death, the Destroyer of Bugs!"


DreamtimeSigBar.jpg


#6
AndalayBay

AndalayBay

    Champion

  • AFK Mods
  • 1212 posts

I think you just broke my brain. :frantics:


Madam, you have between your legs an instrument capable of giving pleasure to thousands, and all you can do is scratch it!
Attributed to Thomas Beecham in reference to the performance of a female cello soloist

 


#7
Sclerocephalus

Sclerocephalus

    Initiate

  • Members
  • 387 posts
  • LocationSaartal

If esmifaking really helps in getting rid of all sorts of obscure problems, everybody will esmifake their mods in the not too long run. What does this mean for game stability ?



#8
tyrel68

tyrel68

    Imperial Swine

  • Members
  • 263 posts
  • LocationCanada

Does this mean all the compatibility bug patchs for the patches overwritging previous stuff need to be undone?



#9
tyrel68

tyrel68

    Imperial Swine

  • Members
  • 263 posts
  • LocationCanada

If esmifaking really helps in getting rid of all sorts of obscure problems, everybody will esmifake their mods in the not too long run. What does this mean for game stability ?

technically, afaik esm is always > then esp so doubt it would hurt



#10
Arthmoor

Arthmoor

    Horkerborn

  • Administrators
  • 4999 posts
  • LocationUnited Socialist States of America

If esmifaking really helps in getting rid of all sorts of obscure problems, everybody will esmifake their mods in the not too long run. What does this mean for game stability ?

 

Well, it led to a mess with Fallout 3. Probably with New Vegas too, but that was largely because the engine was in much worse shape than it is now and couldn't adequately deal with it. Bethesda's continuing refinements to things have made it somewhat clear that they favor the ESM. If they'd just make the CK play along it wouldn't even be that big of a deal anymore.

 

So stability shouldn't be affected, but I'd still caution against doing this until there's a good reason to do so in other mods.

 

Does this mean all the compatibility bug patchs for the patches overwritging previous stuff need to be undone?

Just the records in the DLC patches that exist to override a USKP edit that broke a DLC change.


My Oblivion Mods My Skyrim Mods Donations Welcome
 
"Now, I am become Death, the Destroyer of Bugs!"


DreamtimeSigBar.jpg


#11
Nico coiN

Nico coiN

    Brother

  • Members
  • 169 posts
  • LocationWiwiland, France

Turning a basic esp to a fake esm has more about hacking than an official and approved feature. We could say the same thing about the note/scroll/book flag that can only be altered with TES5Edit or about creating an esp dependancy, for example. But if it works and is safe, why not ?



#12
Gruftlord

Gruftlord

    Initiate

  • Members
  • 254 posts
  • LocationGermany

Just the records in the DLC patches that exist to override a USKP edit that broke a DLC change.

sure?

but that would still mean the DLCchange will overwrite what fix was in the USKP.

before the USKP overwrite the change from the DLC, but in both cases, the U DLC P should provide a merge, or is there some clever new option to work around this by esmifying?



#13
Arthmoor

Arthmoor

    Horkerborn

  • Administrators
  • 4999 posts
  • LocationUnited Socialist States of America

The cases I'm thinking of are of the type where the USKP fixed something and the DLC mad an obviously intentional change that the USKP fix genuinely breaks. There's not very many, but I'd have to go digging for them to be able to tell you.


My Oblivion Mods My Skyrim Mods Donations Welcome
 
"Now, I am become Death, the Destroyer of Bugs!"


DreamtimeSigBar.jpg


#14
Nico coiN

Nico coiN

    Brother

  • Members
  • 169 posts
  • LocationWiwiland, France

If I understand right, the intended 'dirty edits' in UDGP, UHFP and UDBP will all be removed because they'll have no use anymore ? All patches will finally be truly '100 % cleaned' according to TES5Edit's standards and definitions ?



#15
Arthmoor

Arthmoor

    Horkerborn

  • Administrators
  • 4999 posts
  • LocationUnited Socialist States of America

Added a checklist to the bottom of the OP. Bash used to work but doesn't now, that's a setback, but hopefully fixing it isn't too difficult. All that stuff needs to be confirmed as working before we can go forward with the plan.


My Oblivion Mods My Skyrim Mods Donations Welcome
 
"Now, I am become Death, the Destroyer of Bugs!"


DreamtimeSigBar.jpg


#16
tyrel68

tyrel68

    Imperial Swine

  • Members
  • 263 posts
  • LocationCanada

finding and closing the affected bug tracker entries is going to be fun. Assume there could be a substantial amount



#17
Arthmoor

Arthmoor

    Horkerborn

  • Administrators
  • 4999 posts
  • LocationUnited Socialist States of America

Some quick turn around on Bash support, Tes5edit, and Mod Organizer puts this on good footing. BOSS ought to work fine as is, but needs to be verified. The hard part comes from getting the USKP's piled up bug fixes done now.


My Oblivion Mods My Skyrim Mods Donations Welcome
 
"Now, I am become Death, the Destroyer of Bugs!"


DreamtimeSigBar.jpg


#18
tyrel68

tyrel68

    Imperial Swine

  • Members
  • 263 posts
  • LocationCanada

question will this just be a bug squasher or will stabilityfps improvement involved too?



#19
NightStar

NightStar

    Brother

  • Unofficial Patch Project
  • 221 posts

Turning the unofficial patches into false esm's is mostly about squashing bugs, but it will also avoid some engine issues. Most notably it should reduce the number of CTDs on some areas where a lot of game data has been edited by mods, so it could be a stability improvement as well. However, no sure answer can be given about that until it is more widely tested.



#20
Arthmoor

Arthmoor

    Horkerborn

  • Administrators
  • 4999 posts
  • LocationUnited Socialist States of America

We're planning to go into beta on all 4 patches Saturday (October 5). I expect we'll have a minimum of 2 weeks testing, perhaps longer, to see how all this affects things and to validate that none of the new fixes coming in with this are broken.

 

With that in mind, I figured this is as good a time as any to say so. Once this round of beta testing is complete, the USKP will no longer be distributed on Steam. Leaving the 1.3.2c version up there is already causing us problems and we can't in good conscience leave it up there after 2.0 has gone live. So anyone still using Steam to get the USKP needs to transition to one of the other available download sites.

 

We're also going to have to check to see if Steam will accept a falsely flagged ESP file and deliver it back to the user intact as such. I mention this now because if it doesn't, then ALL of the unofficial patches will have to be taken down from Steam and everyone using those too will need to make provisions to change to another download site.

 

So keep that in mind for those of you using Steam.

 

Eh. Well. That was answered rather quickly:

Attached File  NoCanDo.jpg   32.09KB   3 downloads

 

CK won't even bother to try. Looks like all 4 will be coming down.


My Oblivion Mods My Skyrim Mods Donations Welcome
 
"Now, I am become Death, the Destroyer of Bugs!"


DreamtimeSigBar.jpg






Also tagged with one or more of these keywords: unofficial, esm, patch

1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users