Search the Community
Showing results for tags 'esm'.
Found 3 results
Beginning with Patch 1.5.3 of Skyrim Special Edition, the game supports a new type of mod file known as a "light plugin" with the .esl extension. I've spent some time going over these files now that Wrye Bash is able to display the save headers and shown how they are listed based on how they are flagged in your load order. It's somewhat of a convoluted mess, but there is a certain order to it as well. Things are divided up into 6 distinct blocks of files according to the save headers. This post will serve as a complete breakdown of how all types of plugin files will load into the game and be listed in your save headers. Block 1: The original 5 ESM files. The load order of these is hardcoded and cannot be changed. The game will always load them. If any are missing, the game will crash on startup. Block 2: Creation Club content ESL files flagged as masters. This block is loaded after the final standard ESM file. The order of these files is hardcoded to load alphabetically. Even if these files are marked as disabled, the game will still load them. You must completely delete one in order to remove it from your game. Block 3: Standard ESM files produced by modders as well as ESL files produced by modders and flagged as masters. These begin loading immediately following the Block 2 plugins. The load order is as listed in your mod manager. This block also covers ESP files which are flagged to act as ESM files, such as the USSEP. They must be listed as enabled before the game will load them. Block 4: Standard ESL files converted by modders in the CK. They will be loaded according to what's set in your mod manager and must be marked as enabled for the game to load them. Block 5: Standard ESP files. What most mods will actually be. These are the normal types of files generated by the CK when modding. They will load in the order specified by your mod manager and must be marked as enabled for the game to load them. When an ESL of any type is loaded at runtime, the game converts all of the form IDs inside to use the FE index slot. This very likely means that this slot, once useable by normal mods, is now considered off limits. Results of trying to load an ESP in this slot while also having ESL files installed has not been tested and is probably not going to work. Attempts to make ESL files that use standard ESP files as their masters have so far all resulted in the ESP files being bumped to load into the Block 3 area instead of Block 5 where they belong. This is highly undesirable since it will cause problems with the ESP files being out of their expected positions in a functioning load order. This can lead to anything from edits being overridden in unexpected ways to the game crashing. Using the console to check the runtime form IDs of the ESPs that get bumped is confirmed so it would be a very bad idea to ignore this. ESL files also have other important restrictions to consider, which are detailed here: https://www.creationkit.com/fallout4/index.php?title=Data_File
Cleaning the Official Master ESMs This guide assumes using TES5Edit v3.2.1 on Skyrim Nexus, Or SSEEdit v3.2.1 on Skyrim SE Nexus Due to this guide being dual purpose ( For Skyrim and Skyrim SE ) for the rest of this guide I will refer to both tools as xEdit. Screenshots of tools used may be one or the other, or older versions, which does not matter, the images are only to illustrate the method / options used. Why Clean the Master Files ? Firstly because the masters have entries that are identical to the same records in Skyrim.esm or other DLC esms'. They exist because Bethesda may have looked at something in the CK and an unneeded entry was auto included in the plugin even though the item was not altered in any way. The Official Creation Kits are notoriously buggy and randomly create dirty / wild edits, often when the author of the plugin is completely unaware. Wherever that plugin is placed in your load order its records overwrite all the conflicting records from plugins loaded before it ( the rule of one ) resetting the settings back to the values contained in the Official Bethesda DLC. It won't cause crashes, it just changes the values of plugins loaded before it. Which can alter mods that you have for Weapon Damage, Armor, Lighting, Food Effects and so on. The masters are very early in your load order but there is potential for a mod to be made as a fake.esm, and placed among them, and so ITMs in a later loading master may cause problems for that mods esm. Chance is remote that a master will affect another master, and this procedure is best used on all of your mods plugins, but cleaning everything of ITMs ( Identical to Master records ) causes no harm, is more optimal giving the game less to process in your load order, and so it is best to get rid of these completely unnecessary dirty edits. The Second reason is that Bethesda chose to delete some things that are in the Official DLC. Any mods loaded with references to deleted records from the Official Bethesda DLC will cause your game to crash. This problem particularly affects older mods ( especially mods that were made before newer official patches were released, with more deleted references the old mod did not anticipate - It will also become problematic for the Skyrim Special Edition community where old Original Skyrim mods are being converted to SSE, and Bethesda have deleted even more records from the plugins before they released the newer plugins for that version of the game ). xEdit can restore and properly assign values to these records that will disable them and still allow mods to access them. This is done using the "Undelete and Disable References" option. For further explanations of why it is still recommended to clean the games masters .. Read on from this post, Zilav and Arthmoor, most valued technical and vastly experienced modding authors, weigh in on the subject. The following mostly apply to mod authors, but worth knowing about for mod users too : xEdit will also report when a mod has Deleted NavMeshes as part of the report from Automatic cleaning. Like deleted references, any mod that references a deleted NavMesh will cause Skyrim to Crash. Properly optimizing your mods NavMeshes and checking your mod for Deleted Vanilla NavMeshes ( which can also be caused by a CK wild edit even if you did not do it yourself ) is important. Mods altering the same cell and the same NavMeshes when your mod is not optimized will cause Skyrim to Crash. Poorly optimized NavMeshes with errors reported by the CK will make Skyrim unstable. Instabilities like fast travelling to a location and Skyrim crashes. Note the ones found to be deleted in the games masters, cannot be undeleted. To fix deleted Navmeshes in your mods, Arthmoor has provided a walkthrough in Skyrim - Fixing Navmesh Deletion in TES5Edit Manually cleaning your mods is also important to remove wild edits. This is mostly down to the experience of Mod Authors to solve such problems, but there are a few noted later in this guide which are in the DLCs which everyone can easily Manually clean. Some mods can have accidental Wild Edits in them caused by the author looking at how Bethesda did something they wish their mod to do as well. These Wild Edits often prevent Skyrim from doing things like advancing quests, spawning NPCs, assigning dialogue to NPCs, preventing NPCs from patrol areas they are assigned to. They can also alter Vanilla Lighting and Triggers that the author wished to use. All of these things affect any plugin with conflicting records loaded before a mod with Wild Edits. Mod authors - Learn to use xEdit, and ensure the only records in your mod plugins are what you would expect to be in there, its the most important tool the community can make use of when used properly. Mod Users - Follow this guide... Before moving on to the Manual cleaning, something everyone should do prior to Manual Cleaning : Automatic cleaning of Bethesda's ESMs with xEdit With the games Original esm's installed ( You can use Steam to Verify Integrity of Game Cache of Skyrims files to ensure you have good error free copies of the original master files ), and in accordance with the following wiki article http://www.creationk...ty_Plugins_List : Load up xEdit. 1. Right click the plugin selection screen and select "none" 2. Tick the relevant esm to edit, and click okay ( If you have not cleaned any of your Master files yet, the first one to tick will be Skyrims "Update.esm" ), then click Okay After each of the following actions, wait for a message in the message window that the previous operation has finished / Done : 3. Right click the plugin after you get the "Background Loader : Finished" message,and choose "Apply Filter for Cleaning" Wait until Filtering is finished then .. 4. Right click the plugin and choose Remove Identical to Master Records Wait until it finishes then .. 5. Right click the plugin and choose Undelete and Disable references Wait until it finishes then .. 6. Close xEdit, and it should check with you that you wish to save the plugin ( this only happens if you have made any changes to the plugin to save, if it just closes .. Then you have not cleaned anything ) Rinse and repeat the Automatic cleaning ( steps 1 - 6 above ) for each of the master files. Working from first to load, to last, not including Skyrim.esm or any unofficial patches ( No point doing Skyrim.esm, and the unofficial patches are already done and should not be cleaned ) So clean in this order Update.esm Dawnguard.esm Dawnguard.esm ( Yes it needs to be done twice ) Hearthfire.esm Dragonborn.esm Dawnguard.esm needs to be cleaned twice ( as of xEdit 3.1 onwards - After doing the automatic cleaning routine once on Dawnguard.esm, and saving it, load it up again in xEdit and you will be able to clean a further 6 ITMs ) : Skyrim Original Update.esm Apply filter for cleaning [Removing "Identical to Master" records done] Processed Records: 1290, Removed Records: 96 [Undeleting and Disabling References done] Processed Records: 1194, Undeleted Records: 3 Dawnguard.esm ( First time around ) Apply filter for cleaning [Removing "Identical to Master" records done] Processed Records: 98027, Removed Records: 630 [Undeleting and Disabling References done] Processed Records: 97397, Undeleted Records: 82 <Warning: Plugin contains 57 deleted NavMeshes which can not be undeleted> Dawnguard.esm ( Second time around ) Apply filter for cleaning [Removing "Identical to Master" records done] Processed Records: 97396, Removed Records: 6 [Undeleting and Disabling References done] Processed Records: 97390, Undeleted Records: 0 <Warning: Plugin contains 57 deleted NavMeshes which can not be undeleted> Hearthfires.esm Apply filter for cleaning [Removing "Identical to Master" records done] Processed Records: 17797, Removed Records: 184 [Undeleting and Disabling References done] Processed Records: 17613, Undeleted Records: 11 <Warning: Plugin contains 5 deleted NavMeshes which can not be undeleted> Dragonborn.esm Apply filter for cleaning [Removing "Identical to Master" records done] Processed Records: 214386, Removed Records: 68 [Undeleting and Disabling References done] Processed Records: 214318, Undeleted Records: 8 <Warning: Plugin contains 1 deleted NavMeshes which can not be undeleted> Skyrim Special Edition Update.esm Apply filter for cleaning [Removing "Identical to Master" records done] Processed Records: 17644, Removed Records: 239 [Undeleting and Disabling References done] Processed Records: 17405, Undeleted Records: 91 <Warning: Plugin contains 2 deleted NavMeshes which can not be undeleted> Dawnguard.esm ( First time around ) Apply filter for cleaning [Removing "Identical to Master" records done] Processed Records: 98676, Removed Records: 621 [Undeleting and Disabling References done] Processed Records: 98055, Undeleted Records: 82 <Warning: Plugin contains 57 deleted NavMeshes which can not be undeleted> Dawnguard.esm ( Second time around ) Apply filter for cleaning [Removing "Identical to Master" records done] Processed Records: 98054, Removed Records: 6 [Undeleting and Disabling References done] Processed Records: 98048, Undeleted Records: 0 <Warning: Plugin contains 57 deleted NavMeshes which can not be undeleted> Hearthfires.esm Apply filter for cleaning [Removing "Identical to Master" records done] Processed Records: 18539, Removed Records: 178 [Undeleting and Disabling References done] Processed Records: 18361, Undeleted Records: 11 <Warning: Plugin contains 5 deleted NavMeshes which can not be undeleted> Dragonborn.esm Apply filter for cleaning [Removing "Identical to Master" records done] Processed Records: 214498, Removed Records: 69 [Undeleting and Disabling References done] Processed Records: 214429, Undeleted Records: 8 <Warning: Plugin contains 1 deleted NavMeshes which can not be undeleted> ------------------ Dawnguard.esm needs manual cleaning aswell as automatic cleaning After the automated cleaning is done, you can also now manually clean a few more Wild edits xEdit will not have touched ... Recently Arthmoor has brought to the attention of the community additional information regarding manual cleaning of Dawnguard.esm, which everyone needs to do for their own setup same as automatic cleaning ( because nobody can legally upload official master files anywhere, everyone needs to do their own ) First load up xEdit When the plugin selection comes up, right click and select None Then put a tick in the box just for Dawnguard.esm, click Okay After its finished loading, right click Dawnguard.esm and choose "Filter for Cleaning" 1. For "CELL 00016BCF: Remove XEZN subrecord referring to RiftenRatwayZone [ECZN:0009FBB9]. Otherwise it blocks the official fix in Update.esm." .... Expand the records as in the following screenshot, and right click the indicated sub-record, and choose Remove 2. For "CELL 0001FA4C: Wild edit. Remove this record. It's a testing cell." .... Expand the records as in the following screenshot, and right click the indicated record, and choose Remove 3. For "CELL 0006C3B6: Wild edit. Remove this record. It's a testing cell." .... Expand the records as in the following screenshot, and right click the indicated record, and choose Remove NOTE : This guide used to include cleaning instructions for "CELL 00039F67: Wild edit. Remove this record. It's a testing cell" ( The WICourier edit ) - But since the new version of TES5Edit 3.1+ now cleans that as part of the automated cleaning ( which you should have done prior to manual cleaning ), you no longer need to clean it manually afterwards. ----------------------------------------- CRCs of the various stages for comparison ( The following cleaned with xEdit 3.2.1 ) Skyrim Original Update.esm Original - CRC = E5B67BDA MD5 = 2476E42699D4D9236DDD2EA8B31F612D Update.esm Auto Clean - CRC = 18401373 MD5 = 27CF2C3E379BC853F4AC574BCEC7A0B9 Dawnguard.esm Original - CRC = BD72CCF3 MD5 = 5BD2F339925A5FB96F9F0DA7F96FD8CE Dawnguard.esm Auto Clean First time - CRC = 6391C6DC MD5 = 0602D8450CC51F8C9EF8AB531B6EDAEB Dawnguard.esm Auto Clean Second time - CRC = 94A47F31 MD5 = D3D38017BCDA07C1ABB1C343A9266634 Dawnguard.esm Auto plus Manual Clean - CRC = 50FB6A39 MD5 = 74561A2CF8644ACC8BE857E0705996D5 Hearthfire.esm Original - CRC = AF82CE6A MD5 = 71C3D20EA9F8510E29C36F2A426247C2 Hearthfire.esm Auto Clean - CRC = FBB4B8FA MD5 = E91E103C7F541C34F7CDD53AD68EDA3A Dragonborn.esm Original - CRC = A9F83BFF MD5 = 37D8CBFFCB460011CE2BF3AABD007A2D Dragonborn.esm Auto Clean - CRC = E0715431 MD5 = 7D78A3CA1E4D3700DB5D7F5DDAF5F7D2 Skyrim Special Edition Update.esm Original - CRC = fe610a99 MD5 = 70D089E0F55620127E8087A19285EC4B Update.esm Auto Clean - CRC = c18ee18c MD5 = 0D1793153B88691EAD5865DACD4D0EC0 Dawnguard.esm Original - CRC = 81a481e8 MD5 = 055FAAD4A75321F6439ABA05BEA9FD06 Dawnguard.esm Auto Clean First time - CRC = 6f162d5e MD5 = D449836BD497593E967A4AE880E9DD75 Dawnguard.esm Auto Clean Second time - CRC = 6061bcff MD5 = 192F2D3C115B9CD859192DECBA89D7A5 Dawnguard.esm Auto plus Manual Clean - CRC = 141be1e5 MD5 = 18E1A883F28F938AAAF720EAFC39D2BE Hearthfire.esm Original - CRC = a631401b MD5 = B42D76EA5E863FD47DBB83213FF00484 Hearthfire.esm Auto Clean - CRC = 0ed37282 MD5 = 3EF5D1E32AF8BFA6568A0E9BE408737E Dragonborn.esm Original - CRC = 64625bcd MD5 = BF9B92C5B70EDF5C4DE074B64532A30A Dragonborn.esm Auto Clean - CRC = 5eb9eade MD5 = C075133B06C986E7218DAA59C1D8906F ( Note : If there is only one file inside a folder within a zip, 7zip shows that files CRC alongside the folder - I checked each plugins within the folders individually to be sure the Screenshot CRCs above are all correct, and of course they also match the online CRC checks ) Now that the Master files are cleaned, you can go through the rest of your Load Order using Automatic cleaning of ITMs and UDRs on all your mods plugins. The sequence of cleaning mods plugins should be after you have your Load Order correct, clean them with the last to load being the last to clean. Mod authors should have done them already, so most will probably not need cleaning. Also look out for any mod specific cleaning instructions in the mods description. For example the Unofficial Patches will not need any cleaning, they are already done, and the remaining ITMs in those plugins should be left alone because they do have a purpose .. ( its a very rare occasion when this is true ). To check CRCs Drag and Drop resulting files here https://emn178.githu...2_checksum.html To Check MD5 checksums Drag and Drop resulting files here http://onlinemd5.com/ The xEdit Work In Progress Development topic is at the following link http://afkmods.iguan...-20#entry152079
Now that this kalpa for the USKP is complete (for you non-lorehound types, kalpa = cycle) it's probably time we make public our plans for the next one. Due to an increasing number of odd engine related bugs it's been decided among the team that we will need to convert the USKP (and therefore all the DLC patches too) into what's called a "False ESM". That is, and ESP file which is flagged as an ESM by way of changing the flag in TES5Edit. So why would we want this? There are a few classes of bugs this has already been shown to help with in internal testing: Seemingly random crashes in and around major cities, and other areas where a lot of data is edited. Especially navmeshes. The "landscape foliage" bug in which a child worldspace begins displaying landscape foliage from its parent worldspace because an object in the parent worldspace was edited. This is readily apparent in Darkwater Pass. The "LOD trees show up at close range" bug. This is where you will sometimes see trees that are clearly intended to be displayed as LOD sitting within the currently loaded cells, sometimes even merged into the real trees. As described here. There may be others, but these 3 classes of bugs are confirmed to be correctable by converting to a false ESM. So you might be asking then. Why not a proper ESM file? Two reasons mainly: 1. Steam cannot support ESM files, we'd have to cease distribution there all together and not just for the USKP. 2. Other mods are relying on the filenames being ESP files and not all of those mods are currently being maintained anymore. It's already been verified that Wrye Bash is OK with changing the flag. It will not complain when the files get bumped up into the ESM section. With proper CRC checks, BOSS won't complain either. Though I'm hoping some other method can be used to detect it ongoing because having a growing mountain of CRC checks won't be helpful. Perhaps version number parsing from the description field. I cannot vouch for how NMM will react since I don't use it, and I'm not sure anyone else on the team has tried it either. Who knows what Mod Organizer will think. We will bump all the version numbers on all the patches up to 2.0 when this happens. We don't have a time frame yet, because there's also a giant mountain of vanilla bugs piled up in the tracker to address too. But when it happens, all of the patches will be updated at the same time. Our proposed changes in the load order for the game will look like this: Skyrim.esm Update.esm Unofficial Skyrim Patch.esp Dawnguard.esm Unofficial Dawnguard Patch.esp Hearthfires.esm Unofficial Hearthfire Patch.esp Dragonborn.esm Unofficial Dragonborn.esp [The rest of your stuff] The main reason for this should be obvious. It will generate much less interference with things following on in the ESM group, and at the same time allow us to undo some of the changes in each DLC patch that are only there because the USKP canceled them out due to its own edits. Best of all, thus far in testing, it poses no risk for converting them in the middle of an active playthrough, even with a lot of other mods loaded. So anyway, that's where we're headed now. We've already kind of leaked hints at it, so now it can be considered confirmed rather than rumor The ESM Conversion Checklist Wrye Bash Support - Supported as of release 304.2. NMM Support - Works as expected, NMM flags the files with a warning but allows them to move into the ESM group. Mod Organizer Support - v1.0.0rc1 and up has been confirmed to work. TES5Edit Support - Supported as of release 3.0.31. BOSS Support - Working in 2.1.1. (was some confusion here, but it *IS* working) Checklist items need to be completed before we move forward on this. BETA Scheduling We're planning to go into beta on all 4 patches Saturday (October 5). I expect we'll have a minimum of 2 weeks testing, perhaps longer, to see how all this affects things and to validate that none of the new fixes coming in with this are broken. With that in mind, I figured this is as good a time as any to say so. Once this round of beta testing is complete, the USKP will no longer be distributed on Steam. Leaving the 1.3.2c version up there is already causing us problems and we can't in good conscience leave it up there after 2.0 has gone live. So anyone still using Steam to get the USKP needs to transition to one of the other available download sites. We're also going to have to check to see if Steam will accept a falsely flagged ESP file and deliver it back to the user intact as such. I mention this now because if it doesn't, then ALL of the unofficial patches will have to be taken down from Steam and everyone using those too will need to make provisions to change to another download site. So keep that in mind for those of you using Steam.