Zugg Software :: View topic - [3.03a / 2.37] I loose triggers, aliases, variables, etc...

Magician Joined: 18 Dec 2006 Posts: 317 Location: Denmark

I'm loosing triggers, aliases, etc. (is there one word that covers all these elements?) now and then. I cannot figure out a pattern and I usually only find out when suddenly an expected trigger is not firing or an alias called by a script is not recognized.

I just lost a rather complicated alias. Very frustrating... Sad

EDIT: I wonder if this bug is related to the one where elements are copied to the root.

Posted: Wed Feb 04, 2009 7:37 pm

"Settings" is the word we generally use to mean triggers, aliases, and everything else. I know, it's rubbish, but it's what we've got. If you can think of something better, please suggest it :P

Which OS are you using, and where did you tell CMUD to put your user data when you installed it?

Magician Joined: 18 Dec 2006 Posts: 317 Location: Denmark

OS is Windows XP and my user data is here: C:\Documents and Settings\Bjarke\My Documents\My Games\CMUD

Posted: Thu Feb 05, 2009 5:48 pm

Is everything in a single package, or are you using multiple packages? Was the complicated Alias that you lost something that you created and edited in the Settings Editor, or was it something you created from the command line. Did you do a Search for it in the Settings Editor to see if it got put is some other class somewhere?

CMUD updates the database on your hard disk with the settings stored in memory about once per minute. So in theory, if CMUD crashes you might lose any settings that you have added in the past minute.

But once CMUD updates the database on disk, it's very hard to actually remove something from the database unless you specifically delete it (or remove it indirectly like with deleting an entire class folder)

And yes, if you are having problems with settings getting put into the root of your package, then that might be causing some of these other problems. You should do a full XML export of your package and then import it into a new package to make sure there isn't anything corrupted in your package. If an Alias gets put into the root of the package, then it probably won't work anymore.

Magician Joined: 18 Dec 2006 Posts: 317 Location: Denmark

I don't use different packages and the settings I usually make in the settings editor. I create temporary triggers and variables in scripts but that's about it.

I'm quite sure that settings have been lost without crashes as I haven't had any of those since I disabled my status bar references.

Is there any way to tell that a package is corrupted or do you just have to try exporting/importing and see if things get better?

Magician Joined: 18 Dec 2006 Posts: 317 Location: Denmark

Is there a way to export all windows or do I have to take them one at a time?

I ask because I have never exported and imported before so I am not quite sure of the procedure...

EDIT: Think I found a way now but it seems when I do it I get a new main window and my old main window will be placed elsewhere. Any good advice one how it should be done in the most painless way?

Posted: Wed Feb 11, 2009 4:28 pm

CMUD will only export what you have selected (unless you chose export all). So select all the settings you want to export (and not the windows) and you should be set. Of course, you may need to put them back under their specific windows.

Magician Joined: 18 Dec 2006 Posts: 317 Location: Denmark

Posted: Wed Feb 11, 2009 6:29 pm

You could email me your *.pkg file at sales@zuggsoft.com and tell me the name or contents of the settings that disappeared and I can look at it to see if there is a problem in the file.

The only current problems with the Export/Import functions is if you use a script language other than zScript (like Lua), and if you are exporting Events (the name gets changed during Import). Otherwise just importing the XML file into a new package file should work fine.

Finally, are you getting *any* sort of crash error messages when using CMUD? If you continue to use CMUD after getting a crash, then you might have problems with your settings not saving.

Posted: Wed Feb 11, 2009 7:51 pm

Not really. There are some rules of thumb like making sure you only have windows and modules at the root level for instance. Most others are really hard to tell. The few that can be done automatically Zugg usually codes to have run when you load you package.

[Edit]Ninjaed by the master

Magician Joined: 18 Dec 2006 Posts: 317 Location: Denmark

Posted: Tue Feb 17, 2009 6:15 pm

I received your package file and will take a look at it. Be patient...I'm in the middle of some other feature coding right now, so I might not be able to get to it until next week.

Magician Joined: 18 Dec 2006 Posts: 317 Location: Denmark

I'll be patient. Thanks.

Posted: Tue Feb 17, 2009 10:54 pm

You know, I keep losing settings, too. But it's sporadic. For instance, I made a regex trigger last night. Saved the package. Created a folder for a new script. Put two triggers (zscript style) into that folder. Saved. Closed CMUD, restarted. The regex trigger did not save, but the folder created AFTERWARDS did save. Interesting. I've also lost settings that had not been lost in the past, either, like some variables and aliases. Some save, some don't. It is getting frustrating. Hopefully it can be fixed...

Charneus

Novice Joined: 01 Dec 2007 Posts: 49

Same thing happening with me.

Noticed it a while ago, but the missing setting was nothing major. I noticed it in particular, that a trigger line I created was missing next session, remade it, and then was gone a few sessions later. (Using the term session of exiting Cmud and then restarting it some time later).

Although I haven't noticed any missing settings for the last few weeks.

All created within Settings Editor. The ones that usually go missing are in the Default Window/Package for a new session. I also did, FILE > SAVE/AS, after the second time, but it still disappeared..

Posted: Thu Feb 19, 2009 6:41 pm

OK guys, I really need your help on this. Please remember that the purpose of the Beta versions is to help beta testing. If you aren't willing to help with beta testing or don't have the time to help track down this problem, then you shouldn't be using a beta version and should go back to the public version. It's getting very frustrating to get getting such vague information on this potentially serious bug.

Posting the "I have this problem too!" doesn't help me at all. I need someone who is willing to mess around with trying various ways to make this happen. Or at least to take good notes to write down exactly what setting disappeared and what you were doing at the time it disappeared.

Here are some specific tests that I'd like people who are having this bug to try:

1) Are you using the mapper when this happens? In other words, is the mapper window open? If so, then if you close the mapper window and don't use it, do settings still disappear?

2) Is the Settings Editor open when the settings disappear? If you keep the settings editor closed, do settings still disappear?

3) Do the settings disappear only after restarting CMUD (like they are not getting saved), or do they disappear in the middle of the session. The problem kjaerhus is having seems to be with settings disappearing during a session. The post from Charneus is talking about settings that didn't save and don't show up when restarting CMUD, which is potentially a completely different problem. So let's try to stay focused on the original poster's issue.

There is a big difference between a setting disappearing within a single session and having a setting not get saved. There are several ways for a setting to not get saved, but there are many fewer ways to have a setting disappear within a single session. So that's the problem I want to focus on in this thread please.

4) What kind of settings are disappearing? Is it only triggers, or also aliases, classes, etc.

5) Are the settings that disappear in the main window, or are they in some other window or module?

6) Finally (and MOST IMPORTANT): Is this problem new to the 3.x beta version, or does it happen in the previous 2.37 version too? You can install 2.37 and 3.03 to different directories and share the same sessions and packages to test them both.

kjaerhus: I looked at the file you sent me and didn't find anything obviously wrong with it. But there are a *lot* of settings in there, so it's nearly impossible for me to look at every one. And it looks like you have lots of mapper room scripts, so obviously you were also using the mapper.

The reason this is frustrating is that CMUD uses a database to store settings, and a database does not magically lose data. If there was some problem in SQLITE causing it to lose data, the Internet would be full of posts about it. So settings just can't magically disappear. They can only get removed by something specifically deleting them. So I need to track down where in CMUD your settings might be getting deleted, even without using #UNALIAS or something obvious like that.

If this problem is new to the 3.x beta version, then my biggest suspect would be the new mapper. The old mapper used to create/delete settings for the room scripts as you moved around the map, so I'd guess that there is still some code left that is trying to delete a room script, but since room scripts have been moved, maybe it's deleting some random setting instead. That's why it's so important to find out if the problem still occurs when not using the mapper.

I'd really like to track this down before the next beta release, but I'm going to need a lot of help from beta testers to try and track this down. I don't want CMUD to get a bad reputation for randomly deleting settings, so I want to squash this problem right now. But it's really up to you guys to post as much information as you can in your testing reports.

Posted: Thu Feb 19, 2009 7:26 pm

I know I use the mapper in my sessions. I have seen settings disappear while the session is active. The problem with this (and I've tried testing this myself to see if I can recreate it consistently) is that it's not always the case. Tonight, I will do a loop test, with and without the mapper open, and see how many, if any, triggers, aliases, and variables are lost. You may be right about it being the mapper issue. Thought hadn't crossed my mind honestly, but I can say that I've never seen this happen in 2.37, and I used the mapper then, too.

In either case, I'll work on it extensively tonight. It's not that we aren't testing, it's just we want to provide concrete details how to create it, as well. It may even be related to the settings being moved to root, too.

Charneus

Posted: Fri Feb 20, 2009 4:50 am

Ok... as promised, I've done some extensive testing. The results are not what I've expected. I can't tell you what the the problem is, but it does appear to be something with the database, or at least modules. This was a long process (almost 30 steps), but necessary to cover the most possible bases (and actually, repeating these steps led up to what I currently have as my setup, basically).

Step-by-step runthrough:

1. Created a brand new session called 'testdatabase.' Didn't put any specifics in since I'm not connecting to any MUDs.
2. Opened Settings Editor, went to View->Show Default Packages, and removed the default packages (to prevent less interference the first run).
3. Closed Settings Editor, went to command line, and entered:

Magician Joined: 18 Dec 2006 Posts: 317 Location: Denmark

1) I have my map open always as I heavily rely upon it to navigate. For this very reason I am using v2.37 at the moment (got both versions installed) as I really can't live with not being able to move between zones by clicking on links. I could try and close my map for a while but since I know of no way of forcing the problem it might be a while before I loose a setting again.

2) I have my settings editor open a lot. Really can't say if it has been open while loosing settings.

3) Not sure but I think in many cases I have noticed the problem shortly after opening CMUD and running some scripts. This also goes for the "settinges getting moved to root" problem.

4) I have never lost buttons and probably never macros either. I think it's probably only aliases, variables and triggers. Perhaps functions too.

5) Almost all my settings are in my main window (as you have probably noticed). I don't recall having lost settings from the other windows although I can't rule out the possibility.

6) I'm pretty sure the last time I lost settings was in 2.37 as I use this at the moment and since I mentioned this version in my original post. I haven't noticed any loss lately though but I haven't added a lot of settings either and it seems that it's mostly new settings that gets lost. Both settings I create in the editor and settings I create through scripts.

Not sure my answers is a big help but I'll keep them in mind next time I loose a setting. Perhaps then I can fill in more details on the loss.

Posted: Fri Feb 20, 2009 11:25 pm

Charneus: Thanks for the details. I definitely reproduced the problems that you mentioned, except for any problem with the #WIN command. But I can verify that #KILLALL doesn't seem to permanently delete any settings and doing the cut/paste to another module definitely seems to cause weird effects with some duplication. So I'll definitely add those to the bug list.

However, as you said, this doesn't seem to have anything to do with the settings getting deleted problem that kjaerhus was talking about, so it probably should have been made into a separate thread.

I think the mapper is the key to making the settings get deleted, especially having Room Scripts in the mapper, like kjaerhus has.

kjaerhus: Since you seem to be having a tough time getting any disappearing settings, I'm not sure what else I can do. I can't do much about v2.37 since so many things have already changed for the 3.x beta versions. The mapper in 2.37 definitely *could* delete settings. Remember that the mapper in 2.37 is a direct port of the zMUD mapper and is not well integrated into CMUD and contains *many* bugs and problems. So if the problem only happens in 2.37, then it's likely the issue is already fixed in v3.x. As I mentioned before, the mapper in 2.37 tries to delete settings and recreate them every time you execute a room script. If other scripts/triggers fire in the background when CMUD is in the middle of executing a room script, then I can definitely see potential problems with that. But the 3.x mapper doesn't delete settings from the mapper...it just enables/disables the new RoomXXX class folders for where your room scripts are. So, unless there is still something left over that I haven't found that is deleting settings, it shouldn't happen in v3.x.

I understand your reasons for using 2.37 instead of 3.x right now, but since you posted this to this Beta forum, I had assumed that you were definitely having problems with the 3.03 beta version.

Definitely try to post additional information the next time it happens, but we'll probably have to wait on this until you are able to use the 3.x version full time to see if that's really where the problem is.

Finally, to anyone else who thinks they are having problem with missing settings, please read the above posts and provide the requested information when posting your problem.

Posted: Sat Feb 21, 2009 4:18 am

Posted: Tue Mar 03, 2009 9:46 pm

Woohoo, I think I tracked down part of this problem.

In the background thread that does the database update (actually outputs the raw SQL statements to update the file), it was accessing the memory cache of the particular record to determine some properties. For example, it was checking the "Temp" flag to determine if the setting was a temp trigger that shouldn't be written to the database.

However, for *deleted* records, it was still trying to access this memory cache. But deleted records have already been removed from memory, so the pointer to the memory cache is no longer valid. When it was checking the IsTemp field, it was returning random garbage for deleted records. If the IsTemp randomly returned "true", then the SQL operation was aborted. In this case, the SQL DELETE statement was being aborted, so the record was not being removed from the database.

It wasn't the moving of records to another module that was causing the problem. The simple act of using Ctrl-X to delete the existing settings was enough.

The only steps that were needed was to populate the session with random settings, then exit CMUD to force these settings to be saved. Then restart CMUD and select all settings and use Ctrl-X to cut them. Then exit CMUD and restart. It would show that some of the settings were not deleted from the database (because the IsTemp check was blocking the SQL DELETE statement). This caused the duplicate issue with the other module just because the original settings in the main folder didn't really get deleted from the database.

What this means is that it was possible for any setting that was deleted to not actually be deleted from the database file, causing it to reappear when the package was reloaded. This could explain several issues with duplicate settings being created. But only if those settings had been previously saved to the database in a previous session.

So this was a pretty serious bug. I don't see any way for this to cause settings to be mysteriously deleted yet. The only time it would screw up was when a record was already being deleted. So it should never block a INSERT or UPDATE SQL statement. But I'm still going to investigate the mapper a bit more to make sure it's not deleting any settings left over from how the old mapper worked.

Very very nasty bug to reproduce and fix. Thanks to everyone for their patience with this, and to Charneus for his procedure, even though it was a bit more extensive than needed to show the bug.

Posted: Tue Mar 03, 2009 11:05 pm

Oh, you're quite welcome. The only reason why mine was quite that extensive is because I wanted to make sure I hit all possible avenues, or at least as many as I could.

One thing about deleted settings, though.. Last night, I had copied a script from a friend (was helping him debug it). Copied it over, worked on it, then went to bed, leaving CMUD open overnight. In the work I did, I consolidated a number of triggers (used REGEX to match basically the same line with minor changes over several triggers). I wound up taking 10 triggers and making them into 3 separate regexes. This morning, I was playing with the socket.http lua stuff Fang had mentioned, and caused CMUD to hang. Probably my fault. I closed out CMUD using task manager, then opened it up, only to find a couple of the triggers returned from what I deleted (using delete, not ctrl+X) in the root file (consistent with what you said) but one of my regexes was gone. With the autosave in place over 6 hours, I probably shouldn't have lost that trigger. I still don't have mapper scripts, so it's confusing, to say the least. Is it possible the deleted info in the SQL database is being retained as a deleted row, then when new information is written into that row, CMUD still thinks that row should still be deleted? Just a thought...

Charneus

Posted: Tue Mar 03, 2009 11:33 pm

Nope, the database itself is robust and won't do anything like that. All database access is done within transactions, and transactions are very safe and stable unless there is some huge problem with SQLite, which I doubt.

The only way to prevent a setting from being saved to the database is if the IsTemp returns true (set to true for #TEMP triggers), or if the package itself is set to ReadOnly (which would have prevented all of your changes from saving).

Doesn't sound like any of these cases would explain the setting being gone.

The only other way I can see something not getting saved is if the setting gets added to the internal database (setting the record state to Inserted), and then somehow the setting gets changed to a state of "unmodified" before it can be saved in the background.

The background saving thread loops through all internal database records and checks the "state" of record. Records marked as "inserted" generate SQL INSERT statements, records marked as "deleted" generate SQL DELETE statements, and records that are "modified" generate SQL UPDATE statements. Records marked as "unmodified" are skipped. So if the state got changed from Inserted to Unmodified before it could be saved, then no SQL statement would be generated.

In theory this shouldn't be possible. The internal database uses checkpoints, and the checkpoint is only set when the background save thread runs. So even if multiple changes are made to a record, the first checkpoint record should still say "inserted" which is what the background saver checks.

Variables are a bit more complex because of the stored hash table for string lists and database records. So the background saver can't just check to see if the string value of the variable is modified...it has to actually check the modified flag for the hash table as needed. But since you lost a trigger, that means it's not a problem with the variable code.

The IsTemp property probably isn't the cause. It cannot be set by the UI, and is only ever set to true when using the #TEMP command at the command line.

I've searched through the code for other cases that might change the database record state, and can't find anything. Also, nothing in this part of the code has changed since 2.37 (according to my version control system), so I can't imagine why it would be happening now and not in 2.37.

As with the rest of the reported cases of disappearing settings, I'm just completely stumped. I can't get it to fail here, and I can't even figure out how it is possible.

Posted: Tue Mar 03, 2009 11:34 pm

Maybe you can email me your *.PKG file and tell me how to search for the trigger that disappeared. Maybe it's really in the database but just got stored somewhere with a wrong pointer so CMUD can't find it? That's all I can think of.