Zugg Software :: View topic - FASTER string lists and database variables in CMUD v2.0!

Posted: Fri Jul 20, 2007 4:30 am

When I was working on some problems with the Lua implementation in CMUD 2.0, I decided I really needed to change my horribly inefficient routines for string lists and database variables to something better and faster. So far I have just improved the string lists (database variables will get their turn next week). But the results are astounding!

Here are the two scripts that I am testing:

Posted: Fri Jul 20, 2007 7:42 am

That's great news Zugg. My stuff isn't tooo heavy but I know the heavy hitter scripters are drooling over this. I've been talking CMUD up and I suspect 2.0 will blow away 1.0 make others go "zMUD who"

Keep up the great work!

Beginner Joined: 07 Sep 2005 Posts: 12

All kinds of speed improvements are always very welcome, so this is great to hear! And the ultimate part is that it is automatic. What I am curious about though is if it will automatically apply to converted zMUD scripts as well?

//Ice

Novice Joined: 26 Apr 2007 Posts: 44

This is awesome.

Posted: Fri Jul 20, 2007 6:01 pm

Posted: Fri Jul 20, 2007 11:11 pm

#SORT

That's verra noice, Zugg. My whole curing system is based around using %sort on a stringlist. And now you say it will sort itself automatically, faster. I'm quite happy now that I made the right choice in sticking to my stringlist based system, rather than being tempted by #switch. Wink

The begin/end-update part, I will rarely use, though I can see one use for it in capturing my inventory. Nice I suppose, though without needing to do it at any time-crucial ..times, I'm more excited about #sort.

Anyhow, great work.

Posted: Fri Jul 20, 2007 11:21 pm

Surely you can use it for afflictions as well. They're not the kind of thing you need to keep track of between sessions. I was actually thinking about this and there are very, very few things you need to keep track of between sessions - stat maximums if they're not on your prompt and you can't be bothered to #event onconnect score, preferences... that's about it.

Adept Joined: 18 Dec 2005 Posts: 246

NICE!!! I have a couple stringlists that are 1200 or so entries long that I loop through so this will be great!!!! I know 2.0 is coming out, and will be a vast improvement over the already great 1.34, but admittedly it sounds like some pretty root level changes and I'm a bit apprehensive about using it as soon as it's released... Sooo my question is, will 2.0 be released as a public release, or as a beta release???
Just to reiterate, I'm a HUGE fan of CMUD... Just a tad concerned about the debugging period after what appears to be a major overhaul to some core level stuff.
Jed

Apprentice Joined: 12 Jul 2004 Posts: 190 Location: Canada

OK... maybe I'm going WAY too far with this, but is there a possibility of an optional argument to sort... a pointer to a compare function that accepts two parameters, that can be used for the sort function?

Why this might be useful...

Imagine you're storing a list of unique identifier keys in a string list... information about each unique key is stored in different lists.

Let's say the UIDs reference weapons. You could have information such as "item level", "damage type", "damage dice", "avg damage", "weapon type", etc etc stored in parallel string lists.

Now, let's say I want to sort my uid list by level ASC, and then by avgdam ASC... I could easily write a compare function that takes two UIDs (A and B) and returns 1 if A > B, 0 if A ==B, and -1 if A < B. This way, you could sort by any criteria you want.

Now, before I get replies saying "this could easily be done with db"... I currently avoid the db module like the plague. I will probably make HEAVY use of it when it gets rewritten, but personally I dislike using it currently. :)

Posted: Sat Jul 21, 2007 2:50 am

Jed: It will definitely be a BETA release. My guess is that it will take a couple of months to get it back to a Public release state. That's exactly why I am doing so much new stuff at once. If I add something that might break stuff and cause bugs, then I might as well add *lots* of new stuff at the same time and just make it really clear that it's a BETA version.

Thinjon: As you mentioned yourself...doing this would be more appropriate in a database. I could do this for the %sort function, but not the #SORT command. Delphi doesn't have any hook in their component to keep a list sorted in a custom manner as new items are added to it. I'd rather put my time into getting the database module rewritten in the future.

Also, you can probably already do this with the Lua implementation in CMUD v2.0. There is a lot more power in Lua, especially when it comes to stuff like custom functions.

Apprentice Joined: 12 Jul 2004 Posts: 190 Location: Canada

Nodnod... I haven't attempted to do it in Lua yet, but I've already written custom-sort aliases for CMud... the concept of sorting isn't that difficult, I just thought, if possible, that it would make a great addition to the new sorting algorithm... the speed benefit would be awesome :)

Posted: Sat Jul 21, 2007 3:13 am

At the risk of derailing the thread, I'll just mention that all sorting in Lua is custom sorting. You give the function that sorts the table a function as an argument. The argument takes two parameters that're items of the list, and returns true or false depending on whether or not the first should come after or before the second.

Apprentice Joined: 12 Jul 2004 Posts: 190 Location: Canada

Oh, Zugg... another question...

In the usage of BEGINUPDATE and ENDUPDATE... will these prevent accessing a variable between aliases/triggers? i.e. if I have a set of triggers that loop over several lines captured from the mud, and each triggering line is going to add something to a string list, would it work to use BEGINUPDATE before the first line, and then ENDUPDATE at the end of the trigger set?

If so, would the speed benefit still remain?

Posted: Sun Jul 22, 2007 11:28 am

It seems like #endupdate just suspends database writes and not internal cache writes. You'll be able to change the value of something that you've #beginupdated but no changes will be saved to disk until you #endupdate it. It'd be quite useless if you made a "Use Default" variable and then couldn't change its value with anything.

Apprentice Joined: 12 Jul 2004 Posts: 190 Location: Canada

Well, I figured that it would be possible to write to the variable while it was under the effect of #BEGINUPDATE... what I was more curious about was the scope of the effect. i.e. if I intend to append values, do all my writes have to occur in the same alias/trigger between the calls to BEGINUPDATE/ENDUPDATE ?

Let's say I have an alias that checks/catalogs my inventory. The alias enables a class, sends the MUD 'inv' command, and executes BEGINUPDATE on my @invlist string list.
Now, one of the triggers in the enabled class fires for each line in the 'inv' return... let's say the list looks like this:

a wooden spoon [1]
a rusty dagger [5]
(2) a flying potion [1]

On the first trigger catch, I might add 'wooden spoon' to @invlist. In the scope of this trigger call, I'm sure @invlist would then equal "wooden spoon".
No, on the second trigger catch, if I add "rusty dagger", does @invlist = "wooden spoon|rusty dagger" or does the value from the first trigger catch carry over under the effect of BEGINUPDATE ? If not, then my @invlist now equals "rusty dagger" - within this trigger's scope.
The third trigger catch would yield either "wooden spoon|rusty dagger|flying potion" or just "flying potion"

Now if I have another trigger, perhaps that fires on ^$, that disables the class, and executes #ENDUPDATE on @invlist, I could have either an empty list, or have 3 items in it.

Perhaps I don't know enough about the internal workings of z/CMud variables... but I figured it couldn't hurt to ask what this effect would be.

Apprentice Joined: 29 Jul 2003 Posts: 101 Location: USA

Do you have an estimate of what month we may see v2.0? It sounds like its going to be great.

Posted: Sun Jul 22, 2007 5:11 pm

Thinjon: That's why I gave the "Use Default" example. Zugg said above that "Use Default" now works by setting #BEGINUPDATE for the variable as soon as CMUD starts. If that's the case, then the variable must still work exactly the same as it normally would, but database writes would be suspended. Your triggers will all write to the internal cache variable as they normally would, but the new values will only be written out to disk when you use #ENDUPDATE.

Yamabushi: Zugg's last estimate was either before or after the Sony Fan Faire. That's August 2nd-5th, so it'll be around then :)

Wizard Joined: 17 Jun 2006 Posts: 1201

Wizard Joined: 17 Jun 2006 Posts: 1201

Zugg,

Hmm I copied your little scripts of makelist and searchlist and ran them on my computer with version 1.34 and came up with the following:

makelist = 312

searchlist = 219

Posted: Sun Jul 22, 2007 9:10 pm

I get similar results, but that doesn't mean that you won't still see a speed improvement in the new version.

Wizard Joined: 17 Jun 2006 Posts: 1201

Posted: Mon Jul 23, 2007 12:46 pm

Posted: Mon Jul 23, 2007 5:30 pm

oldguy2: The timings that I posted are for the debug version of CMUD that I'm working on. You should expect to get much faster numbers because the version that you are running is the non-debug optimized release version. When I'm running CMUD for development, I have all sorts of debug stuff enabled that slows it down. Also, you never know how fast your computer is compared to mine. My computer is a few years old (I cannot game on this computer, for example).

In any case, while the exact numbers might change once I've got the compiled and optimized release, there should still be a similar huge difference in speed.

Back to the #beginupdate issue...remember that I mentioned that there is an additional argument to #beginupdate to control how the internal cache is updated. Well, I'm going to change things a bit to make this easier:

#BEGINUPDATE varname

This also turns off the writing of the variable to the database, but for string lists and database variables, it also disables the update of the string value of the variable. This case will give you more speed when updating the contents of a string list or database variable. CMUD stores the string list internally now as a hash table. But when you display the value of the variable (like with #SHOW @var) then CMUD needs the string value of the variable (like "Item1|Item2|Item3" etc). So normally, when you use something like #ADDITEM to add to a string list, the item is added to the internal hash table *and* the string value is recomputed. When adding a lot of items, there isn't any need to keep recomputing the string value until you are done. Using #ENDUPDATE tells CMUD to recompute the string value of the variable based upon the hash table.

#NOUPDATE varname

This will simply turn off the writing of the variable to the database. This is the same as the previous "#BEGINUPDATE varname 1" syntax. This has the same effect as setting the Use Default option for the variable (except that you can do it for any setting, not just variables).

Doing a #NOUPDATE varname or #NOUPDATE varname 0 will turn off updates (same as enabling the Use Update option). Using #NOUPDATE varname 1 will re-enable the update flag.

This will help separate the difference between #BEGINUPDATE/#ENDUPDATE which MUST be given in pairs, and mainly applies to string list and database variables, and the #NOUPDATE which can be used any time you want to stop updating a setting to the database file.

So, back to string lists: Using #BEGINUPDATE *will* cause problems if you try to access the value of the variable before the #ENDUPDATE call. For example:

Posted: Mon Jul 23, 2007 6:07 pm

And you can use #beginupdate and #endupdate on a variable that you've previously used #noupdate on, I assume, since they seem to do totally different things.

Posted: Mon Jul 23, 2007 6:34 pm

Yes, the #BEGINUPDATE/#ENDUPDATE are separate from #NOUPDATE.

And actually, I think I'm going to call it #NOSAVE instead of #NOUPDATE. That better matches what it is doing. Also, I'll add a "Saved" checkbox next to the "Enabled" checkbox in the settings editor to control this flag.

To try and make it perfectly clear: If you have used #NOSAVE on a variable, then the BEGINUPDATE and ENDUPDATE will still work (and still prevent string lists and database variables from being updated), but after the ENDUPDATE, the variable still won't save it's value to disk (until you turn off the NOSAVE option).