 |
MrCheviot
Novice
Joined: 02 Dec 2007
Posts: 42
|
 Posted: Sun Jan 13, 2008 4:52 pm
Best Practice? Temporary data storage in database (or db-like) setup
|
My need is for a quick, flexible way of creating/storing/retrieving data from a database-like setup/structure. My own tests in ZMUD showed that it was better to use an external database vs. the internal db that comes with the software (duplicated a database of approx. 500 records, speed difference was incredible). But given Zugg's rewrite of hash tables along with compiled code in CMUD, I'm wondering if the speed advantage of new hash tables is worth writing the supporting code for vs. leaving it in an external database. For data concerns, assume the table is temporary or contains nothing you'd cry about if it became corrupt or was lost.
Let me use a simple example: a small SQL table for damage types
| Code: |
DamType Action Weapons
slash slash|cleave|slice sword|halberd
pierce pierce|poke dagger
bash bash|smash|smote mace|club
|
How could/would this best be represented in datarecords? You could have datarecord with key as damtype, then have a string list for each, e.g.
| Code: |
key = slash value = (slash|cleave|slice)|(sword|halberd)
key = pierce value = (pierce|poke)|dagger
key = bash value = (bash|smash|smote)|(mace|club) |
But if you have a lot of fields (30 enough?) then keeping track of which item in the value list corresponds to a given field becomes cumbersome - you'd almost need to maintain a separate schema list (action|weapons|field3|...), then setup functions so you could easily get or update the information you wanted, e.g. @dr_get(@damtype,key,schema_item) or @dr_write(@damtype,key,scheme_item,newval). , In this case you'd need to write functions to allow you to get all actions or all weapons (akin to SELECT Action FROM DamTypeTable) which would #loopdb and, according to the schema, generate a list of items at that position in each value. I think there's potentially more to worry about re: special characters in the values (escaping chars, use of strings, etc.) And not to bemoan the point, but there are sorting issues as well. Would adding this be more/less noticeable in terms of overhead than using ADO?
You could also create a datarec for each damtype, e.g. damtype_sword with key = action value = slash|cleave|slice key = weapons value=sword|halberd.. etc. to simplify some of the schema and access concerns, but many of the same issues I just mentioned still apply.
Just for comparison, external SQL setup would require functions to read and write to the database, either through ADODB.Recordset Update method or an INSERT statement, as well as functions to parse the data from ADODB.Connection/Recordset back into string lists or even datarecords (a la Vijilante's Toolbox).
I'm not a fan of recreating the wheel, but speed and functionality are quite important to me. Obviously an external SQL database is built to do these things, but before I setup an elaborate proof of concept, I'm wondering anyone else has done the trials on this w/ new hash tables? Does anyone have an opinion about the best method to store and access (semi-)temporary data in CMUD? Any insights into new internal db (which should be the best way to do this when it arrives) in terms of which method would be best to convert?
Looking forward to seeing what this stirs up..
-MrC |
|
|
|
|
Guinn
Wizard
Joined: 03 Mar 2001
Posts: 1127
Location: London
|
|
_________________
CMUD Pro, Windows Vista x64
Core2 Q6600, 4GB RAM, GeForce 8800GT
Because you need it for text... ;) |
 |
|
|
MrCheviot
Novice
Joined: 02 Dec 2007
Posts: 42
|
Posted: Sun Jan 13, 2008 6:34 pm
|
Interesting... I've read that Lua is fast (no experience with it myself), but after looking at it..
We'll use the term dictionary where an element pair has a key and a value.
Wouldn't that make a Lua table/dictionary more or less the equivalent of a data record in zScript? Only diff being, you can set the value in a Lua table to another table, which would be similar to setting value in data record to string list.. or actually another data record.
That gives me an idea.
-MrC |
|
|
|
|
MrCheviot
Novice
Joined: 02 Dec 2007
Posts: 42
|
Posted: Sun Jan 13, 2008 6:44 pm
|
This is pretty cool:
| Code: |
#va dr_db {}
#va recA {}
#va recB {}
#addkey recA field1 "valueA1"
#addkey recA field2 "valueA21|valueA22|valueA23"
#addkey recB field1 "valueB1"
#addkey recB field2 "valueB21|valueB22|valueB23"
#addkey dr_db recA @recA
#addkey dr_db recB @recB
#show db_dr: @dr_db
#show recA,f1: %db(%db(@dr_db,recA),field1)
#show recB,f2: %db(%db(@dr_db,recB),field2) |
Output:
| Code: |
db_dr: recA="field1=valueA1|field2=""valueA21|valueA22|valueA23"""|recB="field1=valueB1|field2=""valueB21|valueB22|valueB23"""
recA,f1: valueA1
recB,f2: valueB21|valueB22|valueB23 |
So much for needed a schema to keep things organized. This is pretty slick, and clearly writing functions to facilitate embedded %db() calls is doable. I wasn't even aware that this could be done w/ datarecs (not sure why...), but it works in both ZMUD and CMUD...
Simplifies my orig post a bit, but the question remains..
-MrC |
|
|
|
|
Fang Xianfu
GURU
Joined: 26 Jan 2004
Posts: 5155
Location: United Kingdom
|
Posted: Mon Jan 14, 2008 12:16 am
|
Bear in mind that if you have a string list within a string list, or within a data record table, it won't be hashed. Only the highest-level string list or data record is hashed.
You're right that the only difference between Lua tables and zScript data record variables/string lists is that values can be tables, but don't discount this, it's very useful. This thread has a good example of a multi-layered data structure that's very complex in zScript but very simple in Lua. |
|
|
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
