Zugg Software :: View topic

GURU Joined: 10 Oct 2000 Posts: 4379 Location: USA

quote:
I can't help but to wander... hard to imagine that mdac could really be goofy on xp... c'mon - major database api on a major OS. I don't think MS is sloppy enough to let a major flaw ship in something so crucial.

Database applications are something that most home users - Windows largest target market - do not use. Anyway, it is not so hard to believe that they would be so sloppy. One would think that they wouldn't be so sloppy with the core functionality of the OS yet look at the BSOD's that plagged pre-2k and XP versions of Windows. Look also at all the security flaws that are constantly being patched (most of them caused by buffer overflows - a major sign of sloppy programming).

Posted: Tue Aug 12, 2003 12:24 am

How about the fact the the Jet 4.0 SP6 installer from Microsoft doesn't properly install the ODBC library? They even state as much on their web site and state that the solution is to install the Jet 4.0 SP3 update first, and then do the SP6 update? They know about the problem and yet they refuse to fix the installer for SP6 so that it works properly.

Microsoft makes a LOT of sloppy mistakes, ESPECIALLY with the ADO/MDAC/JET software. Just do some searching on the MDAC stuff in the newgroups and you'll see how sloppy many of their past releases are.

So, as a Developer, I'm screwed no matter what I do. If I use a 3rd party database engine, then I have to require everyone using zMUD to buy a license for that database product, and then download and install it, hoping that the 3rd party database works, is fully supported for the life of zMUD, updates itself for every new version of Windows, etc. Or, I can follow the advice of Microsoft and use the Microsoft MDAC stuff, which is built into Windows, and force zMUD users to update their MDAC software to the version that actually works with zMUD and hope that when Microsoft releases new versions of Windows that they properly update their MDAC software.

They got MDAC working fine on Windows 2000, then on WinME/98/95. Then they came out with WinXP and screwed it up again.

So, sorry I tried to follow the requests of zMUD users and spent an entire year rewriting 50,000 lines of code from scratch for the new mapper, trying to make it work exactly like the previous mapper in v6.2x (even though the new mapper doesn't share any code with the old mapper). Only to find out that using a real database format has to deal with all of the bugs in the Microsoft MDAC. Even though it now allows cross-zone speedwalking, 3rd party database access tools, and compatibility with the tons of new enhancements given by zMapper.

So, I apologize if the new mapper is slower for some people. Judging from my email, the majority of zMUD users are working just fine with the new mapper. Yes, some WinXP users got screwed and have a slow mapper until Microsoft can properly fix MDAC on WinXP. Until Microsoft releases a new version, or until I'm actually able to reproduce the slowdown and fix it, you are stuck with this.

But please only post to the thread if you have something constructive to add. This is NOT a SIMPLE problem, and MOST users do not have the speed problem. So, it's not something trivial to fix. Don't assume that just because you have the slowdown problem that everyone else does or that it's a problem with everyone using XP. If it effected everyone using XP, then I'd be able to easily reproduce the slowdown and find a fix or workaround for it.

Wizard Joined: 10 Oct 2000 Posts: 1551 Location: Australia

WARNING: THE FOLLOWING PROCEDURE IS FOR WINDOWS XP ONLY AND COULD BREAK YOUR WINDOWS INSTALLATION
MAKE A BACKUP OF YOUR IMPORTANT FILES FIRST!!

Since I haven't upgraded to Windows XP Service Pack 1 (which is how you update your MDAC under XP as opposed to downloading the mdac update which doesn't actually install under Windows XP) I decided to work out how to manually install it instead.

I downloaded MDAC2.7SP1 from www.zuggsoft.com/data along with the mdac version checker, then went to work.

Using winrar (www.rarlab.com) I extracted the mdac27sp1.exe into a temporary directory, then proceeded to manually install components until the mdac version checker reported that I had MDAC 2.7SP1 (2.71.9030.9) THIS IS NOT THE SAME AS MDAC 2.71SP1 (WinXP) THAT IS INSTALLED BY WINDOWS XP SERVICE PACK 1

This is the procedure I used to manually install (AND IF YOU DO FOLLOW THESE DIRECTIONS YOU DO SO AT YOUR OWN RISK):

right click dasetup.inf -> install
right click rspfiles.inf -> install
right click wdsetup.inf -> install
right click mdacxpak.inf -> install

start -> run -> c:windowssystem32odbcconf.exe /S /Lv c:odbcconf.log /F c:windowssystem32mdaccore.rsp

right click msxmlx.inf -> install
right click sqlxmlxp.inf -> install
right click sqlnet.inf -> install
right click sqloldb.inf -> install
right click sqlodbc.inf -> install

start -> run -> c:windowssystem32odbcconf.exe /S /Lv c:odbcconf.log /F c:windowssystem32sqlclnt.rsp

right click newmui.inf -> install

start -> run -> c:windowssystem32odbcconf.exe /S /Lv c:odbcconf.log /F c:windowssystem32redist.rsp

start -> run -> regedit
find key HKEY_LOCAL_MACHINESOFTWAREClassesInterface{0000054E-0000-0010-8000-00AA006D2EA4}
change (Default) REG_SZ from "_Command" to "Command25"

reboot

run the jet4sp6 upgrade

pray

I hope this may help some people. By the 'one-mississippi, two-mississippi' benchmark it reduced my speed issues from 'four-mississippi' to 'one-mississippi' whilst in room creation mode but who knows what it'll do on each individuals system.

Incidentally, MDAC 2.8 is now out, so I might check out if it does anything interesting to map speed later on today (assuming it installs under Windows XP and I don't have to work out how to manually install it again :P)

May the MDAC be with you.
-- Rainchild

Wizard Joined: 10 Oct 2000 Posts: 1551 Location: Australia

Ok I've installed MDAC 2.8 and Jet4.0 SP7 (as opposed to MDAC 2.7SP1 and Jet 4.0 SP6) and didn't notice any difference in speed between the two. The benefit of MDAC 2.8 I guess is you don't have to install 2.7SP1 manually, however as it's part of the Windows 2003 Server platform its still in development so maybe we'll see some benefits when 2.8SP1 comes out...?

Heh, and I also discovered that there's also a "MDAC 2.7 SP1 Refresh" version, with some updates above standard 2.7 SP1... I've installed this (again, manually since it doesn't work automatically under XP) and the only difference that I can see with the mdac checker is 'oledb32.dll' has been updated... no speed difference that I can tell.

Posted: Tue Aug 12, 2003 5:53 pm

Rainchild, thanks very much for the detailed instructions!!

However, before people rush out and try Rainchild's manual installation, please try using the Microsoft MDAC 2.8 and Jet4.0 SP7 files FIRST and see if they fix the speed problem or not.

To update to MDAC 2.8 and Jet4.0 SP7 on Windows XP, please use the Windows Updater program and follow the instructions. Install MDAC first, and then the Jet update. Report back here as to whether this improves the room creation speed.

Wanderer Joined: 09 Nov 2000 Posts: 66 Location: USA

I installed MDAC 2.8 - There was no change in mapping speed.
I then (re)installed jet4.0 SP7 and there was still no change.

By the way, the delay that occurs when I create a room, also occurs when I'm not creating the room but just moving into it. Once I close zmud and open it back up again though, simply moving into those rooms is quick.

Does this mean anything to you zugg?

Posted: Thu Aug 21, 2003 6:09 pm

OK, let's go back to basics:

1) Run zMUD and press the ESC key to open a blank, default MUD window
2) Click the Map button to open the mapper
3) Click the Map mode button within the mapper and click Cancel on the Config Wizard screen.
4) Turn on the Keyboard creation mode in the mapper menus
5) Start pressing direction keys on the keypad to create rooms in various directions.

So, in the above situation, are your new rooms created fast or slow?? If they are slow, then there is still a problem with MDAC/JET files on your system.

Next test:
1) Run zMUD and select your character icon for the MUD that has slow mapping.
2) Click the OFFLINE button to open the character without connecting to the MUD
3) Click the Map button if your mapper is not opened
4) Click the Map mode button in zMapper to enter mapping mode
5) Turn on the Keyboard creation option
6) Click on an existing room on the map and click the Set Position toolbar button (the blue circle)
7) Now start using the number pad on the keyboard to create new rooms.

If the first test was fast, and this second test is slow, then the problem is with the *.MDB map file itself. If you can post this file somewhere where I can download it I can try to recreate the problem.

If this test is also fast, but adding rooms when you are actually connected to the MUD is slow, then the problem is with some sort of triggers or scripts that you have running in the background while mapping that is slowing things down. Try turning off your triggers to see if that makes any difference.

Wanderer Joined: 09 Nov 2000 Posts: 66 Location: USA

The first was fast, and the second was slow.

I mailed support@zuggsoft.com with a place you could download my .mdb.

Posted: Fri Aug 22, 2003 7:24 pm

OK, given my flaky network connection it might take me a few days to get to this. I'll download the map file as soon as I can and test it here. I might also need your *.MUD settings and the ZFG map configuration.

Actually, there is one more test you can try to determine if the problem is just with the map file or is related to other settings:

1) Press ESC from the main screen to open a blank window
2) Click the Map button to open the mapper.
3) Now select File/Open in the mapper and tell it to load the MDB file that is slow
4) Go into Map mode, Keyboard creation, and start creating rooms with the keyboard.

This is the actual procedure that I will be using on my system to load your map and test it, so let me know if this is still slow on your system.

Wanderer Joined: 09 Nov 2000 Posts: 66 Location: USA

yeah, That's slow too. It seems just about the same as when I normally do keyboard creation.

Posted: Fri Aug 22, 2003 11:30 pm

I haven't seen the email with the download location. Of course, my email has been flaky, but could you send it again? Try sending it to sales@zuggsoft.com in case my spam filter did something with it.

Wanderer Joined: 09 Nov 2000 Posts: 66 Location: USA

Done.

Posted: Sat Aug 23, 2003 10:04 pm

OK, got the email. I'll try to download the file soon and let you know what I find.

SubAdmin Joined: 18 Nov 2001 Posts: 5182

Zugg, if you have time when testing the map sent by Megane please do some comparative tests similar to those parrotslave and I did here...
http://www.zuggsoft.com/forum/topic.asp?TOPIC_ID=12197&whichpage=2
As of the last post to that thread there were 3 people who took the time to test and reported noticeable differences in speed between zMapper and zMud. If the shared sections of code in zMapper haven't been updated and you find the same pattern then you may have a short cut to diagnosing the problem.

Newbie Joined: 24 Aug 2003 Posts: 1 Location: USA

MDAC checker is giving me a few errors and I'm not sure how to fix them. It may not be possible since I'm using Windows 2000. Anyways, I have tried your tests Zugg. The first one works quickly, making rooms instantly. The second test (offline mode) and the other test (loading MUD map) both are slow in creating new rooms. I will send you an email with the location where you can download my map file if you wish.

Posted: Mon Aug 25, 2003 9:24 am

Good news Megane! Adding rooms to your map is slow on my system here too!

I don't have my source profiler set up tonight so I haven't taken a look at the code. Mostly I'm here at 2am waiting for the cops that were trying to catch the idiots that tried to break into my house tonight.

Since I was just waiting around with my adrenaline pumping and our flaky cable modem decided to be working, I downloaded your map file and confirmed the speed issue.

Now, you *do* have a large map. Not really with just the number of rooms, but since so many of your zones seem to be wilderness areas with autoconnected rooms, you have a *ton* of exits. There are about 29,000 records in your Exit Table and about 10,000 rooms in your Room Table. I'd consider this a big database for a Microsoft Access database.

On the other hand, I found a relatively small zone and it was still small adding to that zone. So, it doesn't seem to the be speed of updating my internal cache for the zone since that was still small. It's possible that it's the INSERT itself for the large tables and updating the various indexes.

But, I should refrain from more speculation until I have time to run my code profiler to see what part of zMUD is taking all of the time.

Oh, and for comparison, on my 233Mhz Win2000 system, it takes >10 seconds to add a room, so it's definitely significant.

Posted: Mon Aug 25, 2003 9:44 am

WOOT!!!! Found it!!

Man, I would have never thought to look in this routine...

One of the things zMUD needs to do fairly routinely is determine the list of exits from the current room. Normally, it looks in my own internal cache, loops through the exits in the current zone and collects those that originate from the current room.

However, in just one of the last few versions, I added a check just in case we were trying to collect exits for a room NOT on the current level. For example, when someone tries to use one of the scripting commands to query the exits from a room in some zone that isn't even loaded.

So, the LoadExits routine now first checks to see if the room the exits are being queried for is in the current cache. If NOT, then the mapper makes a clone of the ExitTbl and does a Filter on it for (FromID=ID) to match the current room ID.

When creating a NEW room on the map, this LoadExits routine is called BEFORE the room is actually added to the cache! So, since the routine fails to locate the room in the cache, the Filter on the full ExitTbl dataset is done. This is slow. Probably slower than an actual SQL query of the dataset.

It should be fairly trivial to change the code so that when a new room is created, the RoomExits list is already defined (it's a new room! there are no exits until we add some!).

I'll add this to the bug list for the next version. If I can ever get these next versions of zExplorer and zMapper released, I'll consider doing a quick update to zMUD to fix this, the issue with MUDs that use LF instead of CR/LF, and some Pueblo/VT100 issues within the next couple of months.

Thanks for being persistent and for sending me your map file. In all of the other cases that people have sent me their map file I haven't been able to reproduce any speed problem. It's nice to know that there is a real speed issue for certain users hidden amoung the bigger problem with the MDAC versions.

Wanderer Joined: 09 Nov 2000 Posts: 66 Location: USA

It's great to hear that it was found. This has been so frustrating because it wasn't looking like anything was going to happen. I didn't want to move back to 98 just because of problems with the MDAC. Previous to 2.8, I probably had trouble with the MDAC on top of this.

Thanks Zugg, I'll be looking forward to the next version a lot.

Wizard Joined: 10 Oct 2000 Posts: 1551 Location: Australia

Ooh well that will be interesting to see if the "one-mississippi" that I experience will turn into a mere "one-m" or "o-" or something :)

It never hurts for things to be faster, harder, or scooter. [sorry, musical reference, couldn't help myself] :P

Posted: Tue Aug 26, 2003 7:08 pm

Heh, I hate to torture people but...

As you know, zMUD/zMapper/zExplorer share the same mapping code. So, in compiling the new version of zExplorer today, I fixed the problem with the LoadExits routine. Just out of curiosity, I also did a quick recompile of zMUD using the new routine.

Using the 3K.MDB map file that Megane has sent me, the time to create a new room via keyboard creation went from about 5 seconds down to (as Rainchild would say) "one-". And this is on my slow 233 Mhz machine.

So yes, people can expect a huge different in room creation speed, even when they thought it was fast enough before.

The reason this is cruel torture is that I'm committed to first release the new zExplorer followed by the recompiled zMapper this week. Before releasing a new zMUD I want to be able to fix some of the other zMUD issues regarding the handling of LF characters and some Pueblo emulation problems (and whatever else is lurking on my bug list). So, I don't want to release a quick version with just the mapper fix at this time. If my network connection continues to work ok and I have a decent programming week, I might be able to have the new zMUD within 2 weeks or so.

But hey, at least that's better than waiting several months, right? Smile

SubAdmin Joined: 18 Nov 2001 Posts: 5182

I am glad this topic and all related ones can finally be put to bed. Two weeks or a month doesn't matter; what does matter is that the problem was found, and will be eliminated for the next version. Hopefully this will bring the speed up enough on XP systems that those users won't notice how that MS didn't do as good a job implementing MDAC/JET there.

Wizard Joined: 10 Oct 2000 Posts: 1551 Location: Australia

Beginner Joined: 21 Oct 2000 Posts: 19 Location: USA

Ah I am very excited about this. Looking forward to the patch.[8D]

Enchanter Joined: 10 Oct 2000 Posts: 698

How about this? I was for the last two weeks mapping at about 4-5 mississippi per room on this new computer with XP loaded. Today, I loaded the DB in Access and tried the 'compact and repair database' just like I have done several other times with no noticable results.

Anyway, today for no reason I can think of, my map lost almost 4k in size and is now at 9444k. Why did it all of a sudden decide to get really small, when other times, it dropped in size very little (<1k) after running the compact and repair from MSAccess?

I'm now mapping at 1, no mississippi. Very weird! I have the whole MSOffice suite open(each app) plus surfing the web while mapping. I can't slow it down now, not that I want to.

P4 2.6 wHT
512 M
120 GHD
Jet4 - latest
MDAC - 2.8?
zmud 6.62

Posted: Mon Sep 01, 2003 8:04 pm

Ahh, the mysteries of Microsoft MDAC databases. I've never seen something like that. Maybe it finally decided to rebuild the indexes for your map or something like that which would speed it up. I know that MDAC is set up to compact databases itself now and then depending upon the empty space in the database file, but the compact utility should have taken care of that. But my guess is that it's more related to the indexes. I don't think the compact utility rebuilds those.