I can't seem to use the search fuction

Originally posted by Spark
The database server is less than 10 months old and has dual processors with over a gig of RAM. We run a seperate webserver that more than handle's it's own load with ease - but the db server is getting killed. I think it's more configuration than anything else - but I'm no guru. Gabe has generously offered his company's technical knowhow to get this resolved, and failing that, his monetary resources to get things going in the right direction. This offer is very much appreciated by me (and hopefully everyone else).
Click to expand...

That is odd then. I know the larger a database can get, the more work the server has to do, but a dual CPU (not very old), with that much RAM should more than handle it. bladeforums.com on its own on the server? Or other sites too? Our servers are getting upgraded currently, but some of our current servers are P2 450 CPUs with 100+ websites functioning on them. Are you in a Windows hosting environment? (that could cause some issues)

OTF
 
Originally posted by Spark
The database server is less than 10 months old and has dual processors with over a gig of RAM. We run a seperate webserver that more than handle's it's own load with ease - but the db server is getting killed. I think it's more configuration than anything else - but I'm no guru. Gabe has generously offered his company's technical knowhow to get this resolved, and failing that, his monetary resources to get things going in the right direction. This offer is very much appreciated by me (and hopefully everyone else).
Click to expand...

*sigh* Well, I just lost my first reply, so this one's going to be hellishly short and not-so-sweet:

Find a database administrator with more than 50% brain function. MySQL on that kind of server should die of boredom running this forum.

And maybe you didn't call basic members freeloaders, but if monetary contributions are the only worthwhile contributions, then that's basically what you're saying. And sure, I may not be one of the most prolific posters over in the Balisong forum, but on the other hand, I donated free hosting and bandwidth for what is now over 1.25 gigs of bali videos.

Instead of suddenly disabling things, you probably could have asked for help. With people like Gabe, or myself, on the forums, it would've been a much better fix. Not only for the basic users, but for the servers.

-- Sean
Systems Administrator, Programmer, Database Administrator...
 
Yes, because we haven't asked for people to step forward and support the site for the last few years or anything. It's only when I make it extremely clear that we're having problems by disabling features for non-paying members that suddenly people volunteer their services.

Instead of criticizing my decisions, how about you try understanding just what I'm faced with on a daily basis. My first priority is to take care of my paying customers. We pass several gigs of bandwidth a DAY, let alone a month. Our server load is high because a lot of people visit the site daily, and mining our servers for information is very taking when the database is serveral gB in size.

Instead of criticizing me for making sure the paying customers are taken care of, how about you volunteer to lend a hand first? Contributing to conversations & knowledge, while making for a great community, don't exactly help pay the bills. All the 5+ years of knowledge stored here are useless if I can't afford to keep the site open to begin with.

This site does not make money hand over fist. I am not a millionare who can afford to toss hardware at problems. I recognize my own limitations, and have to trust in the people who are my tech support - and when they can't fix things, it leaves me having to make tough choices. When the same issues come up over and over and over again year after year, I have few solutions I can implement.

Hopefully the new software will come out shortly and ease many of the problems. Until then, given a choice between closing the forums to all basic members, and shutting of searchs, I know which choice I'm going to make.
 
This is background information for people.

I spoke to Jacob Wilkins at Nplus1, the hosting service Kevin uses for BladeForums.

There are two physical servers running BladeForums - one to handle the web front-end and the other running the database.

The two machines are both dual-1.2 GHz P3s running a patched version of Redhat 7.3. They are running PHP version 4.3.1, and MYSQL version 3.23.58. They have 1 GB of RAM, and are running RAID 5 with an Adaptec 2000 SCSI controller.

Jacob didn't gather load data prior to Kevin turning searching off, so all performance numbers are for the current setup.

The database is currently 3 GB. There are 78 database threads with about 60 queries per second. The "load" (a proxy for memory usage, CPU usage, ...) is currently .13. The CPU usage is about 40%.

The machine is connected to an OC-3 line via a 100 Mbit switch. Peak bandwidth consumption is about 1 Mbit/second.

Jacob feels comfortable that they have done a decent job of configuring the mysql server. He doesn't feel like they have any expertise at configuring the overall vBulletin configuration. I asked him if they were running PHP script caching (the first optimization listed on the vBulletin support pages) and he said he didn't know what it was and that they weren't running it (in other words mysql is probably reasonably configured and nothing has been done to globally tune performance).

I'll keep people posted.
 
Originally posted by Gabe Newell
This is background information for people.

I spoke to Jacob Wilkins at Nplus1, the hosting service Kevin uses for BladeForums.

There are two physical servers running BladeForums - one to handle the web front-end and the other running the database.

The two machines are both dual-1.2 GHz P3s running a patched version of Redhat 7.3. They are running PHP version 4.3.1, and MYSQL version 3.23.58. They have 1 GB of RAM, and are running RAID 5 with an Adaptec 2000 SCSI controller.

Jacob didn't gather load data prior to Kevin turning searching off, so all performance numbers are for the current setup.

The database is currently 3 GB. There are 78 database threads with about 60 queries per second. The "load" (a proxy for memory usage, CPU usage, ...) is currently .13. The CPU usage is about 40%.

The machine is connected to an OC-3 line via a 100 Mbit switch. Peak bandwidth consumption is about 1 Mbit/second.

Jacob feels comfortable that they have done a decent job of configuring the mysql server. He doesn't feel like they have any expertise at configuring the overall vBulletin configuration. I asked him if they were running PHP script caching (the first optimization listed on the vBulletin support pages) and he said he didn't know what it was and that they weren't running it (in other words mysql is probably reasonably configured and nothing has been done to globally tune performance).

I'll keep people posted.
Click to expand...

All stats there should be more than enough. 3Gb database is probably the source of the issue. I'd say try to cut down the database, maybe that may involve pruning/eliminating posts from X number of months, or anything older than a year. (if this is not currently being done)

OTF
 
Originally posted by Spark
Yes, because we haven't asked for people to step forward and support the site for the last few years or anything. It's only when I make it extremely clear that we're having problems by disabling features for non-paying members that suddenly people volunteer their services.
Click to expand...

I see you're leaving for SHOT wednesday night. There's an announcement for it.

If the server has been having so many problems in the past few days/week that you deemed it necessary to disable the search function, where is the announcement describing that problem? Where's the announcement saying "Sorry, but the servers are about to die, we have to disable the search function"?


Instead of criticizing me for making sure the paying customers are taken care of, how about you volunteer to lend a hand first? Contributing to conversations & knowledge, while making for a great community, don't exactly help pay the bills. All the 5+ years of knowledge stored here are useless if I can't afford to keep the site open to begin with.
Click to expand...

I'd be happy to volunteer to lend a hand, if it helps the basic customers.


This site does not make money hand over fist. I am not a millionare who can afford to toss hardware at problems. I recognize my own limitations, and have to trust in the people who are my tech support - and when they can't fix things, it leaves me having to make tough choices. When the same issues come up over and over and over again year after year, I have few solutions I can implement.
Click to expand...

It's not about having money. And something like this isn't going to make money. It's about finding people who love knives and who have something to offer.
 
Pruning threads is not an option. The huge db size is part of the problem - a great portion of it will be solved with vB 3.x when the attachments are removed from the forums db and turned back into standard images. That will greatly lessen the load.
 
Gabe, I just want to say thanks. You backed up your words and personally I'm impressed. Spark, keep doing what you're doing, you've got no complaints from this corner. I feel my $30 the last two years have been well spent, I just wish more people would do the same.
 
I don't know enough about computers to help one iota in that regard, so I will continue to support the forum the only way I can. Thirty bucks is diddly squat for what I get from this community.

BladeForums needs all of its members, but it does need those that feel that $2.50 a month is a deal for they get from BladeForums to step up and kick in financially.

Gabe, though I have nothing to do with the running of this forum I would sure like to thank you for your very generous offer of support. You really do take the cake.

Thanks to everyone else offering their support as well. What is needed here is a way to figure out how to keep things running as smoothly as possible for all the members and for the fella that runs things as well.
 
Originally posted by Gabe Newell

The two machines are both dual-1.2 GHz P3s running a patched version of Redhat 7.3. They are running PHP version 4.3.1, and MYSQL version 3.23.58. They have 1 GB of RAM, and are running RAID 5 with an Adaptec 2000 SCSI controller.
Click to expand...

They should seriously consider upgrading to MySQL v4.
 
Agreed on MySQL v4 upgrade.

Perhaps also limiting hosted pictures to 600x500 pix. (If it isn’t already)

Just a thought.


I'd also like to voice my personal “thank you” to Spark and Gabe for their continued support, on so many levels, to the community.

What I think needs to be understood by everyone else is that the hindrance here is revolving around technical issues and not prejudice. Let’s attack the problem and not the people.
 
My company runs a vBulletin forum (www.steampowered.com). We were looking at comparison numbers just to understand the load issues.

On a single PC we're handling about 600 queries per second and generating 3 Mbits/sec of forum traffic.

This is good news.

Alfred Reynolds, one of our engineers, is going to help out on this, so if you see posts by him, please make him feel welcome.
 
Code: 
-----Original Message-----
From: Jacob Robert Wilkins [mailto:jrw@nplus1.net] 
Sent: Wednesday, February 11, 2004 1:45 PM
To: Gabe Newell
Cc: [email]spark@1sks.com[/email]
Subject: bladeforums diagnostics...

I turned searhing back on. Running vmstat with 2 second intervals on the
box, I did a search for the word "lock"

"bi" is the interesting field here. It represents the number of blocks
read from the disk.

[root@bladedb root]# vmstat 2
   procs                      memory    swap          io     system         cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy  id
 1  0  0   1216  35620    216 797176   0   0     4     0    3     5   4   3   4
 0  0  0   1216  35620    216 797176   0   0     0     0  187    96   1   0  99
 0  0  0   1216  35584    216 797192   0   0     8     0  269   210   2   6  92
 1  0  0   1216  23356    232 798596   0   0   702    18  433   294   4   5  91
 0  0  0   1216  33472    232 798660   0   0    32     0  266   205  47   6  48

# boom, search starts here

 0  1  0   1216  20028    232 804500   0   0  2920     0  495   512   9   7  84
 0  2  0   1216   9084    232 811848   0   0 16888     0  572   516   5   5  90
 1  1  0   1216   7356    232 817848   0   0 17034     0  458   356   3   7  90
 0  1  0   1216   6136    232 821896   0   0 10788    64  496   358   3   5  92
 1  1  0   1216   6440    232 805328   0   0  8398     0  502   299  47   9  44
 0  1  0   1216   7292    232 817956   0   0 12114    64  569   504   3  10  86
 1  0  0   1216   9044    232 815708   0   0 10836     0  613   536   4   7  89
 0  1  0   1216   7196    232 821632   0   0  9702     0  631   542   4   2  94
 1  1  0   1216   8948    232 815844   0   0  9622     0  720   711   3   9  87
 0  1  0   1216   7304    232 821564   0   0  9864     0  652   595   5   4  91
 1  1  0   1216   7360    232 821464   0   0 12218    64  590   530   3   8  89
 0  1  0   1216   9240    232 819496   0   0  9018     0  599   558   3   4  93
 0  1  0   1216   9880    232 814388   0   0 11224     0  519   431   3   4  92
 1  1  0   1216  13404    232 815164   0   0  5624    64  666   776  10  10  79
 0  1  0   1216   6364    232 819716   0   0 10420     0  438  1138  10   3  87
 1  1  0   1216   9984    232 810552   0   0 14544   256  536   493   8  13  79
 0  1  0   1216   7468    232 815812   0   0 10162     0  386   362   4   5  91
 0  1  0   1216   6620    232 816348   0   0 13432     0  444   386   3   2  95
 0  1  0   1216   6336    232 816604   0   0 13474     0  546   489   4  13  84
 0  1  0   1216   7444    232 815820   0   0 15978     0  634   584   3   6  91
 0  1  0   1216   7416    232 815364   0   0 14144     0  537   482   4   5  91
 0  1  0   1216   9956    232 811228   0   0 17188     0  513   428   2   9  89
 0  1  0   1216   7744    232 818208   0   0 12090     0  594   564   6   4  91
 0  1  0   1216   6648    200 816064   0   0 10494    64  531   526   4  11  86
 1  1  0   1216   7392    200 815332   0   0 15122     0  529   439   5   3  92
 0  1  0   1216   6608    200 818672   0   0 10776     0  695   707   7   8  85
 0  0  0   1216   8972    200 813784   0   0  7388     0  533   625   4  11  85

# search is over


 0  0  0   1216   8972    200 813796   0   0     6     0  306   245   2   1  97
 1  0  0   1216   9960    200 813176   0   0   374    64  374   327   5   8  87
 0  1  0   1216   9824    200 813404   0   0   114     0  277   234   1   2  96
 0  1  0   1216  10056    200 812468   0   0   172     0  360   302   4   4  92
 0  0  0   1216  10056    200 812492   0   0    12     0  238   169   1   6  93
 0  0  0   1216  10056    200 812500   0   0     4     0  236   169   2   1  97
 0  0  0   1216  10004    200 812528   0   0    14    64  199   107   1   2  98
 0  0  0   1216   9736    200 812604   0   0    38     0  320   280   4   7  90
 0  0  0   1216  11492    200 811016   0   0   530     0  399   496   2   6  92


All while the system was running this query, CPU utilization was low.
Both processors were averaging about 80% idle. More CPU time was spent
in system than in user.

Memory on the box is fairly full, but not swapping.

[root@bladedb root]# free -m
             total       used       free     shared    buffers     cached
Mem:          1009        984         25          0          0        784
-/+ buffers/cache:        198        810
Swap:         2047          1       2046

It looks to me like we are I/O, and a little low in memory.

The processor is willing, but the disks can feed the data fast enough. Adding
memory would help with caching, but that'll only get you so far.

jrw
 
I upgraded to Gold today, simply because I like to come here , click on "view new posts", and see what's up, read some funny stuff, hear some latest news (usually beats my local paper by a couple of days, HAHA ), and luckily I have been able to purchase some knives that are hard to find at a fair price. Spark said today that I needed to pay if I wanted to continue to use this search feature. Cool, no problem. too much hastle for me to peruse each individual forum to see what I usually enjoy using the "view new posts" feature.
So be it, But I will somehow feel violated if this feature is returned to the non paying members due to a bunch of griping, now that I am a paying member. Don't get me wrong, I will not expect a refund, just doesn't seem right to ask the few people that step up and upgrade to not feel funny about this if in a month or two the search feature is back available to anybody. Stick to your guns Spark, I showed my support by putting my wallet where my mouth is.
 
This is such a complex problem. For me the computer deficient guy.
But pretty easy for the Guru's.
Thanks to all who are working to get this tweaked and optimized. That's Spark, Gabe and who knows all.
Tom
 
Spark and Gabe, great job working together guys. :D

I WILL be purchasing a Knifemaker membership this weekend (sold a knife :D ) I keep pushing "view new posts" out of habit.

I don't have a dang clue as to what y'all are talking about. Does it have something to do with the pictures and words on the boxy thingy on top of the thingy with the letters? :confused:
 
I have just purchased the Gold Membership and will upgrade to Knifemaker later. It's upsetting to see some of the "griping" going on. As I see it basic membership is like being invited into someones home and being able to enjoy the fellowship of those present. It seems rude to me to gripe about about the "rules" the owner sets down. Let's all get behind Spark and build a greater forum. If I hadn't 'stumbled' across the thread I wouldn't have known about the problems and continued along my merry way. At least now I know I have contributed and feel more of a 'forumite' then before. It's up to each individual to decide which way to go but let's not abuse the privalege of basic membership with 'gripes' which don't help.
 
You guys need to stop bitching about the bitching. As you can see, the kind of expertise that the bitching turned up is worth a helluva lot more than a few people paying $2.50 a month.
More money is not always the solution to problems.
 
You must log in or register to reply here.
Back
Top